This demo
provides an indepth view of how a policy can be defined and enforced
within the Overlord Business Activity Monitoring (BAM) infrastructure.
Monday, December 3, 2012
Define and Enforce a Policy
Saturday, December 1, 2012
Call Trace Capability
This video provides a preview of the call trace capability that will be available shortly within milestone 3 of the Overlord Business Activity Monitoring (BAM) project.
This facility enables users to understand how the business transaction instance executed, as well as the duration it spent within each call, thus allowing performance bottlenecks to be identified.
Tuesday, November 6, 2012
Service Dependency Graph and Policy Enforcement demo in BAM milestone 2
The Overlord team are pleased to announce the second milestone of the new Business Activity Monitoring (BAM) sub-project. A webcast showing some of the new features can be found here: https://vimeo.com/51539146
The new features include:
1) Definition of a service interaction (or dependency) graph, showing colour coding based on a configurable algorithm. The user can hover over services, operations and links to view statistics, and also hover over the situation markers to see a summary of situations that are affecting the specific service or operation. The service overview information represents a rolling window based on recent activity, and therefore reflects an up-to-date view of the activity associated with the system being observed.
2) Situations gadget and associated infrastructure. It is now possible for a user to create rules to detect 'situations of interest'. These situations will then be managed by the BAM infrastructure, providing the details through a REST service for display in the 'situations' gadget, as well as being reported via JMX. A "SLA Violation" example demonstrates how this mechanism can be used.
3) Service definitions active map (available via a REST service) - provides information about the available services and their operations, as well as metrics.
4) Policy enforcement demo, showing how the analysis of activity information can be used to apply business policies, defined outside the scope of the application, which in this case enables further purchases by a customer to be suspended until they make a payment.
5) Many internal improvements to help support the functionality described above.
If you have an interest in business activity monitoring capabilities for your project, then we would love to hear from you! Our user forum is located here, or alternatively please raise feature requests in our jira system.
The new features include:
1) Definition of a service interaction (or dependency) graph, showing colour coding based on a configurable algorithm. The user can hover over services, operations and links to view statistics, and also hover over the situation markers to see a summary of situations that are affecting the specific service or operation. The service overview information represents a rolling window based on recent activity, and therefore reflects an up-to-date view of the activity associated with the system being observed.
2) Situations gadget and associated infrastructure. It is now possible for a user to create rules to detect 'situations of interest'. These situations will then be managed by the BAM infrastructure, providing the details through a REST service for display in the 'situations' gadget, as well as being reported via JMX. A "SLA Violation" example demonstrates how this mechanism can be used.
3) Service definitions active map (available via a REST service) - provides information about the available services and their operations, as well as metrics.
4) Policy enforcement demo, showing how the analysis of activity information can be used to apply business policies, defined outside the scope of the application, which in this case enables further purchases by a customer to be suspended until they make a payment.
5) Many internal improvements to help support the functionality described above.
If you have an interest in business activity monitoring capabilities for your project, then we would love to hear from you! Our user forum is located here, or alternatively please raise feature requests in our jira system.
Tuesday, October 23, 2012
Open Group SOA Governance Framework Becomes an International Standard
The Open Group's SOA Governance Framework has now become an ISO standard: http://blog.opengroup.org/2012/10/12/the-open-group-soa-governance-framework-becomes-an-international-standard/
The approach described in this framework is interesting as it provides a reference governance regime, as well as a process that can be used by an organisation to tailor the framework to their own needs.
As described in most literature on the subject, governance is not just about technology, but also about people and processes. However, being a technology project, our main interest is in delivering the capabilities to support such a general framework.
Much of this functionality is currently underway, but if this governance framework is of interest to you or your organisation, then please consider joining our effort - whether contributing ideas (i.e. feature requests), evaluating what we have developed to provide constructive feedback, or developing code and/or documentation.
The approach described in this framework is interesting as it provides a reference governance regime, as well as a process that can be used by an organisation to tailor the framework to their own needs.
As described in most literature on the subject, governance is not just about technology, but also about people and processes. However, being a technology project, our main interest is in delivering the capabilities to support such a general framework.
Much of this functionality is currently underway, but if this governance framework is of interest to you or your organisation, then please consider joining our effort - whether contributing ideas (i.e. feature requests), evaluating what we have developed to provide constructive feedback, or developing code and/or documentation.
Thursday, October 4, 2012
S-RAMP: A SOA Repository
There is a new standard emerging from OASIS called S-RAMP. S-RAMP is short for SOA - Repository Artifact Model and Protocol.
In this post I'll talk about what the heck S-RAMP is and why it's important to JBoss Overlord.
And at the end, I'll tell you about the JBoss Overlord S-RAMP project and how we just did a preview release (Milestone 2).
I've been asked a number of times why S-RAMP might be used instead of some other content repository (JCR, SharePoint, Drupal, RDBMS, etc). I think the answer to that is in the "SOA-aware" idea. Because S-RAMP defines some common SOA models, and understands those models, it can provide some very useful functionality specific to the SOA domain.
I would encourage you to read the S-RAMP specification for more in-depth information. However, just to give you an idea, once you have stored your SOA artifacts (WSDLs, Schemas, etc) in an S-RAMP repository, you can ask it questions like this:
Between the pre-defined SOA models and the ability for businesses to define their own relationships, properties, and categories, S-RAMP provides a very nice way to store and index SOA content.
For instance, a typical use-case is how to progress a new service from the development phase through to production. One way that can be done is by deploying the service into the S-RAMP repository, and then triggering a Service Lifecycle workflow that will guide the service through the business-specific phases needed to get it into production. This might mean deploying the service into a staging environment, gathering approvals from stakeholders, performing automatic impact analysis, and ultimately deploying into production.
Still interested? Take a look at the following Milestone 2 screencast (I recommend clicking through and watching the HD version, but they're your eyes):
S-RAMP Milestone 2 Screencast from Eric Wittmann on Vimeo.
In this post I'll talk about what the heck S-RAMP is and why it's important to JBoss Overlord.
And at the end, I'll tell you about the JBoss Overlord S-RAMP project and how we just did a preview release (Milestone 2).
What is it?
In short, S-RAMP is a SOA-aware content repository. You can think of it as a repository to store SOA artifacts and then get them back out again in interesting ways.I've been asked a number of times why S-RAMP might be used instead of some other content repository (JCR, SharePoint, Drupal, RDBMS, etc). I think the answer to that is in the "SOA-aware" idea. Because S-RAMP defines some common SOA models, and understands those models, it can provide some very useful functionality specific to the SOA domain.
I would encourage you to read the S-RAMP specification for more in-depth information. However, just to give you an idea, once you have stored your SOA artifacts (WSDLs, Schemas, etc) in an S-RAMP repository, you can ask it questions like this:
- Which WSDLs utilize (import) the "xyz.xsd" XML schema?
- Which XML Schemas import or include the "xyz.xsd" XML schema?
- Which of my Service Instances are in production?
- Which Service Instances are governed by SLA "abc"?
- Which Service Instances are available in North America?
Between the pre-defined SOA models and the ability for businesses to define their own relationships, properties, and categories, S-RAMP provides a very nice way to store and index SOA content.
Why is it important?
JBoss Overlord has decided that an S-RAMP repository will be important for our efforts around SOA Governance. SOA Governance is a broad topic, and it means different things to different people. JBoss Overlord will be introducing solutions to various segments of the SOA Governance space. An important piece of that puzzle is how to govern SOA artifacts. We want to provide users the ability to manage their SOA assets in a central repository, which surrounding that core SOA-aware repository with customizable governance workflows.For instance, a typical use-case is how to progress a new service from the development phase through to production. One way that can be done is by deploying the service into the S-RAMP repository, and then triggering a Service Lifecycle workflow that will guide the service through the business-specific phases needed to get it into production. This might mean deploying the service into a staging environment, gathering approvals from stakeholders, performing automatic impact analysis, and ultimately deploying into production.
Where are we now?
We have, just today, released a preview version of the JBoss Overlord S-RAMP project. This release includes the core repository and a simple web-based UI to browse the repository. In addition, this preview release includes integration with the Maven build framework (to make it easy to deploy Maven projects into the repository).Still interested? Take a look at the following Milestone 2 screencast (I recommend clicking through and watching the HD version, but they're your eyes):
S-RAMP Milestone 2 Screencast from Eric Wittmann on Vimeo.
Some useful links.
- Milestone 2 screencast transcript
- S-RAMP @ OASIS
- Join the conversation (JBoss Overlord)
- Want to contribute? (come and introduce yourself!)
Monday, July 16, 2012
Introducing Overlord BAM and Gadget Server
This post introduces the first milestone release of the new Overlord Business Activity Monitoring (BAM) and Gadget Server components. The Gadget Server can be used independently of the BAM component, and will have its own distribution, but for the purpose of this post we will be focusing on the BAM component (and distribution), which includes the Gadget Server.
The aim of this release is primarily to focus on the infrastructure for both components, rather than higher level features, but there is a small demo to illustrate some of the capabilities, which will be discussed later in the post.
Therefore, although this component is optional, it provides the ability to integrate a collection mechanism into an application or container, which is designed to minimise the impact on the application, and deliver the activity details to the Activity Server in the most efficient manner. A batching approach is used based on maximum time interval and size - both of these values can be adjusted via a MBean.
Collecting activity information in this manner can also provide an implicit correlation between tasks that are carried out on behalf of a business transaction, minimising any post-processing that maybe required to recover this correlation if the events had been independently reported.
If the executing services are running in a remote location from the Activity Server, then one of the appropriate bindings will be used to deliver the information to the server - currently only REST is supported. However, if the services (and thus collector) are co-located with the Activity Server, then the information is reported directly.
When activity information is being stored, the Activity Server performs the following tasks:
Each network defines a graph of nodes connected by links that transfer the results from the source node to the target node. The graphs can subscribe to event subjects, to identify the information they are interested in, as well as the node(s) within the network that will process the information received on that subject. The nodes can also publish their results to event subjects, for other networks to further process.
Each node defines an optional predicate, that can be used to determine whether the event is of interest, and an event processor to perform the actual task. An example of an "out of the box" event processor is one used to trigger CEP rules (using Drools Fusion) to process the events.
The nodes can also be configured to generate different levels of notification, which can be used by other applications/components to observe the information being processed through the network (e.g. events that passed the predicate evaluation, results from the event processor, etc.).
When an EPN is registered, it will become manageable via a JMX MBean:
The Event Processor Networks are also versioned, so that networks can easily be upgraded without impacting events being processed by a previous version. Only new events submitted to the network will be processed by the newer version.
The information within a particular Active Collection is managed by an Active Collection Source, which effectively acts as an adapter between the actual source of the information and the Active Collection.
For example, an "out of the box" implementation of an Active Collection Source is provided to observe different types of information produced by an EPN node. This implementation also includes the ability to aggregate information to store a summary within the Active Collection - this mechanism has been used with the service response time information in the demo to be described later in the post.
As well as the information within an Active Collection being maintained by an Active Collection Source, the configuration of the source can define certain constraints on the Active Collection - e.g. maximum number of items, item expiration duration, etc. These provide a means to keep the Active Collections a managable size.
When an Active Collection Source is registered, the associated Active Collection becomes manageable via a JMX MBean:
A range of gadgets will be implemented to enable users to access information provided by the BAM system. However it will also be possible for users to upload their own gadgets into the server and used along side the Overlord provided gadgets.
Unpack both in suitable locations.
Open a command window in the root folder of the Overlord BAM distribution and set the JBOSS_HOME environment variable to point to the root folder of the Switchyard installation. Then run the following command:
mvn install
(To uninstall, run the "mvn clean" command from the same folder).
Change directory to the samples/slamonitor folder, and then run:
mvn install
This will automatically install the four war files associated with the sample into the Switchyard deployments folder.
./standalone.sh --server-config=standalone-full.xml
(Full config required as JMS is used).
You will be presented by a login screen:
Currently users are not persisted, so each time the server is started (or on first access) you will need to press the "Signup" button:
Fill in some temporary details and press the "Submit" button. This will display an initially blank tab - however when a user's profile is persisted, this will show the profile previously configured by the user.
At the top right hand corner of the screen will be an "Widget Store" link which takes you to the list of available gadgets:
Press the "Add To Page" button for the Response Time and SLA Gadget entries, before pressing the "Back to Tabs" link in the top right hand corner.
mvn clean install -Dcount=1000
This will perform 1000 business transactions, a majority of which will be valid with the occasional transaction that will result in a SLA violation.
This will result in the response time information and service violation details being presented in the browser:
The gadgets are set to refresh every 30 seconds, although this polling interval can be changed as part of the gadget configuration. Future versions will experiment with using active pushing of the changes to the browser.
The Response Time gadget currently shows all response time information for any service and operation. In a future release it will be possible to filter the service and/or operation being viewed. The specific details regarding a particular point on the chart can be viewed by hovering over the point:
The SLA Violations can also be processed as JMX notifications (using JConsole):
The source configuration for the two Active Collections has some configured constraints:
The collection is also constrained to define a maximum number of 100 items. As the collection cleanup cycle runs periodically, this maximum threshold is not strictly enforced, but provides the level at which cleanup will be triggered. So you will notice that during the test, the number of items will exceed 100, but when the test stops, eventually the final cleanup will truncate the list size to 100.
When deploying the SLA Monitor samples above, one of the war files (slamonitor-orders.war) is actually a Switchyard application that includes an 'exchange handler' that intercepts messages passed between services and reports them via the Activity Collector.
To demonstrate activities being reported from the Switchyard application you will need to startup your favourite SOAP client and send the following message:
To generate an SLA violation, simply change the itemId from BUTTER to JAM and then resend the message.
User Guide
Developers Guide
Discussions can be held on the Overlord User Forum.
Problems or feature requests can be reported here.
The aim of this release is primarily to focus on the infrastructure for both components, rather than higher level features, but there is a small demo to illustrate some of the capabilities, which will be discussed later in the post.
Overview of the architecture
Activity Collector
One of the goals of the BAM infrastructure is to support real-time activity collection and analysis. However when monitoring the execution of a distributed system, this can involve the collection of a large amount of information.Therefore, although this component is optional, it provides the ability to integrate a collection mechanism into an application or container, which is designed to minimise the impact on the application, and deliver the activity details to the Activity Server in the most efficient manner. A batching approach is used based on maximum time interval and size - both of these values can be adjusted via a MBean.
Collecting activity information in this manner can also provide an implicit correlation between tasks that are carried out on behalf of a business transaction, minimising any post-processing that maybe required to recover this correlation if the events had been independently reported.
If the executing services are running in a remote location from the Activity Server, then one of the appropriate bindings will be used to deliver the information to the server - currently only REST is supported. However, if the services (and thus collector) are co-located with the Activity Server, then the information is reported directly.
Activity Server
The Activity Server has two responsibilities, firstly enabling clients to record activity information and secondly to provide query methods to retrieve a subset of the information. The server will be accessible via a range of bindings, although currently only REST and the direct Java API are supported.When activity information is being stored, the Activity Server performs the following tasks:
- Pre-Process the activity information to assign unique ids
- Store the data in a persistent store
- Notify interested listeners about the activity information
Event Processor Network
The Event Processor Network (EPN) is a graph based mechanism for processing a series of events. In the context of the BAM infrastructure, one or more networks can be registered to receive the activity information (as notifications) from the Activity Server and process it (filter, transform, analyse, etc) using whatever means is appropriate.Each network defines a graph of nodes connected by links that transfer the results from the source node to the target node. The graphs can subscribe to event subjects, to identify the information they are interested in, as well as the node(s) within the network that will process the information received on that subject. The nodes can also publish their results to event subjects, for other networks to further process.
Each node defines an optional predicate, that can be used to determine whether the event is of interest, and an event processor to perform the actual task. An example of an "out of the box" event processor is one used to trigger CEP rules (using Drools Fusion) to process the events.
The nodes can also be configured to generate different levels of notification, which can be used by other applications/components to observe the information being processed through the network (e.g. events that passed the predicate evaluation, results from the event processor, etc.).
When an EPN is registered, it will become manageable via a JMX MBean:
The Event Processor Networks are also versioned, so that networks can easily be upgraded without impacting events being processed by a previous version. Only new events submitted to the network will be processed by the newer version.
Active Collections
The Active Collection mechanism is a variation on the standard collection concept, where interested parties can register interest in changes that occur to the contents of a collection (e.g. list, map, etc). This is one of the mechanisms that will be used to maintain information that is to be presented to users (e.g. via the Gadget Server).The information within a particular Active Collection is managed by an Active Collection Source, which effectively acts as an adapter between the actual source of the information and the Active Collection.
For example, an "out of the box" implementation of an Active Collection Source is provided to observe different types of information produced by an EPN node. This implementation also includes the ability to aggregate information to store a summary within the Active Collection - this mechanism has been used with the service response time information in the demo to be described later in the post.
As well as the information within an Active Collection being maintained by an Active Collection Source, the configuration of the source can define certain constraints on the Active Collection - e.g. maximum number of items, item expiration duration, etc. These provide a means to keep the Active Collections a managable size.
When an Active Collection Source is registered, the associated Active Collection becomes manageable via a JMX MBean:
Gadget Server
The Gadget Server provides a GWT implementation of the OpenSocial specification. This provides a visual container for gadgets, allowing users to select the gadgets they are interested in, and define the layout in which they wish to organise them.A range of gadgets will be implemented to enable users to access information provided by the BAM system. However it will also be possible for users to upload their own gadgets into the server and used along side the Overlord provided gadgets.
A simple demo
The first milestone includes a simple demo application, with an additional 'test business transaction' generator. This demo works with Switchyard 0.5.Final.Setup the environment
First step is to download Switchyard (0.5.Final or above) and the Overlord BAM distribution for JBossAS. Although Switchyard is being used for this demonstration, JBossAS7 can equally be used - the only dependency on Switchyard is the example application that is deployed.Unpack both in suitable locations.
Open a command window in the root folder of the Overlord BAM distribution and set the JBOSS_HOME environment variable to point to the root folder of the Switchyard installation. Then run the following command:
mvn install
(To uninstall, run the "mvn clean" command from the same folder).
Change directory to the samples/slamonitor folder, and then run:
mvn install
This will automatically install the four war files associated with the sample into the Switchyard deployments folder.
Startup the server
First start the Switchyard environment using the following command from the bin folder:./standalone.sh --server-config=standalone-full.xml
(Full config required as JMS is used).
Start the browser
Once the server has started, then opena browser at the URL: http://localhost:8080/gadget-webYou will be presented by a login screen:
Currently users are not persisted, so each time the server is started (or on first access) you will need to press the "Signup" button:
Fill in some temporary details and press the "Submit" button. This will display an initially blank tab - however when a user's profile is persisted, this will show the profile previously configured by the user.
At the top right hand corner of the screen will be an "Widget Store" link which takes you to the list of available gadgets:
Press the "Add To Page" button for the Response Time and SLA Gadget entries, before pressing the "Back to Tabs" link in the top right hand corner.
Run the activity client
Using the same command window, change directory to the samples/activityclient folder, and then run the following command to generate a set of business transactions:mvn clean install -Dcount=1000
This will perform 1000 business transactions, a majority of which will be valid with the occasional transaction that will result in a SLA violation.
This will result in the response time information and service violation details being presented in the browser:
The gadgets are set to refresh every 30 seconds, although this polling interval can be changed as part of the gadget configuration. Future versions will experiment with using active pushing of the changes to the browser.
The Response Time gadget currently shows all response time information for any service and operation. In a future release it will be possible to filter the service and/or operation being viewed. The specific details regarding a particular point on the chart can be viewed by hovering over the point:
The SLA Violations can also be processed as JMX notifications (using JConsole):
The source configuration for the two Active Collections has some configured constraints:
Response Time
This Active Collection represents a set of aggregated results. Every second an aggregated result is produced based on all of the response time information for a particular service type and operation. This aggregated result represents the average, minimum and maximum values.The collection is also constrained to define a maximum number of 100 items. As the collection cleanup cycle runs periodically, this maximum threshold is not strictly enforced, but provides the level at which cleanup will be triggered. So you will notice that during the test, the number of items will exceed 100, but when the test stops, eventually the final cleanup will truncate the list size to 100.
SLA Violations
The SLA Violations active collection has the constraint that each item will expire after 40 seconds. Therefore, once the activity client has finished generating test data, you will notice that the entries will eventually all be removed.Monitoring a Switchyard Application
The activity client provides the means to generate a large number of transactions quickly, to make it easier to demonstrate the UI. However it does not demonstrate collecting information from a real application.When deploying the SLA Monitor samples above, one of the war files (slamonitor-orders.war) is actually a Switchyard application that includes an 'exchange handler' that intercepts messages passed between services and reports them via the Activity Collector.
To demonstrate activities being reported from the Switchyard application you will need to startup your favourite SOAP client and send the following message:
to: http://localhost:8080/demo-orders/OrderService
<soap:envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:body>
<orders:submitorder xmlns:orders="urn:switchyard-quickstart-demo:orders:1.0">
<order>
<orderid>PO-19838-XYZ</orderid>
<itemid>BUTTER</itemid>
<quantity>400</quantity>
</order>
</orders:submitorder>
</soap:body>
To generate an SLA violation, simply change the itemId from BUTTER to JAM and then resend the message.
References
Further information can be obtained from:User Guide
Developers Guide
Discussions can be held on the Overlord User Forum.
Problems or feature requests can be reported here.
Overlord reloaded
For many years, the Overlord project has been used as a general landing page for information about SOA Governance related initiatives within JBoss, with links to other projects that provide some relevant functionality (e.g. ModeShape, Savara, etc).
However in recent months the role of Overlord has changed. It has now become a fully fledged project in its own right, as the umbrella project for a number of new components.
The current list of components are:
More information about each of these components will become available over the coming months.
However in recent months the role of Overlord has changed. It has now become a fully fledged project in its own right, as the umbrella project for a number of new components.
The current list of components are:
- S-RAMP - a reference implementation of the OASIS SOA Repository Artifact Model and Protocol
- S-RAMP UI - a SOA modeler/repository web based UI
- Gadget Server - a GWT based gadget container, conforming to the opensocial specifications
- BAM - business activity monitoring
More information about each of these components will become available over the coming months.
Subscribe to:
Posts (Atom)