SOA Governance Projects and Organizational Roles

In large organizations building and maintaining services is not a one man job, instead it is a process that touches many people in the organization. Figure 1 is a copy of figure 5.6 on page 95 of the 'SOA Governance' book by Thomas Erl et al.

Figure 1. Common associations of organizational roles with different SOA project stages.

With this many people and roles involved, how to you manage a project like this? Especially since in a large organization you will have many of these project running simultaneously. We would have loved to implement a full project workflow based on the figure 1 process, but two things are currently standing in our way: time and real world input. Who is going to pick up the challenge?

To get you started we created a 'Simplied Project Lifecycle Workflow' demo that implements just 3 boxes; Requirements gathering, Service Design and Service Implementation, with one role responsible for each stage, which are a Business Analyst, SOA Architect and SOA Developer. Each role can be fulfilled by more then one person. In the demo we assigned all three roles to the 'admin' user so we don't have to log in and out as different users all time. The demo uses the workflow shown in Figure 2. Each column represents one of the phases. The first phase being Business Analysis. The hope is this demo provides you with the building blocks to create the real world implementation we talked about earlier.

Figure 2. Simplified Project Lifecycle Workflow.

Some benefits of using this workflow are:

• Helps your organization with adoption of SOA by following proven processes. It is clear who is responsible for approval.
• Provides insight in where your projects are.
• Helps your organization work together in different teams.
• Audit features allow full history tracking.
• Released artifacts are in the repository, the artifact is automatically in escrow this way, and documentation and sources are stored along side the binaries all in one place.

Some optional benefits:

• It is possible to send a BPMN event at the end of the workflow (or from anywhere else), which can kick off a release workflow to automate deployment. Though one could also write a governance query looking for a service implementation artifact with classification #ImplPass.
• Easy integration with other systems (think bug track systems, or time management systems)

If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Enjoy

Customize Managed Deployments Workflow

DTGov ships with a number of BPMN2 based processes that you may want to update so they fit your business processes. In this demo we will update the 'Manage Deployments' workflow. It is recommended you go over the Managed Deployments usinf DTGov demo if you haven't already done so.

If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Managed Deployments using DTGov

DTGov ships with a workflow to perform deployments of released artifacts to the dev, qa, stage and prod runtime. Some benefits of using this workflow are:

• Releases artifacts are in the repository before they go to a runtime, the artifact is automatically in escrow this way, and the sources are stored along side the binaries.
• Automated deployments (using file copy, RHQ RestAPI, or JBoss Domain RestAPI).
• Email notifications.
• Sign off using Human Tasks in the workflow. Users and roles can be tailored to fit your organization. The sign off enhances the visibility of the where an artifact is in the deployment lifecycle.
• Audit features allow full history tracking of the artifacts.
• One repository for everything; so things can be found easily. This answers the "Where is my stuff" question. The XPath like query language of S-RAMP can used for searching.

If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Enjoy

Bleeding Edge Governance - Getting Started

We just completed an ant script that will help you get started with the latest Governance code in five easy steps and a bit of building time (your mileage may vary depending on the speed of your machine).

First download the latest script overlord.zip script (or if you prefer to use git: https://github.com/KurtStam/overlord)

Project Overlord consists of a repository (S-RAMP), Design Time Governance (DTGov) and Run Time Governance (RTGov). This script will build S-RAMP and DTGov. Both of these projects have a GUI component. All components share login information using SAML bearer tokens. The SSO code lives in a shared project called overlord-commons. Because of these SSO complexities we currently only support deploying to JBossEAP.

1. Check out the code from Governance on Github

This clones overlord-commons, s-ramp and dtgov from github. If you want to check out your own clones, then you can update the urls in the build.properties file.

ant clone

going forward you can update using ant pull.

2. Dowload JBossEAP-6.1

Navigate to http://www.jboss.org/jbossas/downloads, login and download JBoss-6.1.0.Final.zip to this directory.

3. Build

While your download is running you can kick off the build

ant install

4. Deploy to JBossEAP

When both 2 and 3 are finished you can deploy to JBossEAP by running

ant deploy

When ready start of EAP in standalone mode (jboss-eap-6.1/bin/startalone.sh) using a different shell.

5. Seed

You can now add the following section to your .m2/settings.xml

<server>
   <id>local-sramp-repo-snapshots</id>
   <username>admin</username>
   <password>your-admin-password</password>
</server>
<server>
   <id>local-sramp-repo-snapshots</id>
   <username>admin</username>
   <password>your-admin-password<</password>
</server>

Insert the admin password you supplied during step 4.

Make sure JBossEAP is fully booted up before you start the seeding

ant seed

This will upload the ontologies as well as the workflows. You can verify success using the s-ramp-ui. For now you will need to reboot JBossEAP to activate the workflows.

Good luck!

Please use the overlord user forum if you need help or have any other questions. We hope you will become part of our community and start writing your own extensions. Looking forward to seeing some good Git pull requests.

Cheers,

The Overlord team.

Other useful getting started links:

• http://www.jboss.org/overlord/
  
• https://github.com/Governance/s-ramp/wiki/GuideGettingStarted
  
• https://github.com/Governance/dtgov/wiki/GuideGettingStarted
  

Overlord 1.0.0.Final has been released

The Overlord team is proud to announce the release of version 1.0.0.Final. There are three major components within this release:

1) S-RAMP - SOA Repository Artifact Model and Protocol (based on the OASIS standard)
Downloads Documentation

• Artifact repository
  • Built on top of ModeShape
  • Stores artifact content and meta-data including:
    • Core properties (name, description, version, etc)
    • Custom ad-hoc properties (name-value)
    • Relationships between artifacts
    • Hierarchical tagging system called Classifiers
  • Rich XPath-like query language to search the repository
  • Atom based REST API
  • Automatic decomposition of artifacts based on type
    • Built in support for common S-RAMP types (XSD, WSDL, etc)
    • Extensible to support additional types (Teiid VDB, switchyard.xml, etc)
• Command Line Interface
  • Supports interactive and batch modes
  • Built-in commands to manage and search for artifacts in the repository
  • Extensible to allow contributing custom commands
• Maven integration
  • Can "mvn deploy" a deployment artifact directly into S-RAMP
  • Able to use S-RAMP artifacts as dependencies in maven projects
• Web UI
  • Based on Errai 2.4
  • Allows granular management of content in the repository
  • Can be used to manipulate all artifact meta-data

2) DTGov - Design Time Governance
Downloads Documentation

• Built on top of the S-RAMP artifact repository
• Supports triggering Governance Workflows based on repository changes
  • Embeds jBPM 6 to support workflow execution
  • Maps an S-RAMP query to a workflow definition
• Includes an out-of-the-box Governance Workflow to handle Deployment Lifecycle Management
• Implements several REST services to support Governance Workflows
  • General purpose Notification (email) Service
  • Artifact meta-data Update Service (used to add meta-data to the S-RAMP artifact being governed)
  • Deployment Service used to deploy binary artifacts into runtime EAP systems
    • Supports RHQ
    • Supports JBoss AS CLI
    • Supports direct file copying
    • Can also deploy into a Maven repository
    • Supports multiple, configurable logical deployment targets (e.g. DEV, TEST, PROD)
• Web UI
  • Includes a Task Inbox for human tasks created by the Governance Workflows
  • Provides a UI specific to the Deployment Lifecycle Governance Workflow
    • Can drill into a deployment to see its contents
    • Shows the change history for a deployment
    • Provides a way to see which deployments are deployed to which environments

3) RTGov - Run Time Governance
Downloads Documentation

• Activity Collector
  • Activity events can be reported directly to the Activity Server via REST service, OR
  • Embedded collector to intercept activities, pre-process the information to extract correlation details and business relevant properties, before batching events for distribution to the server.
    • Events can be 'validated' using pluggable event processors, which can be used to block business transactions
• Activity Server
  • Provides a REST interface to enable events to be reported
  • Persists activity information
  • Distributes reported events to subscribed components.
• Event Processor Network
  • Subscribes to the Activity Server, to receive reported activity events
  • Distributes activity events through a series of registered networks, that consist of event processors to validate, transform and retransmit events. Processors can be specified using Java, Drools CEP rules and MVEL.
• Maintaining Derived Results
  • Information derived by the Event Processor Network can be presented as a set of 'active' collections
  • Components can directly access these collections, or issue queries through a REST API.
• Web UI
  • Runtime governance information is provided through a set of gadgets.
    • Service Response Time Gadget - displays response time information per service and optionally operation.
    • Situations Gadget - table for displaying situations detected by the event processing (e.g. SLA violation)
    • Call Trace Gadget - provides a tree based visual display of the call stack associated with a business transaction
    • Service Overview - graphical representation of the services and their dependencies, showing response time and invocation details.

If you have any questions about the Overlord governance capabilities, please post to the User Forum. If you find any issues or would like to raise a feature request, then please access the Jira issue tracking system for the particular sub-component. Links to these are found on the Overlord website.

Runtime Governance release 1.0.0.M5

The Overlord team are pleased to announce the availability of version 1.0.0.M5 of the Runtime Governance (RTGov) sub-project.

This release adds the following capabilities:

• Collection of activity events from SwitchYard BPEL and BPM components
• Enable security on REST services
• Ability to enable/disable activity event collection based on a property, and switchable at runtime through JMX
• XSLT transformer for processing activity event content
• Information processor support for header values
• Event processor for sending emails
• Event processor for persisting events (e.g. situations)
• Added a new context type called Link, for correlating two activity events within a specified timeframe, where the correlation value may not be unique outside of that timeframe
• H2 database configured on installation
• Various bug fixes and minor changes - see release notes for more details.

This release is compatible with SwitchYard 1.0.0.Alpha1, and should be installed in JBoss EAP.

For more information, please see the release notes at:
https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12313521&version=12322009

Runtime Governance 1.0.0.M4 released

The Overlord team are pleased to announce the availability of version 1.0.0.M4 of the Runtime Governance (RTGov) sub-project. This sub-project was until recently called 'Business Activity Monitoring', and was renamed to reflect the wider scope of its capabilities.

This release adds the following capabilities:

• Inline (synchronous) policy enforcement. The previous version enabled policies to evaluate activity events "after the fact" to determine if a situation had occurred, and then provide some enforcement capabilities for future business transactions (e.g. to block a customer's purchase requests if their account had been suspended). This new capability enables event processors to be configured to immediately evaluate activity events and if necessary block a business transaction.

• For business transactions that have associated 'principal' information, this information will be recorded with the activity events are made available in derived results (e.g. situations and call trace).

• Location information, recorded as part of the activity events, is propagated to derived results (e.g. situations and call trace) to enable users to understand where problems have occurred.

• Runtime governance now works in a clustered JBossAS environment.

• Activity events stored in H2 in-memory db. Other databases will be supported in the next version.

• Lightweight activity collector client configuration. It is now possible to install just the activity collector mechanism within an execution environment, and have it report the activity events to another fully configured runtime governance server via REST.

• Various bug fixes and minor changes - see release notes for more details.

This release is compatible with SwitchYard 0.8.Final.

For more information, please see the release notes at: https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12313521&version=12321356