Tuesday, January 6, 2015

Overlord RTGov: 2.1.0.Beta1 Released!

We are pleased to announce the release of version 2.1.0.Beta1 of Overlord RTGov. This release includes:

  • Wildfly Support (currently 8.1 as SwitchYard does not support 8.2)
  • Migration of authentication/authorization and single sign-on (SSO) support from Overlord's own (Picketlink based) mechanism to KeyCloak
  • Individual distributions per platform (e.g. wildfly) and type (full installation or client only). This is intended to reduce the distribution size in preparation for providing docker images.
  • And many bug fixes.....

 Overlord Team.

Tuesday, December 9, 2014

Overlord S-RAMP: 0.7.0.Final Released!

Overlord S-RAMP 0.7.0.Final has just been released!  For the full details, please see the release notes in JIRA: https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12311420&version=12319965.

One important request: We're renaming Overlord S-RAMP!  In order to highlight the project as a viable standalone solution, as well as to better distinguish between the project and spec names, we're currently brainstorming new ideas.  If you have a suggestion, please add it to https://developer.jboss.org/thread/249956!

Another topic to highlight is the deprecation of Jetty, Fuse, and Fuse Fabric support.  It was decided that the amount of work necessary to support them was taking too much time away from new features and bug fixes.  Further, it's our understanding that none of them were used in the community.  Note that they have not been completely removed.  Rather, they're simply pulled out of the Maven model under SRAMP-620.  If you feel strongly about either of them and would like to see continued support, please let us know: https://community.jboss.org/en/overlord.  I am definitely not against re-enabling them if there is an actual need.

New Feature and Enhancement Highlights:

  • Overlord S-RAMP is now around 95% compliant with the OASIS S-RAMP spec.  Remaining tasks are tracked under SRAMP-462.  A notable task was full support for the SOA and ServiceImplementation logical models (SRAMP-167).
  • 100% spec compliance should be expected in the next release, which would most likely constitute a 1.0 release candidate.
  • SRAMP-609
    A new "ArtifactBuilder" extension contract was created, replacing the old "Deriver" and "Linker" concepts.  This provides a powerful means to automatically generate custom metadata, relationships, and derived content whenever relevant artifacts are deployed to S-RAMP.  A suite of built-in Builders are provided, covering most use cases.  However, developing your own is also supported. See http://docs.jboss.org/overlord/sramp/0.7.0.Final/html/_overlord_s_ramp_implementation.html#_extending_custom_artifactbuilder for more info.
  • SRAMP-580
    If an artifact is deployed to S-RAMP, without explicitly providing the artifact's model and type, the server now automatically attempts to detect it.  Another extension contract was added to support this: "ArtifactTypeDetector".  As with "ArtifactBuilder", S-RAMP also provides several Detectors OOTB, but custom implementations are possible.
  • SRAMP-531
    Previously, archive expansion (discovering artifacts within a jar/war/ear/zip) was handled by each individual client.  Instead, this was moved to the server.  So, regardless of the client, archives will be exploded and parsed in a consistent, automatic manner.
  • SRAMP-601
    The deleteContent action is now supported, allowing a document artifact's content to be completely deleted (the spec, as well as some use cases, require this).  See the next bullet for a discussion about constraints.
  • SRAMP-541
    The update, updateContent, delete, and deleteContent actions now have a set of restrictive constraints that must first be dealt with.  These include custom metadata, relationships, etc.  This guarantees data integrity, as well as furthering the "impact analysis" use cases.  Further, updateContent and deleteContent now automatically regenerate and delete all applicable derived content, respectively.  See https://developer.jboss.org/thread/250173 for more discussion and details.
  • SRAMP-466
    When deploying artifacts through S-RAMP's "batch" capability, the entire batch is now given priority when relationships are resolved, as opposed to first checking the existing artifacts in the repo.

Thursday, October 9, 2014

Overlord RTGov and Elasticsearch integration

A quick demo of RTGov and elasticsearch based on the order management and SLA quickstarts. The demo shows

  • Using Kibana to visually analyze the response time information from order management service and and internal components 
  • Identify a slow running component that may breach a SLA
  • Deploy a EPN to react on these SLA breaches and create situations
  • Query elasticsearch and drilling down into data that is of interest and building custom Kibana dashboards from the RTGov data

RTGov and Elasticsearch

Tuesday, October 7, 2014

Overlord RTGov: 2.0.0.Final Released!

Along with the recent S-RAMP and DTGov releases, we have also released a new major version of RTGov.

This version includes the following main changes:

  • One of the most significant changes is the replacement of the Gadget Server UI, with its RTGov gadgets (response times, call trace, situations, service overview) with a new UI. Although the gadget server approach was useful for displaying individual widgets, it did not work well for navigating between related information. The new UI has been designed to enable the user to navigate around related information. It is only the first version of the new UI, so more features will be added - and any feedback on how it can be improved is always welcome.
  •  Integration with Elasticsearch and Kibana. The previous version of RTGov stored activity information in a relational database. RTGov 2 now supports storage of this information in Elasticsearch (a NoSQL database), along with UI integration with Kibana, to provide analytic capabilities over the stored information.
  • Fuse support. RTGov, along with the other Overlord projects, can now be used within Fuse.

For further information on the new features and bug fixes included in this release, please see the release note.

Friday, October 3, 2014

Overlord DTGov: 1.4.0.Final Released!

After reading about the S-RAMP release I'm betting you're not shocked to learn that we also released a new version of Design Time Governance!  These two projects continue to be linked pretty tightly, but Hint that might be changing in the future....

For now, we hope you enjoy the latest 1.x version of Design Time Governance.  For full details, you can view the Release Notes.  If that's tl;dr then here are the highlights:

  • Full support for custom Deployers (including in the UI) [DTGOV-205]
  • View both deployed workflow definitions and running instances in the Workflows UI [DTGOV-209]
  • Better (full) support for the RHQ deployer [DTGOV-197]
  • Much better installation experience on Fuse!  [DTGOV-144]
  • Tons of bug fixes (of course)...
Download now and give it a whirl:




Overlord S-RAMP: 0.6.0.Final Released!

Overlord S-RAMP 0.6.0.Final has just been released!  For the full run-down, have a look at the release notes in JIRA: https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12311420&version=12319965.  Here are a few highlights:

Get it while it's hot!

Tuesday, September 9, 2014

Preview/Alpha Release of Overlord API Management


The Overlord team has been working on a new project ( https://github.com/Governance/apiman ) that provides rich API Management functionality.  The project is still in its early days, so any thoughts/feedback is even more appreciated than normal.

The project site is here:

  http://www.projectoverlord.io/apiman/

The quickstart (linked from the web site) can be run straight from the maven source and ultimately runs in WildFly 8.1.

If you're interested in runtime API Management governance capabilities, give us a look and join us in the conversation!

New website launch!

We're excited to announce the launch of the Overlord project's new website: http://www.projectoverlord.io  We'd sincerely appreciate your feedback or ideas!

Wednesday, July 2, 2014

Overlord 1.1.0.Beta1 Released

The Overlord team is pleased to announce the release of Overlord 1.1.0.Beta1, comprised of SRAMP 0.5.0.Beta1, DTGov 1.5.0.Beta1 and RTGov 2.0.0.Beta1. We will also be launching our new website shortly (http://governance.github.io/). Please take a look and let us know what you think.

The common features across the components are:
  • Support for EAP 6.3 and Fuse 6.1 (Fabric support to follow shortly)
  • Numerous bug fixes

 

SRAMP:

  • Maven facade (early preview)
  • Many doc improvements
  • Moved from jline to aesh
More details: http://red.ht/TBjjAl

 

DTGov:

  • Jetty 8 Support
  • New Admin UI to manage Workflow Triggers
  • Notification Service email templates can now come from S-RAMP
More details: http://red.ht/1lyY0pF

 

RTGov:

  • New UI to replace previous Gadget Server approach
  • Integration with Elasticsearch and Kibana for analytics
More details: http://red.ht/1iVa0r6


Speak thanks to Ivan McKinley for Elasticsearch contributions, Michael Clay for work on the new RTGov UI, and help from the SwitchYard, ModeShape and Fuse teams.

Friday, March 14, 2014

DTGov Release: Version 1.2.0.Final

We are very pleased to announce the release of version 1.2.0.Final of Overlord DTGov.  This latest version contains a number of bug fixes along with support for Tomcat 7!

Please download it and give it a try!  We welcome all ideas and feedback.

S-RAMP Release: Version 0.4.0.Final

We are happy to announce the release of version 0.4.0.Final of the Overlord S-RAMP project.  This new version contains a number of a bug fixes, but most importantly we now have support for all of the following runtime platforms:

  • JBoss EAP 6.1
  • JBoss Fuse 6.1
  • Tomcat 7

Additionally, the S-RAMP User Interface now includes an Ontology Manager, which is long overdue.  It's pretty hard to manage ontologies without an editor, due to the format (RDF + OWL Lite) of the file.  To help with that, version 0.4.0.Final includes a user interface that will allow users to create, upload, and edit their ontologies.



It is of course recommended that users finalize (as much as possible) their ontologies prior to using them to classify artifacts.  Once an ontology is heavily used to classify artifacts, it becomes much more difficult to make significant (read non-addition) changes to it!

Monday, February 10, 2014

S-RAMP specification 1.0 is public!

We are proud to announce that OASIS just officially published the S-RAMP 1.0 specification:

https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=s-ramp

The Overlord project had two members on the TC (Eric and yours truly), while Randall Hauch joined from the ModeShape project. The Overlord S-RAMP implementation is an Apache 2.0 licensed Open Source implementation of the specification, and it has already found adoption as the Enterprise repository of choice; one repository for all enterprise artifacts. You can download the complete spec bundle from http://docs.oasis-open.org/s-ramp/s-ramp/v1.0/cs01/s-ramp-v1.0-cs01.zip. For our implementation see: https://www.jboss.org/overlord.

Cheers!!

--Kurt

Thursday, December 12, 2013

DTGov release workflow using 'Domain Deployment'

Introduction

DTGov ships with workflows which can help you with deploying releases to runtime environments. At the moment the deployment service supports 4 ways to deploy an application:
  • copy, file copy to a directory
  • rhq, using the RHQ REST API to deploy an application to a resource managed by RHQ.
  • as_cli, using the JBoss AS Client to deploy to a JBoss Domain.
  • mvn, deploy the application to a maven repository.
In this blog post we demonstrate how to deploy an application using the JBoss AS Client.

DTGov configuration

To use the JBoss AS Client we need to configure a target in the dtgov.properties file referencing the as_cli protocol. For example to deploy to a domain on server 192.168.1.31 we'd use

governance.targets=   qa|http://www.jboss.org/overlord/deployment-status.owl#InQa|as_cli|admin::admin123!::192.168.1.31::9999

Where admin/admin123! are the credentials to log into the remote domain.

JBoss Domain Setup

The DTGov deployment service can deploy to a JBoss Domain. Look here if you need more information on how set one up.

1. Open the domain/configuration/domain.xml file for editing, and make sure you have a 'qa' group, which is name of the target mentioned above. You should end up with something like
    <server-groups>
        <server-group name="qa" profile="full">
 
2. Open the domain/configuration/host.xml file for editing, and assign servers to this group. In this example we add 'server-one' to the 'qa' domain group. You want to add more then one server.
    <servers>
        <server name="server-one" group="qa">

 

3. Now you can start your domain using something like
   bin/domain.sh -b 192.168.1.31 -bmanagement=192.168.1.31

You can test your credentials (admin/admin123!) by logging into the management console, which in this case would run on http://192.168.1.31:9990, and/or using the client: bin/jboss-cli.sh. You may have to add an admin user using the bin/add-user.sh script. Also verify that you see the 'qa' domain group in the management console.


Deployment of your application

At this point the release workflow is ready to deploy an application to the 'qa' domain group. Once you've approved the 'dev' task, you should see the dtgov deployment REST API logging on the dtgov server:

Calling POST TO: http://localhost:8080/dtgov/rest/deploy/qa/0e7a209c-f298-433c-b9a6-3d40d4ac2f5c



This means that the dtgov is invoked the jboss client on the 'qa' domain to deploy the application, where '0e7a209c-f298-433c-b9a6-3d40d4ac2f5c' is the UUID of the application artifact that will get deployed. If something is wrong at this point you can use your favorite REST client tool and perform POST's to this url until things work. 



 Figure 1. Screenshot of the management console after the requirements jar was deployed (probably a bad choice to deploy the requirements!!)

 I hope this worked for you too, and that you picked a better artifact then I did as deploying the requirements from the dtgov-demos-project doesn't really make a whole lot of sense! Well other then demonstrating that it works :).

Cheers,

---Kurt


 

Tuesday, November 26, 2013

SOA Governance Projects and Organizational Roles

In large organizations building and maintaining services is not a one man job, instead it is a process that touches many people in the organization. Figure 1 is a copy of figure 5.6 on page 95 of the 'SOA Governance' book by Thomas Erl et al.

Figure 1. Common associations of organizational roles with different SOA project stages.

With this many people and roles involved, how to you manage a project like this? Especially since in a large organization you will have many of these project running simultaneously. We would have loved to implement a full project workflow based on the figure 1 process, but two things are currently standing in our way: time and real world input. Who is going to pick up the challenge?

To get you started we created a 'Simplied Project Lifecycle Workflow' demo that implements just 3 boxes; Requirements gathering, Service Design and Service Implementation, with one role responsible for each stage, which are a Business Analyst, SOA Architect and SOA Developer. Each role can be fulfilled by more then one person. In the demo we assigned all three roles to the 'admin' user so we don't have to log in and out as different users all time. The demo uses the workflow shown in Figure 2. Each column represents one of the phases. The first phase being Business Analysis. The hope is this demo provides you with the building blocks to create the real world implementation we talked about earlier.

Figure 2. Simplified Project Lifecycle Workflow.

Some benefits of using this workflow are:
  • Helps your organization with adoption of SOA by following proven processes. It is clear who is responsible for approval.
  • Provides insight in where your projects are.
  • Helps your organization work together in different teams.
  • Audit features allow full history tracking.
  • Released artifacts are in the repository, the artifact is automatically in escrow this way, and documentation and sources are stored along side the binaries all in one place.
Some optional benefits:
  • It is possible to send a BPMN event at the end of the workflow (or from anywhere else), which can kick off a release workflow to automate deployment. Though one could also write a governance query looking for a service implementation artifact with classification #ImplPass.
  • Easy integration with other systems (think bug track systems, or time management systems)

If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Enjoy

Thursday, November 21, 2013

Customize Managed Deployments Workflow


DTGov ships with a number of BPMN2 based processes that you may want to update so they fit your business processes. In this demo we will update the 'Manage Deployments' workflow. It is recommended you go over the Managed Deployments usinf DTGov demo if you haven't already done so.

If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Wednesday, November 20, 2013

Managed Deployments using DTGov

DTGov ships with a workflow to perform deployments of released artifacts to the dev, qa, stage and prod runtime. Some benefits of using this workflow are:
  • Releases artifacts are in the repository before they go to a runtime, the artifact is automatically in escrow this way, and the sources are stored along side the binaries.
  • Automated deployments (using file copy, RHQ RestAPI, or JBoss Domain RestAPI).
  • Email notifications.
  • Sign off using Human Tasks in the workflow. Users and roles can be tailored to fit your organization. The sign off enhances the visibility of the where an artifact is in the deployment lifecycle.
  • Audit features allow full history tracking of the artifacts.
  • One repository for everything; so things can be found easily. This answers the "Where is my stuff" question. The XPath like query language of S-RAMP can used for searching.
If you want to follow along with the demo you should

1. Have DTGov running; see http://jboss-overlord.blogspot.com/2013/11/bleeding-edge-governance-getting-started.html.

2. Install the Eclipse BPMN2 Modeler into your Eclipse IDE, or you can try the early access JBoss Developer Studio.

Enjoy

Friday, November 1, 2013

Bleeding Edge Governance - Getting Started

We just completed an ant script that will help you get started with the latest Governance code in five easy steps and a bit of building time (your mileage may vary depending on the speed of your machine).


First download the latest script overlord.zip script (or if you prefer to use git: https://github.com/KurtStam/overlord)

Project Overlord consists of a repository (S-RAMP), Design Time Governance (DTGov) and Run Time Governance (RTGov). This script will build S-RAMP and DTGov. Both of these projects have a GUI component. All components share login information using SAML bearer tokens. The SSO code lives in a shared project called overlord-commons. Because of these SSO complexities we currently only support deploying to JBossEAP.

1. Check out the code from Governance on Github

This clones overlord-commons, s-ramp and dtgov from github. If you want to check out your own clones, then you can update the urls in the build.properties file.
ant clone
going forward you can update using ant pull.

2. Dowload JBossEAP-6.1

Navigate to http://www.jboss.org/jbossas/downloads, login and download JBoss-6.1.0.Final.zip to this directory.

3. Build

While your download is running you can kick off the build
ant install
 

4. Deploy to JBossEAP

When both 2 and 3 are finished you can deploy to JBossEAP by running
ant deploy
When ready start of EAP in standalone mode (jboss-eap-6.1/bin/startalone.sh) using a different shell.

5. Seed

You can now add the following section to your .m2/settings.xml
<server>
   <id>local-sramp-repo-snapshots</id>
   <username>admin</username>
   <password>your-admin-password</password>
</server>
<server>
   <id>local-sramp-repo-snapshots</id>
   <username>admin</username>
   <password>your-admin-password<</password>
</server>
 
Insert the admin password you supplied during step 4.
Make sure JBossEAP is fully booted up before you start the seeding
ant seed
This will upload the ontologies as well as the workflows. You can verify success using the s-ramp-ui. For now you will need to reboot JBossEAP to activate the workflows.
Good luck!
Please use the overlord user forum if you need help or have any other questions. We hope you will become part of our community and start writing your own extensions. Looking forward to seeing some good Git pull requests.
Cheers,
The Overlord team.

Friday, August 2, 2013

Overlord 1.0.0.Final has been released

The Overlord team is proud to announce the release of version 1.0.0.Final. There are three major components within this release:

1) S-RAMP - SOA Repository Artifact Model and Protocol (based on the OASIS standard)
Downloads Documentation

  • Artifact repository
    • Built on top of ModeShape
    • Stores artifact content and meta-data including:
      • Core properties (name, description, version, etc)
      • Custom ad-hoc properties (name-value)
      • Relationships between artifacts
      • Hierarchical tagging system called Classifiers
    • Rich XPath-like query language to search the repository
    • Atom based REST API
    • Automatic decomposition of artifacts based on type
      • Built in support for common S-RAMP types (XSD, WSDL, etc)
      • Extensible to support additional types (Teiid VDB, switchyard.xml, etc)
  • Command Line Interface
    • Supports interactive and batch modes
    • Built-in commands to manage and search for artifacts in the repository
    • Extensible to allow contributing custom commands
  • Maven integration
    • Can "mvn deploy" a deployment artifact directly into S-RAMP
    • Able to use S-RAMP artifacts as dependencies in maven projects
  • Web UI
    • Based on Errai 2.4
    • Allows granular management of content in the repository
    • Can be used to manipulate all artifact meta-data

2) DTGov - Design Time Governance
Downloads Documentation

  • Built on top of the S-RAMP artifact repository
  • Supports triggering Governance Workflows based on repository changes
    • Embeds jBPM 6 to support workflow execution
    • Maps an S-RAMP query to a workflow definition
  • Includes an out-of-the-box Governance Workflow to handle Deployment Lifecycle Management
  • Implements several REST services to support Governance Workflows
    • General purpose Notification (email) Service
    • Artifact meta-data Update Service (used to add meta-data to the S-RAMP artifact being governed)
    • Deployment Service used to deploy binary artifacts into runtime EAP systems
      • Supports RHQ
      • Supports JBoss AS CLI
      • Supports direct file copying
      • Can also deploy into a Maven repository
      • Supports multiple, configurable logical deployment targets (e.g. DEV, TEST, PROD)
  • Web UI
    • Includes a Task Inbox for human tasks created by the Governance Workflows
    • Provides a UI specific to the Deployment Lifecycle Governance Workflow
      • Can drill into a deployment to see its contents
      • Shows the change history for a deployment
      • Provides a way to see which deployments are deployed to which environments

3) RTGov - Run Time Governance
Downloads Documentation

  • Activity Collector
    • Activity events can be reported directly to the Activity Server via REST service, OR
    • Embedded collector to intercept activities, pre-process the information to extract correlation details and business relevant properties, before batching events for distribution to the server.
      • Events can be 'validated' using pluggable event processors, which can be used to block business transactions
  • Activity Server
    • Provides a REST interface to enable events to be reported
    • Persists activity information
    • Distributes reported events to subscribed components.
  • Event Processor Network
    • Subscribes to the Activity Server, to receive reported activity events
    • Distributes activity events through a series of registered networks, that consist of event processors to validate, transform and retransmit events. Processors can be specified using Java, Drools CEP rules and MVEL.
  • Maintaining Derived Results
    • Information derived by the Event Processor Network can be presented as a set of 'active' collections
    • Components can directly access these collections, or issue queries through a REST API.
  • Web UI
    • Runtime governance information is provided through a set of gadgets.
      • Service Response Time Gadget - displays response time information per service and optionally operation.
      • Situations Gadget - table for displaying situations detected by the event processing (e.g. SLA violation)
      • Call Trace Gadget - provides a tree based visual display of the call stack associated with a business transaction
      • Service Overview - graphical representation of the services and their dependencies, showing response time and invocation details.

If you have any questions about the Overlord governance capabilities, please post to the User Forum. If you find any issues or would like to raise a feature request, then please access the Jira issue tracking system for the particular sub-component. Links to these are found on the Overlord website.

Tuesday, June 11, 2013

Runtime Governance release 1.0.0.M5

The Overlord team are pleased to announce the availability of version 1.0.0.M5 of the Runtime Governance (RTGov) sub-project.

This release adds the following capabilities:

  • Collection of activity events from SwitchYard BPEL and BPM components
  • Enable security on REST services
  • Ability to enable/disable activity event collection based on a property, and switchable at runtime through JMX
  • XSLT transformer for processing activity event content
  • Information processor support for header values
  • Event processor for sending emails
  • Event processor for persisting events (e.g. situations)
  • Added a new context type called Link, for correlating two activity events within a specified timeframe, where the correlation value may not be unique outside of that timeframe
  • H2 database configured on installation
  • Various bug fixes and minor changes - see release notes for more details.

This release is compatible with SwitchYard 1.0.0.Alpha1, and should be installed in JBoss EAP.

For more information, please see the release notes at:
https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12313521&version=12322009

Wednesday, March 27, 2013

Runtime Governance 1.0.0.M4 released

The Overlord team are pleased to announce the availability of version 1.0.0.M4 of the Runtime Governance (RTGov) sub-project. This sub-project was until recently called 'Business Activity Monitoring', and was renamed to reflect the wider scope of its capabilities.

This release adds the following capabilities:
  • Inline (synchronous) policy enforcement. The previous version enabled policies to evaluate activity events "after the fact" to determine if a situation had occurred, and then provide some enforcement capabilities for future business transactions (e.g. to block a customer's purchase requests if their account had been suspended). This new capability enables event processors to be configured to immediately evaluate activity events and if necessary block a business transaction.
  • For business transactions that have associated 'principal' information, this information will be recorded with the activity events are made available in derived results (e.g. situations and call trace).
  • Location information, recorded as part of the activity events, is propagated to derived results (e.g. situations and call trace) to enable users to understand where problems have occurred.
  • Runtime governance now works in a clustered JBossAS environment.
  • Activity events stored in H2 in-memory db. Other databases will be supported in the next version.
  • Lightweight activity collector client configuration. It is now possible to install just the activity collector mechanism within an execution environment, and have it report the activity events to another fully configured runtime governance server via REST.
  • Various bug fixes and minor changes - see release notes for more details.

This release is compatible with SwitchYard 0.8.Final.

For more information, please see the release notes at: https://issues.jboss.org/secure/ReleaseNote.jspa?projectId=12313521&version=12321356