Business Process Analytics using BPAF

Over last 6 month I've been working on several components that will be assembled to a bigger business activity monitoring solution. I started off with a CEP based activity monitor (SAM) that allows for advanced correlation across heterogeneous event sources. After that I've spent some time integrating the BIRT reporting runtime with the BPM console, which allows you to run complex reports on process-related audit data. Only to realize that I've started on the wrong foot.

Process analytics bottom to top

I made a mistake to begin with the most complex components first, without realizing that the key to a successful BAM solution is underlying data model. Without a solid baseline the creation of report templates becomes a daunting task because the missing unification of audit data basically requires you to customize the report algorithms for any instrumented subsystem. Creating rules for the CEP component that fire when certain SLA's are breached doesn't get more simple either, when working on a heterogeneous event model.

I finally spend some time prototyping around an idea that has been in the wild for quiet some time: Baseline process-related audit data on BPAF: A WFMC standard that describes a data structure that enables the aggregation and correlation of audit data across multiple platforms.

In the following sections I would like to walk through a simple protoype, that already shows the benefits of this approach. It is really impressive how little code it requires to deliver a solution that already matches an important number of the use cases.

Data model, storage and retrieval

To begin with an implementation for storing and querying BPAF event data was needed. Fortunately JAXB and Hibernate made this a trivial task:

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "", propOrder = {
    "eventDetails",
    "dataElement"
})
@XmlRootElement(name = "Event")
@Entity
@Table(name="BPAF_EVENT")
public class Event {

  @XmlElement(name = "EventDetails", required = true)
  protected Event.EventDetails eventDetails;

  @XmlElement(name = "DataElement")
  protected List dataElement;

  /**
   * A globally unique identifier for the individual event
   */
  @XmlAttribute(name = "EventID", required = true)  
  protected long eventID;

[...]

}

The current implementation allows two approaches for querying the BPAF data: Either leverage the default model and it's persistence implementation, or adopt a proprietary audit data structure to the BPAF model when needed. For later use case the API fo querying is split from the default data model:

/**
 * @author: Heiko Braun 
 * @date: Mar 10, 2010
 */
public interface BPAFDataSource
{
  /**
   * Get a list of distinct process definition ID's that are known to the system.
   * @return a collection of process definition ID's
   */
  List getProcessDefinitions();

  /**
   * Get a list of distinct process instance ID's that are known to the system.
   * @return a collection of process instance ID's
   */

  List getProcessInstances(String processDefinition);

  /**
   * Get a list of distinct activity definition ID's that are known to the system.
   * @return a collection of activity definition ID's
   */

  List getActivityDefinitions(String processInstance);

[...]
}

Now that we have components in place for storing and querying raw BPAF data, let's move on to a more interesting topic: How to make sense of it.

Turning data into information

The whole purpose of this exercise is to provide a way to gain insights about the execution characteristics of the processes that drive your business. While BPAF itself is a compact but still very expressive format (BPAF State Model) to store process-related audit data, it requires additional steps to turn it into consumable information that participants and decision makers can make sense of.

For the sake of this example we've added some algorithms that allow grouping and sorting of the underlying data, which will be used by the presentation layer.

Let's take a look at the prototype UI (yet another BPM Console plugin) and see how much information can already be derived without going through hoops and loops.

Presenting audit data to decision makers

The prototype UI demos a simple use case: "How did the processes execute in the past?" Although a simple question, it already implies a lot of fine grained functionality a user might expect:

• Selecting a relevant process
  
• Restricting the query to a certain time period
  
• Scrolling through business related events that occurred over a period of time
  
• Zooming in and out on a time scale (hourly, daily, weekly, etc)
  

Concepts represented by the prototype UI

For the sake of simplicity the current implementation (at the time of this writing) is limited in functionality. Nevertheless I would like to walk you through the current features and explain some of the key concepts.

To begin with a user selects the Process Defintion and the relevant time period:




The time period offers a set of commonly used intervals, like "Last Week", "Last Quarter" or "Last 24 Hours".

The BPAF data is loaded and grouped according to the time period and presented in an interactive chart:




The charts allow you to zoom in on the time scale (i.e. "1 day, 5 day, max") and scroll through the event data on each level.
(Shout-Outs to the chronoscope project)

You can comment on the information and create markers on business relevant points that either require clarification or otherwise mark notable changes in the execution of a process:




Conclusion and next steps

As you see, this simple prototype already covers important features that you would expect from such an implementation.
And we didn't yet touch sophisticated topics like comparison of process executions or creation of key performance indicators that trigger notifications (do the markers ring a bell?).

In the next few month you can expect this approach to be married to SAM, our CEP based BAM component that allows correlation and execution on process activities close to real time.

Stay tuned, now that we have a common baseline for representing and querying process-related audit data we'll look into more advanced topics.