Confessions of a Software Developer

Tips for Installing Oracle BPM Suite 11g FP4

2011-09-21T15:28:00.006+02:00

Edited on October 20, 2011 regarding the location of OPatch (due to progressive insight, and some comments).

When installing Oracle BPM Suite 11g FP4 I ran into a couple of things, of which I thought I better write that down for some next time. If you are a SOA Suite install crack, don't bother to read on, as I probably have nothing new to tell you. In case you are new to it or do installations only occasionally, you might find this saving you considerable time. It would have done so in my case.

You typically install the SOA/BPM Suite on top of WebLogic. The MW_HOME is the (middleware) folder containing the wlserver_10.3 folder (among others). The Common Oracle Home will be [MW_HOME]\oracle_common, while the SOA_ORACLE_HOME will be [MW_HOME]\Oracle_SOA1. In the readme for the PS4 installation ORACLE_HOME refers to both, so read the instructions carefully.

The first instruction is to make sure you have the latest OPatch. When you download it, you will find that the instructions of OPatch itself say you can find OPatch in ORACLE_HOME. There actually is one in both ORACLE_HOMES. Probably the same one, I did not bother to find out and copied the download OPatch to both folders.

The instructions are particularly written for Unix, so needs some "translation" to make it applicable for Windows:

Where it reads "$ORACLE_HOME" you should replace that by "%ORACLE_HOME%" (duh!)
To patch the Oracle Common Home it tells you to use the following command:

$ORACLE_HOME/OPatch/opatch napply -invPtrLoc $ORACLE_HOME/oraInst.loc

It took me some time to find out that the invPtrLoc only applies to Unix only, and that for Windows you can simply leave it out. On Windows the inventory information is located in the registry instead of the oraInst.loc file. So the command for Windows is simply:

%ORACLE_HOME%\OPatch\opatch napply

PSA stands for Patch Set Assistant. I skipped the backup of the database, because as a developer I can simply recreate the SOA INFRA any time without loosing any valuable information.

The instructions will tell you to run PSA from the ORACLE_HOME\bin folder. Now that's a bit confusing as there is a psa.bat in the bin folder of both ORACLE_HOME's. But if you read the PSA instructions carefully, you will find it refers to SOA_ORACLE_HOME ([MW_HOME]\Oracle_SOA1. An educated guess already made me already think so.

If you are using OTN you may not be referred to the proper post-installation guide. The one it currently points to, is the post-installation steps for Oracle SOA Suite for Healthcare Integration. If you need that do that first, but then you still have to do the BPM post-installation instructions.

BTW, this guide discusses the commands for Unix as well as Windows, and specifically uses SOA_ORACLE_HOME. But that's for weenies, and not us developers, as we like finding out the hard way, right?

While I'm at it I might as well past the command I used for updating the policy store, as it may just fit your environment as well:

wlst.cmd %soa_oracle_home%\bin\bpm-ps4fp-upgrade.py --username weblogic --password welcome1 --wlsHost localhost --adminServerListenPort 7001

The post-installation instructions tell you to delete the DOMAIN_HOME\servers\...\tmp. Many of us developer probably choose to deploy the SOA Suite on the Admin Server, so the only servers\tmp folder you then have is DOMAIN_HOME\servers\...\AdminServer\tmp. According to the documentation about domain configuration files, you can simply delete the contents of that folder, so that is what I did. Have some patience, as that folder contains GB's of files.

If you followed my example, the next post-installation instruction may make you start cursing, as it will tell you to copy some library into a sub-folder of the tmp you just deleted. I, on the contrary, kept my usual cool, and just started the server. I still kept my cool when looking at several exceptions, not recalling if those were new, of whether I've seen those before. So I bounced the server and found that the exceptions disappeared. Also the WebLogic Console and the Enterprise Manager appear to function normally (pfew!).

I stopped the server, and continued with task 3, regarding copying the jar file adflibSOAMgmt.jar. And bounced the server again.

Finally, the readme on OTN mentions that you should have a soa-jdev-extension.zip. The one I have also mentions a bpm-jdev-extension.zip. Both instructions do not mentions to actually install them. I did both in the usual way (JDeveloper -> Help -> etc.).

That's when I was ready.

Oracle BPM FP4 Is Out!

2011-09-20T10:57:00.002+02:00

Finally, it's there, the long awaited Feature Pack 4 for the Oracle Business Process Modeling Suite.

Instructions for Customers to get this are as follows:

This is available as a patch under the following bug id: Patch 12413651: SOA PS4 BPM FEATURES PACK

The patch is password protected and is available to those BPM customers that request it from support by filing an SR

It is intended for use by BPM customers only. SOA Suite customers should check with Support and Product Management before requesting it.

Information on PS4FP will be blogged at http://blogs.oracle.com/bpm soon.

Documentation is available at http://www.oracle.com/technetwork/middleware/soasuite/documentation/11gr1ps4featurepackdoc-462677.html

OBPM 11g: Showing More Detailed Log Info

2011-09-14T11:28:00.012+02:00

One of those too stupid to be discussed items is how you can make the SOA Suite show detailed information about the payload for activities like a script activities, or what is going on inside a business rule. After all, every SOA/BPM developer is assumed to know how to set log levels, right?

Well every developer is also assumed to know what they say about assumptions, and how frustrating it is to loose valuable time on finding out how to do simple things. So bare with me while I state the obvious.

Showing Detailed Payload Information

When the audit level is set to Production (which seems to be the default), only data associations for asynchronous activities are logged. You can see which level you are using, e.g. in the Audit Trail page, as shown below.

Because I have set the log level to Development, I do not only see detailed information about the payload that left the Handle Time-out sub-process, but also of the instance system fault.

This log level can be set in the Enterprise manager, but (as the pop-up with the information indicates) not in the Log Configuration, but in the SOA Infrastructure Common Properties. You can find it as shown below.

Show What's Going on in Business Rules

Another one of those "too obvious" things is showing detailed information about what is going on inside a business rule.

When you ask you always get an answer like: "You can turn on debug tracing in the Rules SE by setting the log level for oracle.soa.services.rules to TRACE:32 to get more detailed logging of what is happening." Absolutely a valid answer, but when you have not had to much exposure to setting log levels, this can take you some time to find out how to do.

This is done by changing the Log Configuration, as follows:

Oracle SOA/BPM - Searching and Reporting Using Process Payload

2011-08-12T13:01:00.008+02:00

For searching business process instances using specific attributes from the payload, the regular way to do so when using the Oracle SOA Suite is by using mapped attributes (formerly known as flex fields).

Using mapped attributes is pretty straight-forward. You configure them in the Administration tab of the Workspace, and map a specific element of the payload onto a mapped attribute. Once it has been mapped, the attribute can be added as a column to the task list and is available for filtering. The mapped attributes can also be used in custom code via the API.

Things to Consider

Mapped attributes have some important aspects to consider:

They can only be used for simple type attributes (String, Number, Date),
Mappings are task-specific, so to be able to search for example on an order.status element throughout the process you have to map it for each activity,
There is a limited amount of mapped attributes (20 Strings, 10 Number and 10 Dates),
Changes to mapped attributes are only applied to instances instantiated after the mapping took place,
When instances are purged all historical data of those instances will no longer be available. In a production environment, purging of instances typically is done by a Systems Administrator with no (functional) knowledge of specific processes.

Especially the latter two aspects may require a different approach to secure full flexibility regarding searching and reporting on instances. This might for example be the case when there is a requirement that historical data should be kept indefinitely, or should only be purged in a controlled way by a Applications Administrator. In case of BPM, the requirement for a flexible approach on searching and reporting on process instances is pretty common.

In such cases an alternative to using mapped attributes (and reporting using the dehydration store) is to have a some custom database in which significant updates to process data are being stored. The advantage over using flex fields would be that:

There is no limitation in the amount of attributes,
The data of old instances can be manipulated, e.g. by providing default values for new attributes,
Management of the custom database can be delegated to some Application Administrator that does have (functional) knowledge about the process.

The advantage of not needing to create a mapping per task, will obviously be over-compensated by the fact that the process has to do a service call every time the data needs to be saved, but again this buys back a lot of flexibility.

Instead of using service calls to save this data, you may consider composite sensors. A composite sensor is a specific type of BPEL Process Manager Sensors. Be aware though that composite sensors can only monitor incoming and outgoing messages, and not changes of the payload within a process instance. For this reason in most cases this won't be an alternative.

Using JAXB for Manipulating Payload of Human Tasks in SOA Suite

2011-08-09T18:58:00.007+02:00

In some cases you may want to manipulate the payload of a human task of an OPBM or BPEL process instance using JAXB. An example would be when you are using some other framework than ADF Faces for creating the UI, and you want to work with Java objects instead of manipulating XML programatically.

To do so with JDeveloper, you can generate the JAXB content model by right clicking the xsd and choose "Generate JAXB x.x Content Model".

Be aware that you must do this using the human-task-specific payload, and not the xsd that was used to define the variable that gets passed into that human task, otherwise to your sad surprise you will get all kind XML validation issues when trying to push the data back to the process. Fortunately on its turn the xsd of the human-task-specific payload imports the original xsd, so when the original xsd changes, the human task specific payload automatically changes with it. You still have to regenerate the JAXB content model to let it reflect the changes.

The following picture shows the OrderCreationPayload.xsd of an OrderCreation human task. It imports an Order.xsd. The JAXB content model has to be generated using the OrderCreationPayload instead of the Order.

DWM FT and Hyperion BPM Solution Workbench for Essbase Retired

2011-06-03T09:49:00.003+02:00

As of December 2010, OUM provided full support - Estimating, Delivery, and Training - for Business Intelligence (BI) Custom engagements (including Hyperion Essbase). This support represented a major milestone in the evolution of OUM and enables BI Custom (including Hyperion Essbase) practices to transition from the legacy methods to OUM.

The following methods and their associated estimating models therefore have been officially retired:

Oracle Data Warehouse Method Fast Track (DWM FT)
Hyperion BPM Solution Workbench for Essbase Engagements

Use Cases or User Stories

2011-05-24T15:09:00.006+02:00

When teaching use cases, a question that comes up now and then is what the difference is between a use case and a user story as used on agile projects. There is a lot to be found on the internet discussing this. Two useful references and a start for further investigation are mentioned below. What I try to do here is capture my conclusion of these discussions. But before coming to this, let's briefly discuss what is what.

User Story

A user story is a short statement about what a user wants to do, and why. A user story typically fits on an index card. The idea is that the development team uses this user story as a starting point of a conversation with this user about how this should work in practice, to get an understanding of what needs to be done to support that.

On some projects a particular format is being used for user stories, being "As a ... I want ... so that ...". Reviewing this format it becomes clear that the first ... can be compared with the actor of a use case, and the last one with the goal of a use case (although in many cases tend to be finer-grained). So the big difference apparently is in the middle, where it is described what the actor does to achieve that goal.

Use Case

Next to specifying a goal, a use case captures one or more scenarios describing the interaction between an actor and a system to achieve that goal. Scenarios may be captured using a format like:

1. This use case starts when the actor does ...
2. The system responds by doing ...
3. The actor does ...
...
x. The use case ends when ...

There is one main success scenario (happy path), there may be one or more alternate scenarios (other ways to achieve the same goal), and there may be one or more exception scenarios (describing what happens when that goal is not achieved).

You also capture what triggers the use case (which in many cases will be repeated as the first sentence of the main success scenario), pre-conditions (what needs to be in place when the use case starts), and post-conditions (what will have been achieved when the use case ends).

What's Different?

So, apparently, a use case elaborates more on what the actor does to achieve a goal. A user story has just one statement regarding that and, compared to a use case, concerns just one scenario. Also, compared to a user story, a use case adds a trigger, pre-conditions and post-conditions to that.

Some people say that use cases capture too much detail, and claim that user stories are better for this reason. Other people state that in practice user stories tend to oversimplify things, resulting in the first iterations taking much more time than anticipated. The way to resolve that is by elaborating more on the user story. When you think about it, that makes sense, doesn't it? I mean, at the end of the day even on agile projects the developers need to deal with details as well. They only may do it in a different way.

Conclusion

In my opinion the core difference has best been captured by a guy called Jim Standley who stated that a user story is a promise to have a conversation and a use case is the recording of that conversation.

So rather than arguing if user stories are better than use cases or vise verse, I think the better question is: "How formal do you need to be?". The more formal, the more appropriate use cases start to become.

Added to this, you should also think about "When do we need to be this formal?". I can imagine systems being developed starting with user stories, and then write use cases after the fact, because some external testing team requires that, or because the customer needs it for system maintenance.

Some Suggestions

If you are on an agile project creating use user stories, but you are required or may need to create use cases later on, make sure that a user story matches exactly scenario, and name them uniquely. That should not be too difficult.

Then, when the user stories have been realized and you need to deliver the use cases, you can then use the name of the user story as the name of the corresponding scenario. You combine all user stories that share the same user goal into one use case. Add the trigger, pre-conditions, and post-conditions to that, stir for one minute, and you're done!

Further Reading

Stellman/Green: User Stories vs Use Cases

Discussion at Allistair Cockburn: A user story is to a use case what a gazelle is to a gazebo.

Old Methods Die, Long Live OUM!

2011-05-10T11:08:00.003+02:00

Just wanted to let you know that the retirement of the following methods is planned for June 01, 2011:

* Oracle Data Warehouse Method Fast Track (DWM FT)
* Hyperion BPM Solution Workbench for Essbase Engagements

OUM 5.4 Has Been Released!

2011-03-17T09:07:00.002+01:00

Yesterday OUM 5.4 has been released. Among other things, the following improvements has been made:

Tactical SOA View added
Various techniques regarding monitoring and improving SOA (instrumentation)
A white paper about how to apply OUM with Scrum
Several SOA templates

The Tactical SOA View helps to get up-to-speed with OUM on SOA projects that do not (directly) have an enterprise approach.

The templates concern a Service Contract, a Service Catalog spreadsheet in case you don't have an Enterprise Repository, and some more.

Check it out!

Installing OBPM Suite 11g PS3 using XE

2011-03-03T13:42:00.003+01:00

When installing the OBPM Suite 11g PS3, I ran into an issue with the Metadata Services (MDS) schema on my XE database. The error I got when I tried to start the WLS SOA domain was "ORA-04063: package body "DEV_MDS.MDS_INTERNAL_SHREDDED" has errors". I checked the schema and found that all three existing packages were invalid. Recompile did not solve the issue.

I Googled a bit and found out that I had to re-run the Repository Creation Utility, as apparently there is an extra environment variable to set before you run this. I fixed it as follows:

Open a dos-box in the ..\rcuHome\bin folder
Enter: set rcu_jdbc_trim_blocks=true
Run rcu.bat
Drop the existing repository
Recreate a new one
All invalid objects disappeared!

BTW, the tips in a previous posting, are still valid. Especially the one regarding setting the JVM memory setting of the WLS SOA domain:

set DEFAULT_MEM_ARGS=-Xms512m -Xmx768m

OPBM and External Activities

2010-10-13T19:56:00.007+02:00

I'm currently involved in a project where OBPM 10g is being used in combination with another system providing the user interface.

The idea is that this external system (let's call it system X), polls the engine to find out if there is a task to perform by some logged on user, and if so supports that task as (what is called) an external activity. This polling mechanism would allow you to go from one activity to another where each activity supports one single screen. As a result you are basically modeling a screen flow rather than a business process.

The problem is that you cannot model the screen flow 100%. Think for example about a screen that should provide a pop-up to do some things and then return. There is no (reasonable) way to use OPBM to support this pop-up functionality.

Another problem becomes apparent when you need to call services inside a screen flow. What are the criteria to decide whether OPBM or system X should call the services?

After a couple of weeks and progressive insight, I have come to the conclusion that you should try to prevent modeling a screen flow as a process. So my guideline is to let system X handle the complete screen flow (including the service calls that are specific for that screen flow), in a similar way how you would use a native OBPM screenflow. That really makes life simpler. More over, if you later decide to implement the screen flow in OPBM instead, there is a straight-forward "migration" that won't impact the business process itself.

Some Tips for Installing Oracle BPM 11g on Windows XP

2010-05-31T11:28:00.002+02:00

Some tips to get Oracle BPM 11g running on Windows XP, that you may not easily find in the documentation available.

What to collect before you install Software used:

Oracle 10g XE Universal
Repository Creation Utility 11.1.1.3.0
Weblogic 10.3.3 + Coherence - Package Installer
SOA Suite 11.1.1.2.0
SOA Suite 11.1.1.3.0 (patch on 11.1.1.2.0 with BPM)
JDeveloper 11.1.1.3.0

When it is not possible to connect JDeveloper to the internet for a check for update:

soa-jdev-extension
bpm-jdev-extension

Install the software according to the Quick Installation Guide for Oracle SOA Suite as can be found on the Documentation for Oracle SOA Suite page. On XP I did not need to configure anything out of the ordinary. You can install it in the order listed above. Do a typical install of the SOA Suite in a new Oracle home, e.g. named Oracle_SOA1 (the default). Right after that install the 11.1.1.3.0 patch the same way you installed 11.1.1.2.0.

BPM Domain

Create a domain for the BPM Server, using the config.cmd in the [ORACLE_HOME]\common\bin. On the Select Domain Source tab select:

Oracle BPM Suite
Oracle SOA Suite
Oracle Enterprise Manager
Oracle Business Activity Monitoring
Oracle WSM Policy Manager
Oracle JRF - 11.1.1.0

Call the domain e.g. bpmdomain. Use the Sun JDK, and configure the JDBC Component Schema according to the schemas created by the Repository Creation Utility.

On the Select Optional Configuration screen you now have the option to select a single server or managed server configuration. Due to restrictions in the way XP uses memory, choose the latter by NOT checking anything and just press Next. This will result in a separate admin and BPM server.

Once finished, locate the setSOADomainEnv.cmd file e.g. in the [ORACLE_HOME]\user_projects\domains\bpmdomain\bin\ folder. Change the memory settings as follows:

set DEFAULT_MEM_ARGS=-Xms512m -Xmx768m

I could not get it running with -Xmx1024. Do not set -Xms512m to 768, as that will result in both servers taking a minimum of 768 Mb, and also a significant longer start-up time.

JDeveloper

In JDeveloper create an application server connection to the admin server, pointing to the bpmdomain (in my case), and test the connection. All 9 tests should succeed.

To prevent a 502 error while deploying without any further information about what is going wrong, make sure you disabled the proxy settings in JDeveloper.

BPM 11g Generally Available

2010-05-12T17:08:00.007+02:00

You may already have noticed that BPM 11g is generally available. As BPM 11g runs on the SOA Suite, it would not be correct to call it the "BPM Suite" anymore. As a matter of fact, BPM 11g currently is only available as a patch on the SOA 11g Suite.

To run BPM 11g on your laptop you will need the following:

Oracle Server (to setup a playground environment Oracle Server 10g XE is good enough)
Weblogic Server 10.3.3 (unless you need it for something else, without OEPE)
Repository Creation Utility 11.1.1.3.0
SOA Suite 11.1.1.2.0 (SOA Suite 11g)
SOA Suite 11.1.1.3.0 (which includes BPM and is a "patch" on 11.1.1.2.0)
JDeveloper 11.1.1.3.0
JDeveloper extensions for SOA and BPM (via the check for update feature of JDev)

Excluding the database, for Windows that is 769 MB + 254 MB + 1.7 GB + 1.9 GB + 1.2 GB + 262 MB, is uh ... only 6 GB! Goodbye "next-next-finish", and hello download managers!

The good news is that for the same money you also get all the rest of JDeveloper and also BPEL with that!

BPM & Use Cases, Who's Counting? Revisited

2010-04-29T17:53:00.005+02:00

Whenever I have questions about use cases myself, I grab my use case bible which is Writing Effective Use Cases from Alistair Cockburn. One of the sections in that book is named Your Use Case Is Not My Use Case, which discusses that different people writing use cases for different purposes or audiences, may write use cases differently. This posting I also could have named My Use Case Is Not My Use Case, at least not at Different Points in Time, as you will soon find out.

In one of my earlier postings called BPM & Use Cases, Who's Counting? I discuss an example and suggest that some automatic notification activity is part of the preceding interactive activity. Then in a total different context I noticed myself arguing to a colleague that a notification in her use case model was a use case of its own. It then realized that in general this is a better approach.

The following example will show why:

Assuming that the Customer is not a direct user of your system, for the Notify Customer activity you could argue that it would be part of the goal of the Account Manager to notify the customer. But for the Notify Warehouse I find it way more intuitive to claim that it is a goal of the Shipping Clerk to be notified that there is work to do. But that would imply that the first notification would be part of the Review Order use case, while the second notification would be a use case of it's own. Does that sound inconsistent or what?

So I therefore want to change my opinion and suggest that in principle you should consider a notification activity to be a use case of its own, having the one being notified as primary actor (i.e. having the goal). Make sense, not?

Why in Oracle BPM Automatic Activities Belong to the Automatic Role (at least one of the reasons)

2010-02-24T20:51:00.002+01:00

Strictly speaking, an automated activity does not have a human actor as role involved, and from that perspective the automated one does not belong in the swim-lane of that role, but in the one of the "automatic" role instead.

You may say that this is smells like over-designing the thing, and moreover it takes extra space, so rather not.

But there also is a practical edge to this. I happened to notice that there are certain circumstances in which automatic activity can actually show up in the workspace of people having that role. That can happen when the automatic activity takes some time to complete, and (if memory serves me well) I also saw this sometimes happening with timers. Very confusing for the user, because it looks like he or she has something to do, while the workspace does not allow doing so.

I therefore recommend always to put automatic activities in the "automatic" role, at least once you start implementing the process. The obvious exceptions being automatic activities inside a screen-flow.

How to Recognize the Requirements in Use Cases?

2010-02-24T20:28:00.002+01:00

One of the core values of using a method is that it provides a common "language". That holds for all kind of methods, including those for software engineering. It for example helps when everybody has a same understanding of what a "use case" is. It also helps to have a common format for use cases. Similar to the agreement that (at least in the western culture) we write from left to right, top to bottom, contents at the beginning, index at the end. As is the case with a book, you will appreciate that a common format for use cases not only helps those writing them, but especially those reading them (business as well as IT people).

Next I will discuss a format that I have learned to appreciate very much, and can recommend to everyone who is writing use cases.

If you have not already decided upon a common format, or when you have the opportunity to do so, you may consider writing use cases in a two column format. I learned this technique more than a year ago from a great guy called Vince Bordo (hi Vince!). I practice this technique since then, as I find it to be more effective than what I used to do (i.e. a one-column format).

In practice this looks as in the following example, which is a reformatted version of a use case I used in some previous posting:

The beauty of this format is that it visually depicts what the system needs to do in order to satisfy the requirements, as that is exactly what the right column is about. The left column describes how actors (primary as well as secondary) interact with the system in order to satisfy the goal of the use case.

In theory (I would not try this in practice, but in theory) you should be able to cut out the right hand column of the use case and only hand that over to the developeras and tell them that this is what they need to realize. That, plus the supplemental requirements of course.

Use Cases, Explaining Main Success, Alternate and Exception Scenarios

2009-12-03T19:39:00.006+01:00

Once you understand the difference between a main success scenario, an alternate scenario and an exception scenario, you may wonder why it took you so long to get there. The reason probably being that the explanation was pretty abstract, talking about scenario's that do or don't have the same goal, post-conditions that add to or are instead of those of the main success scenario, blah, blah, blah...

Having gone this road myself, I started to wonder if there is a simpler way to explains what should be pretty natural for most of us. And as always, finding an example that relates to day-to-day life did the trick for me to explain it to others. The example being the following.

Suppose that have to drive from A to B. Your goal being to be in time for a customer meeting. The post-condition being that you successfully reached your destination in time.

So the main success scenario is that you drive from A to B, without interruptions.

An alternate scenario would be that you have to take a small detour by going through a gas station to get some gas. An extra post-condition this detour might add to the one of the main success scenario, could be that you have to obtain the gas bill, otherwise you employer won't cover your expenses.

An exception scenario might be that your car breaks down, and you will never be in time for the customer meeting. A post-condition of this exception scenario might be that you have to inform you customer that you have to cancel the meeting. This post-condition replaces the post-condition of the main success scenario.

Other exception scenario's might concerning a serious traffic jam, getting busted for speeding, car-jacking, and so on.

Now that was not too difficult, was it?

Is There Life After Oracle BPM Studio 10g?

2009-11-29T11:50:00.005+01:00

Let me begin with the answer, which is: absolutely!

After a Thomas Kurian discussed the positioning of Oracle BPM (fka ALBPM) a year ago (is it that long? Yes it is!), I must admit I was worried about the strength of Oracle BPM being properly appreciated. For quite some time focus seemed to be solely on the Oracle BPA with BPEL (from the SOA Suite) combination only.

But times (and vision?) have changed since then. Recently I was in a conference call in which a preliminary version of BPM 11g was demo-ed. And I was surprised in a positive way. Oracle BPM 11g seems to preserve all the good of 10g, while at the same time it is really integrated with the rest of the product stack.

Some Things That Struck My Eyes

With BPM 11g Oracle managed to continue the ALBPM "experience" of easy process modeling and implementation. In a series of short iterations you transform the BPMN 2.0 process you collaboratively modeled with the business analysts in an executable one, without a paradigm shift like you for example would have when going from BPMN to BPEL. For me this always has been one of the major strong point of BPM. What has been added to this is a so-called zero-code environment of developing screen and including external resources (like services).

Some of the more technical features that I found appealing were:

Usage of the common adapter framework, as is already available in the SOA Suite,
Support for easy (ADF-Faces) development of rich task forms with AJAX support (without bothersome JSP development),
A web-based UI for modeling processes by business analysists,
Native integration with Oracle BAM,
A composite (SCA) view on business processes, which (among other things) supports native integration with BPEL,
Native integration with Oracle Business Rules,
Unification of the BPM and BPEL worklist,
With all this, an improved support for true Business Process Management.

What About The BPA Suite?

The positioning of BPA has not changed, but became more clear instead. BPA is typically for a more enterprise approach to business analysis, including a top-down approach to business process modeling, including BPMN. When you have a human-centric business process, you can decide to export these to Oracle BPM 11g. Otherwise, BPEL is the obvious choice.

Boy, I can't wait to get my hands dirty with BPM 11g, which hopefully will be somewhere in March 1010!

OUM 5.3 Has Been Released

2009-11-27T20:48:00.002+01:00

November 13 there has been an announcement that the Oracle Unified Method version 5.3 has been released. As explained in a previous posting, we do offer OUM to our customers.

What's New?

Although the increase in version number (from 5.2 to 5.3) seems to suggest this to be a minor upgrade, some of us will receive as a major upgrade.

For example, this version includes an initial support for Business Intelligence (BI) and Enterprise Performance Management (EPM) implementation. We also added a view for Software Upgrades, which will help in in quickly determining which tasks to consider for an upgrade of Oracle software products, including middleware, database, enterprise application products and Business Intelligence solutions.

Regarding the other, existing views, a lot of improvements have been applied. Especially Envision (the focus area of OUM covering enterprise / business level aspects) has been upgraded, and is rapidly reaching a "mature" state. Two of the topics that I would like to point out are IT Governance and IT Portfolio Planning, mainly because I have been personally involved in that (sorry, too hard to resist)!

Training

Perhaps even more important than having a comprehensive method, is being able to provide training on that, and assist in applying the method. In the last year The Oracle Methods Team therefore has put considerable effort in creating various training modules, from high-level overviews that make you understand what OUM is all about, to more task-oriented modules around requirements gathering or analysis and design.

Customers won't find them in the curriculum of the Oracle University (yet) but don't let this keep from for asking for it, as we can deliver customer training.

See you in class!

OBPM Business Exceptions and Heirs

2009-09-30T18:47:00.005+02:00

Often the solution of a problem is having an eye for detail, like properly reading an error message most of the times is half the solution. Finding out how to create heirs for the Oracle Business Process Modeling Suite (OBPM) appeared to be yet another thing requiring an eye for detail.

A colleague insisted it worked for him. However, no matter how I tried it simply did not for me. We checked versions numbers, build numbers, both the same. Then another colleague came by the other day, and for him it worked too. Then I started to pay attention to the details and watched how he did it.

What I did was right-click an existing Business Exception -> Create Heir, resulting in an ordinary business object. No matter how hard I kicked it, it refused to becoming throwable. What he did was this:

Create a new business exception, but not as an heir.
Then in the properties tab set Type Inheritance to "Behavior Inheritance"
And set Inherits Behavior to the Business Exception you want to inherit from

As simple as that.

BPM & Use Case, Who's Counting?

2009-09-28T23:15:00.003+02:00

(O)BPM and Use Cases

Business process modeling and implementation using the Oracle BPM Suite should be an iterative process. Because of that you try to postpone the need to create (paper) specifications as long as possible. After all, your running BPM prototype is the specification. Otherwise, while you are iterating, the paper specification would need to be changed as well, resulting in a lot of overhead which would not add much value, but cost the more.

However in some situations it me be necessary to create (preferably nearly after-the-fact) specifications, for any combination of the following reasons:

Acquire official approval of the detailed requirements
Support change control
Provide the base for test scenarios.

When your customer is standardized on use cases, that is a reasonable format for those specifications.

A Quiz

Now let's do a quiz and count use cases in a simple BPMN diagram that has been created with OBPM. For those that are not familiar with use case modeling: a use case captures requirements from the perspective of an actor that wants to achieve some goal using the system. So how many use cases do you count in the following business process, when for now you ignore the Receive Order by JMS activity?

The good answer is three. In case you had another answer, read on!

Counting Use Cases

The first use case is Place Order. The (primary) actor is the customer, and the goal is placing an order. The global interactive activity Place Order covers that, and will kick of the process. As a result the first thing that will happen, is that the system automatically notifies the account manager through the automatic Notify Account Manager activity. You may think this is the second use case, but consider this:

The notification is triggered by entering the order by the customer
The notification will take place immediately after entering the order, right after the user pressed the submit button.

So from the actor's perspective, sending the notification is part of the single-setting and a direct result of the actor's doing. The account manager is a secondary actor in this use case.

Still Not Convinced?

When your thinking was that the notification is part of the second use case, which by the way is Handle Order, consider this. The receiving of the notification by the account manager may have happened much earlier than the actual handling of the order by that account manager. So the notification itself is not an integral part of the single setting in which the account manager handles the order. So, yes receiving the notification is the trigger of the Handle Order use case, but not part of that use case itself (i.e. not the first step of the scenario).

For a similar reason as discussed for the Place Order use case, the (automatic) Create Back-order activity is part of the Handle Order use case. The creation of the back-order actually is handled through an alternate scenario of the Handle Order use case.

The last and final use case is the Ship Order, obviously.

Requirements vs Solutions

Now let's review the Receive Order by JMS activity, and let's assume that this only supports a different channel through which the same customer can place an order. The JMS queue may for example be used to pass in some SMS message. From a requirements perspective you can state that this is part of the Place Order use case. The argument being that using a OPBM screen or sending a text message are just two different solutions to the same actor goal: placing an order. Supporting different channels is "just" a supplemental requirement for the Place Order use case.

Now if you really think about it, you probably will agree with me that when doing a more detailed analysis of the Place Order use case, will reveal several reasons for (in the end) having at least two use cases, one for placing the order through a screen and another one for placing it by SMS. If you are a fancy use case modeler you could even model the Place Order use case as a super-type use case, having to siblings: Place Order through Workspace and Place Order by SMS.

So when you had "four" or "five" as your answer, depending on your reasoning, you might have been correct as well.

Introducing a Timer

Another interesting aspect to consider is what happens when the Notify Account Manager activity is not done directly after the customer issued the order, but for example by a trigger at 8:00 in the morning. In that case, the notification is no longer part of the single setting Place Order.

Mind that for the reason explained earlier, this still does not mean that the notification becomes part of the Handle Order use case. But who is the primary actor here and who is having the goal? In this case the primary actor is the timer. It depends on how you formulate the goal, but for reasons of simplicity, let's assume that it is the customer having a goal of the account manager handling the order as soon as possible. But granted, it is debatable.

Conclusion

So, in conclusion: unless there is a timer involved, all automatic activities are part of the interactive use case that precedes them, which may be outside the OBPM process (in another system sending the message).

Oracle Enterprise Repository Installation Issues

2009-07-21T21:24:00.010+02:00

The other day I installed the Oracle Enterprise Repository 10.3 on my laptop. In principle the Oracle Enterprise Repository, or OER for short, is a web application featuring a rich user interface that requires Java Web Start.

Apart from the fact that the order of the steps in the installation guide is not exactly as you want to follow them, the installation on Windows XP is relatively easy. However other than for simple tools like TextPad or Total Commander for some reason in my case installations are not supposed to be bump-free (some higher God is making sure they are not). With the finish line in sight, I already thought this was an exception to that rule.

I started the application and the welcome screen rendered OK, the final thing to do is to click on Edit/Manage Assets, and then ...

Yes, you see that right, the browser equivalent of a blank face. Now what?

As we all know, logging and JavaScript are the perfect couple, so other that checking, double checking / triple checking all the steps made, I had no clue what to do. Except for contacting a colleague who just might have ran into the same issue, and surprise surprise, she had! Clever as she is, she already found out that Java Web Start of Java 1.6 and OER 10.3 are not the best friends.

So like in her case I was able to fix the issue by disabling Java 1.6 in the user Java runtime settings, et viola!

Fortunately for me she has better things to do that blogging about silly installation issues (like taking care of a baby).

OUM & Software

2009-06-19T13:30:00.003+02:00

Some time ago I got a question in a comment if the Oracle Unified Method (OUM) is or will contain any software components like we used to have with CDM (like CDM RuleFrame and Headstart).

The answer is no. And the reason is as follows.

CDM was restricted to Oracle Designer/Developer and the usage of the Oracle Database with that. OUM is not based upon any specific tool set, just simply because there are that many that (currently) it simply is impossible to do so. Maybe someday, when we "fused" all our products into one consistent tool stack, who knows ... But I don't see that happing in a future near me.

On the other hand, I hope we will be able to add some technology and even tool specific guidance to OUM for some of the tools we have.

Oracle BPM and Java objects

2009-06-19T13:10:00.003+02:00

Why using Java for business objects?

When creating business objects for Oracle BPM (10.1.3), I have a couple of compelling reasons for basing them on a Java class model. The reasons being:

Ease of migration
Testing
Reuse

At the end of the day, Oracle will get rid of the proprietary PBL language and target for Java instead. So building your business objects on top of a Java class model, might be a good strategy to ease migration to any one of the new versions.

Oracle BPM leverages CUnit (and PUnit), both being proprietary testing frameworks. When you base your business objects on Java, this also offers the opportunity to test your objects using the de facto standard unit testing framework JUnit.

BPM business objects can only be used by BPM. A Java class model can be reused almost anywhere, including BPEL, and (obviously) Java applications.

How using Java for Business Objects?

Just follow the instructions for catalogueing components in your catalog. To leverage the extra functionality that BPM business objects offer (including creating object presentations), create a BPM business object for every top-level object type that you need, as an heir of the Java class. Don't do them all, only the ones you need the extra functionality for.

I put the Java objects and BPM objects in the same catalog, and name the BPM objects after the Java classes, post-fixed by "BO", for example CustomerBO for the Customer Java class.

Finally, to ease synchronization between JDeveloper (or your preferred IDE) and Oracle BPM, after first-time catalogueing, generate the jar file with the Java classes directly to the lib folder of your BPM project.

Service Oriented Confusion (SOC)

2009-03-18T23:05:00.007+01:00

In a previous article I presented a service taxonomy. As I explained over there, deciding upon a proper service taxonomy supports a proper service layering. I should have said this is only one of as set of taxonomies (plural) that you could use, even at the same time.

Moreover, I could also have explained how different service taxonomies can help the process of service discovery. For example, a taxonomy along business domains (like Customer Relationship Management) can help to get an overview of all services provided by a specific business domain.

But I would also like to state a word of caution here. I have seen organizations using an IT related taxonomy including classes like data services and application or utility services in the communication with business analysts. The problem is, that to a business analysts such a taxonomy is difficult to understand, and may look pretty arbitrary. Where does a data service differ from a business service? Do they not both deal with "data"? Yeah, but eh ...

Apart from that, what value does it add to them? Why should they care? I can't tell you that to be honest. What I can tell you is that I've seen situations where business analysts only thought they understood and started to do analysis. The damage done ...

It is my experience that the only two classes of the taxonomy that I presented in my previous article, and that make sense to the average business analyst, are process service (a service implementation of a business process) and business service. From an IT point of view, a business service may translate into one data service, utility service, or some composite service that orchestrates two or more data/utility services, etc.

I've been with an organization that does not strive for business services to be reusable, because in their definition, a business service is a service that supports a specific business purpose, that may be unique across the organization (as shown in the picture above). However, from an IT point of view the services used to construct the business service definitely should be reusable.

Now this may or may not work for you, so you may want to use a different taxonomy for layering. As long as you make sure that it clear what kind of taxonomy is used for what kind of stakeholder, and you only use a taxonomy that makes sense in the universe of interest of the stakeholder.

If It Ain't Broken, Fix It!

2009-02-06T21:27:00.002+01:00

For already more than three years I'm the Java developer of some Oracle-internal sales application. Up to that point the application had been maintained by several people who obviously had different development styles.

I'm more a kind of guy that practices principles like coding by intention, responsibility driven design, and refactoring with the main goal to achieve a simple design that is easy to understand and therefore better to maintain. Those principles had not been on the top list of my predecessors. The result being that, to implement a change request or bug fix, you almost always had to debug the code and do extensive code reviews before you even knew where to start. I'm not joking when I say that the same kind of functionality was never implemented twice the same way. There was hardly any pattern to discover, except anti-patterns and a lot of bad practices:

lengthly methods, often of 100+ lines
names for classes, attributes and methods that do not resemble what it does
classes and methods having too many responsibilities
no proper layering and separation of concerns
and so on, and so forth

At a specific point in time I had (and took) the opportunity to start refactoring key parts of the application, and never stopped since. I did refactor for the sake of refactoring, but restricted myself to the code that I needed to touch anyway. While doing so I tried to apply best practices and extrapolated these to other parts of the application to keep the solution consistent.

The result being that after less than half a year, newcomers needed only half the time to understand the application good enough to do their job. And those parts that had been refactored properly, required only half the time to change. Cross my heart: I dare to state that any investment in this making the application code base simpler, has paid back itself several times already!

Many developers will recognize this situation, and may want to do something about it, but run against a brick wall called "project manager", who is just a plain idiot, not understanding the value of good coding practices, but only focusing on getting the job done in as less time as possible, right? Wrong! The key responsibility stake of the project manager is exactly that: delivering on time and within budget, and most of them are doing that job pretty well.

Unlike what you may think the project manager is not the key stakeholder, but the customer is. So rather than trying to convince the project manager, you must make the customer aware of how a simple design:

will reduce their maintenance costs (which still is about 80% of the original development time) and perhaps even more important,
will make their IT better aligned with the business, because it will be quicker to change a well designed system than spaghetti.

Try to convince the customer that to achieve this, a simple design is an absolute must, and therefore from time to time you need to clean the kitchen because otherwise at some point the kitchen will get so dirty that almost any meal you make in there, will be a serious health hazard. That cleaning the kitchen is what refactoring is all about.

Even when the project plan is already finished and the budget already fixed, it is likely that there still are plenty of opportunities to help the project manager with reducing the remaining development time, if only you take the time to find them. The biggest problem for applying best development practices is probably you not spotting these opportunities or not being able to communicate them well.

OUM Available for Customers

2009-02-02T10:02:00.002+01:00

Finally, the Oracle Unified Method (OUM) is available for all our customers! Until now it was only available for Oracle Certified and Oracle Certified Advantage Partners, but that has changed now.

Customers that contract Oracle Consulting for an engagement of two weeks or longer can download OUM if they meet some additional criteria. Contact your Oracle Consultancy sales representative about the details.

Be aware that soon we can also deliver training on OUM (we already do so internally), so don't forget to ask for that too!

Service Taxonomy and Requirements

2008-12-19T18:47:00.006+01:00

Having seen how organizations can struggle with it, convinced me that before you start to capture the requirements of a serious amount of services, you rather have a proper classification scheme identified up-front. A service taxonomy as such a classification scheme is called, is important for a couple of reasons:

The requirement attributes are likely to differ somewhat between classes, making that you might want to use different templates,
A service taxonomy supports a proper positioning of services in the application and technical architecture,
A service taxonomy supports a proper layering of services.

There is no single best taxonomy, as it depends on aspects like the number of services, and the nature of those services. However, a good starting point could be a classification like the following one.

First of all the following (more or less atomic) classes of services can be distinguished:

Business Services that are recognized as such by the business, like Retrieve Customer Profile, Print Order Confirmation.

Data Services that query or store data, like Retrieve Customer, Retrieve Customer Orders, Update Order.

Application Services, being all other services, like Print File and other 'utility' services.

The above classes concern atomic services, meaning that the services have a restricted, clear responsibility and therefore in principle are re-usable. When used in the context of use cases (or business processes) they typically will not cross steps.

Whenever applicable, you can also recognize Process Services as being the type of services that deal with service orchestration. Process services typically have a specific use case / business process as a scope. That use case can be a user-goal or summary use case (spanning multiple lower level use cases).

Batch Services are a specific kind of orchestration services that process multi-record messages in an asynchronous way.

Other Taxonomy Aspects

Your taxonomy can leverage more than one kind of classification, for example one that organizes business and process services by business domain.

Also different levels of services could be distinguished. This is where we talk about service granularity. For example, a higher level data service Retrieve Customer Data can consist of two lower level services Retrieve Customer and Retrieve Customer Address. The level of a service won't have much impact on the way you specify it functionally though. Also, specifying finer grained services is more likely to be the subject of analysis and design rather than requirements definition. The discussion about service granularity for requirements definition is likely to be that of user-goal versus summary use cases.

Be aware that the taxonomy you use during requirements does not necessarily need to be 1:1 with the one that is used for analysis and design. The meta-requirement for the taxonomy used during the requirements definition, is that it should be recognizable by business people. There should be a clear mapping to the taxonomy that you will use during analysis and design, though.

Generic Requirement Attributes

The requirement attributes of all service could consist of the following:

name
purpose (one, two sentences max)
description (what does it functionally, i.e. without getting technical)
type (within the taxonomy)
triggering events
pre-conditions
post-conditions
synchronous/asynchronous (request/response vs fire & forget)
message format in
message format out
list of values (for attributes where applicable)
transformation
validation of incoming message
list of (logical) errors that can be the result
metrics
business indicators (e.g. to collect BAM metrics)

Typical standard pre-conditions (that you might decide not to specify but capture as an architectural principle) are:

consumer has been authenticated and authorized to use the service
message format in complies to the XSD, meaning failing to do so will result in technical error

Message format in and out have the following sub-attributes:

structure
attribute names
data type
format mask
mandatory

Attributes can also have a list of values that applies. List of values can be static (i.e. a list of fixed values, typically implemented in XSD) or dynamic (to be implemented with DB-query). Static list of values should not be captured in the service descriptions itself (for reasons of maintainability) but in a separate document.

Transformations typically are captured using a two-column-format, one column describing the "query" (just one source attribute, or multiple attributes with some operations), the other the target attribute.

The validations to specify concern checks that are not considered to be part of the pre-conditions.

Metrics can have the following sub-attributes:

average calls/per time unit
peak volumes
max response time

A service can call one or more other services, each having their own message format in and out and transformations involved in that, which then also needs to be specified.

Specific Attributes

In case of Data Services also the following attributes can be applicable:

data source (name of the database, or other)
a (restricted) data model
key attributes
(logical) query, insert or update statement

Application and Batch Services typically also can have the following attribute:

algorithm

Process Services also have a process orchestration that is described using an activity diagram (preferable using the BMPN notation) or a sequence diagram. Process Services often have a human interaction being involved.

Generic Supplemental Requirements

In case of asynchronous services you are likely to have a need for an "error hospital". Preferably you have some generic mechanism for that. For each individual service you then only need to specify how long and with what frequency a retry needs to be done, before it will end up in the error hospital. The requirements for the error hospital itself will be generic, describing how a systems or application administrator can cancel or retry the service (when possible).

Other generic supplemental requirements might concern:

audit requirements
logging
security

Per service you only specify how the generic mechanisms apply.

Oracle BPM Enterprise 10gR3 for WebLogic on XP

2008-12-10T09:21:00.001+01:00

Officially Oracle BPM Enterprise 10gR3 is not supported on Windows XP. When you try to install it using the default path, you probably will experience some exceptions like a ClassDefNotFound or an IllegalStateException. So what to do when you're bound to XP but want to juggle with BPM Enterprise nevertheless? This is how I did it. But remember it's not supported, so don't start complaining when it does not work for you.

What I did was to run the installer up to the point where it asks you to start the configuration wizard, and then quit. The configuration wizard offers the option to install a WebLogic Server (WLS) for you. However, I created a WLS domain myself, with only an admin server and no managed server, and called it OBPM (but you can call it DonaldDuck or whatever you like). Once successfully created, I started the server.

The OPBM Enterprise configuration wizard has an option that let's you choose to change an existing WLS domain rather than to create a new one, to deploy the process engine and web applications on. So I chose my OBPM domain.

The ClassDefNotFound error I prevented by not choosing to install the Oracle BPM transport provider. You can install it later, and perhaps when you concentrate on that, the ClassDefNotFound error can be tackled as well. I did not yet get that far. I explicitly configured the tablespaces of the engine and directory schema's, but had to key in the names in upper case (e.g. in my case USERS and TEMP, instead of users and temp), otherwise the installation of the schema's failed telling me that the tablespaces did not exist. Weird.

Anyway, using the installation path as I described above, resulted in a flawless installation on my XP Professional

Need to know more? Check out the Oracle BPM 10.3 Configuration Guide.

Dynamically Switching Rules Dictionary Versions

2008-12-05T18:41:00.003+01:00

Suppose that you are using Oracle Business Rules in a Java application. And suppose you want to be able to change and test your rule set without bothering other people. Wouldn't it be nice to be able to work in a copy of a dictionary and activate that only for your user session? This can be achieved as follows.

I assume one dictionary of which there can be multiple versions. Of course the same principle can be applied to multiple dictionaries.

First implement some mechanism for storing the default version of the dictionary, and that will be retrieved every time before calling the rules repository. I use a properties file for that, because you can change a properties file dynamically. Then have some page from which the user can pick an available version from a list. Once a version has been chosen that differs from the default, that will be the version used for that user session.

Now how to get the versions of a dictionary from a repository? That can be done using the following Java code snippet, which is based on a WebDAV repository:


// repositoryURL is a String containing the URL to your repository
// rulesDictionaryName is a String containing the name of the dictionary

RepositoryType repositoryType =
RepositoryManager.getRegisteredRepositoryType("oracle.rules.sdk.store.webdav");
RuleRepository rulesRepository =
RepositoryManager.createRuleRepositoryInstance(repositoryType);
RepositoryContext repositoryContext = new RepositoryContext();
repositoryContext.setProperty("oracle.rules.sdk.store.webdav.url", repositoryURL);
rulesRepository.init(repositoryContext);

dictionaryVersions = rulesRepository.getMarkerNames(rulesDictionaryName);
for (int i=0; i < dictionaryVersions.size(); i++)
{
dictionaryVersions.set(i, ((String)dictionaryVersions.get(i)));
}

When the test was satisfactory, the chosen version can be stored in the properties file, making the default for every new user session.

Moving a WLS Domain

2008-12-01T08:55:00.000+01:00

The following describes how to move your own sandbox kind of WebLogic Server domain. This method is not supported and should not be used to move any other kind of domain. It is also not guaranteed that any applications deployed on it, will move flawlessly with it. And it I tried it only on Windows.

So why describe it anyway? Well, just like me, you might have not payed attention when creating a domain and it therefore may have ended up in your product home directory. And you might, just like me, be too lazy to want to run the wizard and create a new one. Not that it is so much more work to do so. Finally, just like me you might be interested to know where WLS stores this kind of information.

Moving a domain can be done as follows:

Copy the domain directory and everything under it, to the location where you want it to be
Search all the files in the domain directory for the fully qualified path name where the domain was located (e.g. d:\oracle\product\bea\domains\MyDomain, and replace that by the fully qualified path name where you want the domain to be (e.g. d:\wls\domains\MyDomain). You will find a couple of files, among them .cmd files.
Find the nodemanager.domains file (in my case in the d:\oracle\product\bea\wlserver_10.3\common\nodemanager\ folder). Find your domain in there and modify that manually (e.g. to d\:\\wls\\domains\\MyDomain)
Finally, change the short-cuts to start and stop your domain to point to the correct locations.

That did it for me!

To BPA or to BPM, that's the question

2008-10-24T13:55:00.001+02:00

The other day I heard Thomas Kurian talking about the future of the Oracle technology stack. Among other things, he mentioned the Oracle BPA Suite and the Oracle BPM Suite (fka BEA AquaLogic BPM). According to Thomas, the "structured" BPA Suite aims at rigorous process modeling and simulation, where the "agile" BPM Suite aims at iterative process modeling.

Methodological me tended to hear that he was saying that the BPA Suite was more suited for plan-based development, where the BPM Suite would better support agile development. Knowing both of these tools, I found that a somewhat rigorous statement by itself, so I dug into the subject somewhat more.

Yes, the Oracle BPA Suite has a huge amount of modelers and an astronomical amount of symbols. And yes, being able to use the tool to its full extend requires a learning curve that looks the inverse of the stock markets of these days. But does that mean you cannot use it in an agile way? Not really, because being agile is less about how long it takes to learn something, but more about being able to apply it to create new or change existing software.

On the other hand, would the Oracle BPM Suite not be suitable for rigorous process modeling and simulation? Again I don't see why. You could easily have a very rigorous way of developing systems with strict procedures to follow, including an extensive QA process and still use a tool like the BPM suite.

Maybe we need to have a look at the subject from a different angle. Let's start with a limited feature comparison. Limited in that I restrict it to a development environment based upon a SOA with a focus on the high level topics of the models that are supported, how Governance can be enforced, and the type of systems that you can build.

Models and Governance

In a nutshell, with the BPA Suite you can start by defining business processes from a high-level context diagram up to a detailed design. At the higher levels you can use Value (Added) Chain Diagrams, or VACD's for short, that at a specific point map on BPMN diagrams. Those BPMN diagrams can then be transformed into a BPEL design, which finally can be implemented using JDeveloper. The higher-level modeling can be done by Business Analysts while the detailed BPMN model typically will be done by someone with a developers background.

With the BPA Suite you can define roles and restrict access to specific modelers of the tool. This allows you to implement a rigorous Governance model you might have, where the responsibilities for the different roles are well supported by the tooling.

With the BPM Suite you can define processes using BPMN (or similar notation, each being just another view on the same source). Next to that three different "profiles" are supported: Business Analyst, Business Architect, and Developer. The only difference being that a Business Analysts sees the least and a Developer the most details of the same process.

Unless you start doing really fancy things, the BPM Suite does not support a rigorous Governance model as far as responsibilities are concerned, as nothing prevents a Business Analyst from changing his profile to developer to see what has been coded under the hood and even change that.

So the BPA Suite supports a rigorous enforcement of Governance and capturing of requirements, where the Suite BPM supports that hardly if at all. But as such I don't want to call that an "agile" aspect of the BPM Suite.

Regarding Governance, there is one other significant difference. The BPA Suite itself does not result in an executable model. BPMN can be transformed to BPEL, but that is only a blueprint. After sharing the blueprint with development (as it called), a developer picks up the BPEL blueprint and implements it.

The BPM Suite leverages XPDL which provides a means to draw business process models, but also is an executable model. So for the BPA Suite you could say there is a strict hand-over from design to implementation, where with the BPM Suite there is no such thing.

Type of Systems

Currently the BPM Suite clearly is coupled to BPEL. In our practice BPEL is heavily used to integrate systems, with little to no human workflow involved. But that does not mean that you cannot use it for processes that almost completely involve human interaction, and in practice it probably will.

Although at first sight the BPM Suite primarily seems to target processes that involve human interaction, nothing will keep you from developing processes that are fully automated. BPM processes can be deployed as services with a WSDL interface, and call other services, allowing it to be used for service orchestration only.

My conclusion is that regarding the type of system you can build, there is not really a significant difference between the two. Although with the BPM Suite it is much easier to create a user interface than with the BPEL human workflow. But are processes with a lot of human interaction agile and processes that are not rigorous?

Conclusion

When talking about rigorous versus agile and iterative, the significant difference I see is that with BPM it is hard if not impossible to build a solid wall between analysts and developers. And yes, it is true that agile development methodologies do promote breaking down that wall wherever possible.

By the way did I already say sorry for the lack of pictures this time? No? Sorry!

AIA & Canonicals, Just Good Friends?

2008-10-09T18:39:00.001+02:00

Last time I talked about the Canonical Data Model (CDM) pattern. What I haven't discussed yet, is how this pattern is implemented in the Oracle SOA Suite.

Actually the SOA Suite itself does not explicitly support this pattern, simply because enforcing any specific data model should not be seen as the task of any infrastructure tooling, including the SOA Suite. Nevertheless, the concept of a common data model can very well be implemented for specific industries. The goal being that parties in that industry can communicate more easily because by using a CDM they actually talk a common language, at least to some extend.

This concept already is being used in various "industries". Examples are CDM's for governments allowing central and local governments to exchange information with each other about their citizens. CDM's in the health care industry allow doctors, insurance companies and other stakeholders to exchange information about patients.

More recently, Oracle introduced the Application Implementation Architecture or AIA for short. As in the communication about it the focus seems to be on the common process aspect perhaps a little bit of a hidden feature of AIA, but the industry foundation packs of AIA could not exist without the CMD's they are based upon. Perhaps the common data model is of even greater value than the common process.

Once defined a CDM should be accessible by every process that is making use of it, preferably by means of XML schemas. In case of the SOA Suite, rather than importing (and with that copying) the schemas in each and every BPEL project, a way to do this is by including all coherent XML schemas of a CDM in one single, dummy BPEL process, as explained by Marc Kelderman.

The Case for Canonicals

2008-10-06T11:37:00.002+02:00

You might have heard people talking about a "Canonical Data Model" or CDM for short. You might have even heard the rumour that having a CDM is critical success factor in achieving the true benefits of a Service Oriented Architecture. But in the meantime you still have to encounter the first situation in which it actually is being used. Is a CDM really a must-have or just another buzzword?

First let me try to explain what a CDM actually is, apart from just being one of the integration design patterns. In short you could say that a canonical data model provides a generic view on the structure of data that systems deal with, like for example a generic concept of what a Customer is, what attributes it should have and the data types and formats of those attributes are.

It might surprise you that having a common view of an entity like "Customer" often is far from common practice. Imagine a big organization like a bank having many systems with different purposes, because of merges often from different companies. Such an organization can easily have as many definitions in their data dictionary of "Customer" as they have systems that deal with customers.

Now what if such an organization needs to integrate all these systems with SOA using XLM transformations for that? If there are N systems to integrate, than in principle there are N * (N - 1) mappings possible for each type of Customer. In case of 4 systems that need to exchange customer data, that already means 12 mappings, as you can see in the following picture. But if you define one generic definition and map to and from that definition, than the maximum number of mappings are 2 * N. In case of 4 systems that means only 8.

A larger bank easily has hundreds of applications with dozens of different definitions of "Customer", let's say 30. Then the difference is 870 versus 60! And that is only for Customer, and there are plenty of other entity types that needs to be exchanged as well, like Account, Address, etc. You get the picture?

So the incentive to use the Canonical Data Model design pattern, is to reduce the number of mappings and with that the inter-dependency between systems, the complexity of the overall integration and, last but not least, the maintenance of all that. For larger organizations this can make a huge difference.

Having said all this, it probably never is the case that all systems need to integrate to each other, let alone that this all the time requires a two-way mapping of every entity involved. To know if an entity should have a definition in a CDM, it depends in how many mappings it will be involved in. When there are more than three systems that all need to exchange an entity in a bi-directional way, than a canonical definition of the entity starts to make sense.

The case against canonicals is that in some organizations it might prove to be far from trivial to get a common view of how the generic definition of a specific entity should look like.

About Business Processes, Use Cases & SOA (4 & End)

2008-08-22T23:46:00.002+02:00

This posting is a follow-up on a previous posting in which I explained what artifacts might be appropriate in case of creating an analysis model out of a set of system use cases for a SOA project.

In this third and last posting, I will show how the (platform independent) analysis model can be transformed into a (platform specific) design that leverages BPEL.

Design

The design involves mapping the analysis to artifacts that actually are implemented. Assuming that the Technical Architecture dictates a Service Oriented Architecture that uses BPEL to implement services and service orchestration, the design would at least include the following:

WSDL’s
A diagram design of the BPEL processes to implement
A database design (that supports retrieval of resident information and storage and retrieval of parking permits and their applications)

An example of an XSD already has been provided. Including WSDL’s in this case would not add much value, and a database design is too obvious. A design of the BPEL process to implement, in this case sufficiently has been provided by the activity diagrams already created. This probably will often be the case.

To show how the implementation might look like the following two BPEL process diagrams are included.
A part of the ParkingPermitProcess looks as follows:

Not surprisingly the initial business process diagram and the implementing BPEL process flow look very similar.

The same holds true for the user goal level Validate Parking Permit use case that looks as in the following picture (that actually zooms in on the ValidateParkingPermitApplication step of the previous picture):

In practice, you can imagine a human workflow step to be included that supports manual intervention of the outcome by Parking Services.

This ends a short history about the road you could walk from business process modeling to a BPEL implementation, and that with use cases as well. Now was that impressive or what?!

About Business Processes, Use Cases & SOA (3)

2008-08-14T20:55:00.002+02:00

This posting is a follow-up on a previous posting in which I explained business process models can be drilled down to user-goal level use cases and how to do service discovery based upon process models.

In this third posting, I will show how the requirements that have been captured so far, can be transformed into an analysis model that will provide the base to build the design on.

Analysis

What kind of artifacts should be created during the Analysis process depends on the nature of the use cases and the architecture that is going to be used to implement them. The most apparent candidates for adding detail are activity or sequence diagrams, class diagrams and (where useful) collaboration diagrams (used to describe how various “components” work together). For SOA projects also message descriptions and message transformations are obvious candidates.

For services that support more than one operation you also need to specify the names of each individual operation, as well as the types of their iand output messages.

Depending on its granularity, a service supports a summary use case (like the Parking Permit process), a user goal use case (like Apply for Parking Permit) or a subfunction use case. No example of the latter has been included, but you can imagine that for each separate channel (letter, email, SMS) a subfunction use case could be created that describes a generic notification services that can be used to send all kind of notifications to third parties.

Unless parallel development is be done where other (parts of the) system(s) depend on a service to be build, at this point there is not much value in specifying the exact WSDL of the service. For any external service you use, you need to have at least the WSDL location if not the WSDL itself.

Activity Diagrams

When there is a flow involved, it might be useful to create an activity diagram. An obvious example would be a use case for a service that will be implemented as a BPEL process. As already noted, the activity diagram for the Validate Parking Application is an example of that. But use cases that involve a user interface with a complex screen flow, also are good candidates.

Class Diagrams

When there is data involved, a class diagram seems to be the obvious choice to document that. Many people doing SOA projects never seem to make a class diagram. Realize though that class diagrams add as much value to SOA projects as they for example do to pure Java/XML projects, as messages are about handling data as well. Having class diagrams available can add great value to optimizing the process of defining message formats and transformations.

When analyzing the Parking Permit use case the following classes can be recognized: Resident, Parking Permit Application, and Parking Permit. A distinction has been made between Parking Permit Application and Parking Permit, as not all applications will result in a permit, and the application follows a process with statuses that do not apply to the permit itself and vise verse.

Not very explicit in the description but obviously needed if you think about it, is a notion of an Area. You will need this in the process of determining if there are sufficient parking lots available. You might argue that the reason for not having discovered this earlier is because the way the waiting list is being processed has not been worked out to the proper level of detail.

The class diagram for the Parking Permit use case could look as in the following figure:

Message Descriptions

What format to use for a message descriptions probably foremost depends on who needs to validate it. Some people will prefer more “logical” descriptions in Word, while more technical oriented people probably have no problems with XSD’s.

Mind that to be able to create XSD’s that translate 1:1 to an implementation, it is important to understand the technique of implementing messaging. For example, initially one XSD has been created for the Resident and one for the Parking Permit Application, only to discover that it was more practical to create one XSD containing both and use that as the format for the message going from the resident to the Parking Process service. If you pay more attention to detailing the Apply for ParkingPermit use case, you probably would find that out up front.

For the sake of example the response message has been kept simple by returning a string stating that the application has been received successfully. The description of the request message looks as follows:

Other message descriptions that need to be made are the request and response messages for the Verify Ownership, Verify Electoral Registers and Verify Residence use cases.
You should keep the message descriptions separate from the use case descriptions, to support reuse. Refer to them from the use case descriptions instead.

Message Transformations

How message are transformed from one format to the other depends on the situation. For example, in case of BPEL, transformations can be done by doing one or more assign operations or by using an XSLT transformation, which in both cases may involve complex XPath queries or regular expressions.

For this reason transformations probably are best described in text, as has been done in the following example:

The example is trivial in that transformation consists of using a subset of the fields of the source and mapping those 1:1 on fields of the target message.

You can image than in practice often more complex transformations need to be done, for example in case of n:m mappings. In practice using a two-column format often suffices, where the (left) Source/Transformation column specifies which source fields are involved and any logic that needs to be applied to that, and where the (right) Target column specifies what the (single) target field is.

As with message formats you should keep the message transformations separate from the use case descriptions to support reuse.

Where to Stop

Once the analysis model has been worked out to a sufficient level of detail, you are ready for the design. What “sufficient” means in this context, depends on many factors, the most important ones being the following:

Nature of Engagement

In some cases a formal process needs to be followed that requires every change of the specifications to be approved up-front. On the other side of the spectrum there is the agile approach by which part of the (detailed) requirements are captured while validating iterations of a working program.

Skills and Habits of Developers

Developers are most comfortable with what they are used used to working with. However, you should be aware that when every developer involved needs to get used to one broadly accepted way of creating specifications almost always outclasses the situation in which developers need to get used to as many different styles as there are other developers. Not to mention the effort that needs to be put into the validation process involved with that.

For most projects that involve requirements analysis, class models are very useful to developers, even in the case of BPEL development.

Application and Technical Architecture

The development of data-oriented systems that depend on a well-structured database can highly benefit from creating a detailed class diagram. In case of use cases that involve a user interface, activity diagrams normally only add value when there is a complex dialog involved.

For SOA projects that primarily deal with messaging it probably suffices to detail classes to the level that all attributes and their types are known. Message formats and transformations often need to be worked out in detail. In general, activity or sequence diagrams add great value to business processes and services, as they support a more effective implementation.

Other architectures, for example identity management/security, on their turn require yet other details.

A final remark that needs to be made at this point is that use case analysis should not be confused for a technical design. Although some people state that class models, and activity diagrams that describe system-internal behavior, are “technical” they actually only capture detailed requirements or validate higher-level requirements from a different angle.

To be continued ....

About Business Processes, Use Cases & SOA (2)

2008-04-11T20:02:00.003+02:00

This posting is a follow-up on a previous posting in which I explained how business process models and use cases can be used to capture requirements. It especially highlighted how use cases can be used for aspects that might not that easily be captured using business processes alone.

In this second posting, I will give an example how the business process model of the previous example is drilled down to ensure that all nodes in that model are of the level of user-goals, and what the benefit of that is. I will also discuss a way to discover services from the business process model.

Drilling Down Business Process Models

When looking at the business process of the previous posting, you might notice that each activity actually is a use case of its own, most of them being at the level of user-goal use cases (which is a use case for which the user can execute it in one session and walk away happily after that). The only exception might be the Validate Parking Permit use case, that for that reason has been drilled down further to investigate if this really is at a level of a user-goal.

The result is the following Validate Parking Permit Application business process model:

As illustrated in this model, a couple of other organizations are involved in the validation, possibly making it a lengthy process, and initially suggesting that the upper level activity indeed is not a user goal level use case. But what if the other organizations have automated services available that respond within seconds? What this shows is that the level of a use case in some cases depends on the options for implementing it. This certainly is true in case of a Service Oriented Architecture.

For the following it will be assumed that such services do exist, making the Validate Parking Permit Application activity indeed an activity at the level of user-goal use cases. In the context of the Oracle Unified Method this automatically classifies the diagram above part of the use case details rather than a business process.

However, the question if it is a business process model or just use case details, is just an academic one. What is important is that you have a clear idea of when a business process needs to be further detailed. It is advisable to drill them down to the level of user-goals, for the following reasons:

User goals provide the most effective level to start from and create system use cases from business use cases to describe what the system should do (*). For example, it is at the level of user-goal business use cases that you will decide whether a goal will be kept manual or supported by a system.
You typically either implement a user-goal level use case or you do not implement it at all, making it the optimal unit to estimate and trace requirements from there.

(*) Business use cases have the business as subject and are written by business analysts to document business processes. System use cases have the system as subject and are written by system developers to document how actors communicate with the system to achieve their goals.

Service Discovery

Assuming that the requirements so far point to a solution using a Service Oriented Architecture, it is imperative to discover the services in the requirements. This is not only important for creating the application architecture but also provides important input to the estimating process.

One of the reasons to do business process modeling in the first place is that it relates very well to a Service Oriented Architecture, which makes implementation of the business process itself possible. You can imagine that the Parking Permit process is implemented as a longer running process that may take weeks to complete. But that will not be the only service involved, as the Parking Permit process itself needs other services to complete.

There are several approaches to discover services. One of them being by reviewing a business process and identify where the transitions are from an activity of one actor (swimlane) to that of another. In general this indicates a candidate service. In the example the candidate services are in the activities:

Apply for Parking Permit
Notify Resident of Waiting List
Notify Parking Service of decision
Notify Resident Permit Ready for Collection
Collect Parking Permit
Verify Vehicle Ownership
Verify Electoral Register
Verify Residence

The Apply for Parking Permit candidate service obviously is the starting point for the Parking Permit process itself. You can imagine that the web site of the municipality provides some form that the resident can fill out and on submit calls the Parking Permit process.

The Notify Resident of Rejection, Notify Resident of Waiting List and Notify Resident Permit Ready for Collection (from Parking Services to the Resident) are candidate services whereby the Parking Service process needs to call out some provider that handles multi-channel notifications to third parties. Whether this will be one service that can handle some generic notification, or three different services that each handle a specific notification type, or any other combination, is a design decision and at this stage is yet to be determined. Mind you, we are analyzing the business and capturing requirements, not yet defining the solution!

The Notify Parking Services of Decision points to yet another candidate service that the Resident can call to get the application removed from the waiting list.

The Verify Vehicle Ownership, Verify Electoral Register and Verify Residence already have been indicated as existing services. However, if they were not, creating them would have been outside the scope of the Parking Permit process, as those services would then have to be created by third parties not involved in the project.

From the last example above, we can conclude that transitions to activities within the scope of secondary actors that are not within the scope of the project, only indicate calls to external services.

Prototyping the User Interface

Any use case that has a user interface associated with it, might be detailed with a conceptual screen layout. Note that a conceptual screen layout is not a design, as it does not specify an actual screen layout, let alone aspects like styling. It’s main purpose is to specify which data should be shown, and should be validated during the review.

Depending on if and when you start creating a Domain Model to identify the business entities involved, conceptual prototypes provide input to or can be use to validate that model.

For example the use case Apply For Parking Permit could be detailed with a screen layout as follows:

Other use cases that may require some conceptual layout are the following:

Notify Resident of Rejection (mail/email/SMS)
Notify Resident of Waiting List (mail/email/SMS)
Notify Parking Services of Decision (mail, screen)
Notify Resident Permit Ready for Collection (mail/email/SMS)

When analyzing the Validate Parking Permit Application use case, you may find that Parking Services wants to be able to override the decision that comes out of the verification, for example by blocking a parking permit for someone that appears to hire a garage box (which none of the external organizations would note). In that case that use case also would require a conceptual screen layout.

To be continued ....

About Business Processes, Use Cases & SOA (1)

2008-03-21T19:41:00.004+01:00

It is my observation that since the introduction of SOA (Service Oriented Architecture) many developers are kind of lost when it comes to capturing requirements and doing design. Quite a few people already got lost during the "giant leap" from relational to object-oriented analysis and design, and (around the same time) had to jump from waterfall to iterative, and more recently from plan-based to agile project approaches. But since SOA too many people have no clue whatsoever how to properly write down customer requirements instead of producing designs of some services (if they even understand the difference).

Apparently some of us are convinced that "legacy" methods like the Unified Process do not properly address the issue of services. What I sometimes hear (and I don't exaggerate) is that a service delivers some reusable piece of functionality that cannot be pinpointed to a specific use case, so use case are useless. The best they can deliver is some flow diagram representing an orchestration of some services and call that a "business process".

Even when I was still clueless about SOA, I already found that hard to believe. What has become so different about user requirements since there is SOA that we cannot capture them as we used to? Has SOA really changed the way we think about requirements similar to how the Star Trek transporter will redefine the concept of transportation in a future near us? Does this question sounds rhetorical to you?

With this article and a couple yet to come, I hope to shed some light on this matter and help you to become aware that the world still turns and it is not you who is spinning around. As an example of this I will try to explain a way (not the but a way) of how you can capture requirements in a business process, and from there go to use case descriptions and finally implement services based on that.

As you might expect from me I will present this in the context of Oracle Unified Method (OUM), but don't let that scare you off. The targeted development environment will be BPEL.

The Case

The case I will use is of moderate complexity. The following activity diagram documents a business process concerning the application of a parking permit by a resident of some municipality.

At this point it is important to point out that this is a true "business process", as it only concerns business level actors like people, or organization units, and totally abstracts from any system supporting it. At this point the whole process could be manual, for all you know.

I find that important to point out as too often I have seen the situation in which the "analysis" is limited to a description of how some system is going to be implemented rather than on capturing requirements. You should only skip that latter step when you do it consciously and the risk of missing something that proves to be important later on is relatively small.

What might surprise you, is that the same business flow can be documented as a use case description, typically of the scope "enterprise" (that is cross organization-unit). In this case it also concerns a business use case, as it describes how the resident communicates with Parking Services, which in this context is an organization, rather than how this resident communicates with some system.

The Use Case

Presented as a use case the same business process could look as follows:

Normally a use case will not be created in one blow at this level of detail. It is more likely to be created in a few iterations of which the first one might suffice to document only the stakeholders, goal and main success scenario together with some business rules. In OUM this is what the Business Requirements process is supposed to be limited to. The example use case has been worked out to a greater level of detail, which would typically be done in the Requirements Analysis process.

How Use Cases Add Value

Compared for example with the situation where only a business process diagram is available, adding use cases adds value in that a (detailed) use case description provides the opportunity to specify aspects of the business process that goes beyond that of what you can express with only a diagram.

One of those aspects is stakeholders that are not directly involved in the process. In some cases identifying such stakeholders might give reason to extend the scope of the use case, like identifying the stakeholder Neighbor might have lead to considering to include in the business process a step that notifies them as well.

Other aspects that the diagram does not cover are the goals, the triggers, preconditions, and postconditions of the use case. All of them helping to validate the use case and some of them providing useful input for the next step in which the use case is going to be analyzed.

An advantage of putting it down in writing that may not be so clear at first sight is that by forcing yourself to express the business process in words might help you to discover flaws and omissions in the original process.

The value of putting effort in converting a business process into a use case description is not always easy to determine. It depends on many factors like the ability of people reviewing the business process to identify flaws in it, or how easy it is to correct errors later in the process. Probably the best advice would be: when in doubt, don’t hesitate and just do it. In case of the example the business process has been converted into a use case description in less than half an hour. The total process took much more time, but that was because the business process model needed to be corrected, because of errors found while describing the use case.

To be continued ...

No Country for Old Men

2008-02-18T20:35:00.002+01:00

With the risk of giving you the impression that I go to the cinema every week, I want to tell you about the movie I saw this Saturday. Still not the Kite Runner, as I went with my wife and she already saw that one. Instead I saw No Country for Old Men.

It is dangerous to compare it with anything else, but if I have to I would compare it to a gloomy version of Pulp Fiction. However, that resemblance is limited to the level of violence being used combined with a peculiar sense of humor. There the resembles stops as quick as it started.

Perhaps it is my lack of imagination but there was only one scene that I found to be predictable. For the rest of it I had no clue where it was going, and that pleasant feeling lasted until the end. I can't mention any character that was not convincing, including the side-kicks. Come to think of it, except for the hunter and the hunted every other character made a pretty "normal" impression. That is, as far as people from Kansas can be considered to be normal (no offense meant).

On a scale of "worthless" to "genius", I gladly give it an "excellent".

Cloverfield

2008-02-11T22:11:00.000+01:00

Yesterday I went to the movies as a kind of birthday present with a nephew of mine who recently came of age (being 18 in his case). Offered the options to either go to The Kite Runner or Cloverfield, for some reason he choose the latter. Now why would that be?

Pehaps the best way to describe Cloverfield is as a crossover between The Blair Witch Project and Godzilla, inheriting the better aspects of both of them. The whole movie is shot with a hand-held camera. Somewhat disturbing in the beginning but I got used to it pretty soon. Bending your head to the left or right on a regular base is a good exercise anyway.

It already has been described as an exponent of the YouTube generation, and some critics state that it redefines the concept of monster movies. Only time can tell if they are right. What I can tell you now, though, is that you don't need to be afraid of too much character building (there is none in any significant way), and in a pleasent way the monster is more a sidekick than the main dish of the story.

If you already saw the Kite Runner, you might consider Cloverfield as your next target. It's worth that big screen.

Name-value Pair Transformation in BPEL

2008-01-30T21:44:00.000+01:00

A pattern you often see is that some generic message type containing a number of 1..n name-value pair attributes needs to be transformed to specific attributes of a specific message type. For example, a portal of a some local government accepts a range of service requests that need to be transformed to a specific request for a passport, a parking permit and so on, to be able to be processed by specialized back-office systems.

The generic message type could for example be defined in an ServiceRequestForm.xsd as follows:

As you can see, the name-value pair attributes are passed on a a repeating set of formItems.

Let's assume that the specific type of message it needs to be transformed to, is based on the value of the serviceType attribute. In case the value of that attribute is "passport request" the ServiceRequestForm needs to be transformed to a PassportRequest.xsd consisting of the following attributes:

Now let's assume we are using BPEL to process the messages and let's assume no performance reason is keeping us from using an (XSLT) transformation (Transform) instead of a series of copy actions in an Assign. So for passports we create a transformToPassportRequest.xsl in which we would like to map the formItems at the left side of the following picture to the specific name, address and city attributes at the right side.

The simplest way to achieve this is by creating a so-called user-defined XSLT template that takes the name of an attribute as argument and transform the corresponding generic name-value pair attribute to the specific attribute. To this purpose we put a template called "getItemValue" at the bottom of the XSL file, as follows:

After you have done so, you will will find the getItemValue template in the User Defined Named Template drop-down in the component pallet of the XSL editor.

After you dragged & dropped that template from the component pallet in the middle section of the mapper, you can create a link from the formItem attribute to the template and from the specific attribute to the template, as has been done for the address attribute in the following figure:

To make that the template transforms any formItem attribute for which its name attribute has the value "address" you need to pass in 'address' as argument.

In a similar way you can transform formItem to the two remaining attributes, finally resulting in something like the following:

Ah... if everything would be as simple at this!

Where To Download OUM

2008-01-14T18:23:00.002+01:00

Edited on August 2, 2011 with the correct link and partner types.

As a follow-up on my previous post about the Oralce Unified Methodology (OUM) being out, you can download the method from here.

Mind that OUM is only available to Gold, Platinum and Diamond Partners. On the download page is an email address that can be used to ask for a password that you will need to be able to unzip the OUM download file.

There is a known "bug" with OUM, being that if a Word work product is exchanged between two people that have not installed OUM at the same location (by default C:\method), the person receiving the template will get an error when trying to open the document, saying that the template (.dot file) cannot be found. The solution is to make sure that everyone installs OUM at the default location and by letting people that have not done so, uninstall and then reinstall it at the default location.

OUM Is For Weenies, Real Developers Create Their Own Method!

2008-01-10T16:49:00.000+01:00

When I encounter a situation where people could greatly benefit from a methodology to structure their development process, and suggest to have a look at Oracle Unified Method (OUM), I often get as a reaction that OUM is too big and too complex.

Funny thing is that I hear this often being said by people that some years ago took months if not years to learn methods like the Oracle's Custom Development Method (CDM) and Project Management Method (PJM). Sometimes it is even said by people that never had a look at OUM themselves and only judge based on hear-saying.

But what would be hilarious would it not waste so much resources is that, rather than pick a method of the shelf, people start working on some method of their own. It usually starts small, initiated by an imperative need to write down a specific procedure and capture some standards & guidelines etc. But from there the library grows and before you know it they have spend man-months on defining their own method, spending a multiple of the budget it would have taken to adapt OUM, including a customization to the organization specific needs.

Therefore guys, before you start inventing your own wheel, have a look at OUM and check if that can make you start rolling.

Publish Your (BPEL) Process

2007-12-19T15:59:00.000+01:00

As sometimes silly things can take a disproportional amount of time, I thought I share with you what I found on documenting a BPEL process diagram.

Agile as I am (ahum) I try to prevent putting too much in written documentation as possible. The reason is obvious, in general after the initial publication documentation is hardly ever used again, and its expiry date passed before you know it. Among others, ways to prevent superfluous documentation are an iterative development involving end-users, simple but effective design, coding standards, effective naming strategy, and sufficient inline comments.

I think there will be not many occasions making it worthwhile to publish a BPEL process diagram, as developers rather open it in JDeveloper and other people probably will not understand it anyway. Actually, the only occasion I can think of is when you want to document how to create a BPEL process, like in a how-to, or like in my case, where I want to document how you can get from business processes via use cases to an actual implementation (in some posts to come more about this subject).

Anyway, if you feel the need for whatever reason, this is how you can do it.

Unlike the UML diagrams, for BPEL process diagrams there is not a menu option like Model -> Publish that you can use. In finding out how to do it I first tried the hard way of course, Googled on it and read online documentation to no avail. It was only then that I did the obvious and had a second, better look at the interface and found out a couple of "mystery buttons" at the top of the diagram. Of course, I had seen and actually used two of them before (for validation), but the others? Never gave them a thought.

Obviously there is a print button, but to the right of that sits a camera-shaped icon Create JPEG Image, which (surprise, surprise!) creates a JPEG image! There are a few other buttons
that makes me curious to find out what kind of people actually wants to use them, but also another one that I can picture to be useful, which is the pallet-shaped icon "Diagram Properties".

This pops up a window with various options to "optimize" the diagram (still need to find out what that means, as trying it did everything but that), options for coloring, and so on. An interesting option is the possibility to add annotations that (as the online help states) "provide descriptions in activities in the form of code comments and name and pair value assignments". If you have experience with that, drop me a comment as I sure would like to know.

OUM is OUT!

2007-12-04T17:52:00.000+01:00

It's official now, the Oracle Unified Method, or OUM for short, is available outside Oracle! Currently only for Certified Partners, but it is a start.

Now why should you care? Well you might care when you are in an organization that uses (part of) the Oracle product stack and:

Already uses the Rational Unified Process, but having difficulties with figuring out how to use the Oracle product stack with that, or
Is not yet using any formal method, but clearly recognizes the need for one, or
Has a need for some method that covers more that "just one project" at a time.

As I always explain, OUM is for Oracle what RUP is for IBM / Rational, and more. Next to covering the Unified Process, OUM also addresses issues that are specific for the Oracle tool stack, and goes beyond the Unfied Process by addressing cross-project, enterprise-level issues as well.

Oracle tool stack specific issues are covered in three ways. First there is the concept of supplemental guides. With a supplemental guide extra guidelines are added to the method that cover a specific architecture or product. For example, a supplemental guide can add extra, specific tasks. Currently there is the SOA Supplemental Guide, and work is being done on the Identity Management and a Business Intelligence Supplemental Guides.

Secondly Oracle tool stack specific issues are covered in task guidelines, for example by giving examples of how to execute a task and how the tools fit in there. Also, the task guidelines can provide links to more information, for example online-documentation or white papers (of which a few have been written by me). A bit similar to the so-called tool mentors in RUP.

I hear some of you say, "Duh, you already had something similar with CDM for Designer/Developer, so big surprise (not)!". But then you are not fully appreciating the magnitude of the issue we have with a huge (and still growing) tool stack. It's not just Designer/Developer anymore. And to be honest, because of this magnitude OUM is far from what CDM used to be for Designer/Developer where it comes to offering tool-specific guidance. But we will get there, one day. After all, there is a limited number of IT companies in the world, and although you might think other wise, also a limited budget, so it will stop somewhere eventually.

As I said, OUM also covers enterprise-level issues. For that we have added what we call "Envision". Envision is not an extra phase. It's more like a collection of processes that cover aspects that should be dealt with not only in the context of a project. Instead, they should be handled at an enterprise level.

The process currently covered by Envision are:

Enterprise Business Analysis
Adoption and Learning
Enterprise Architecture
IT Portfolio Management
IT Governance

In the future we might also cover Operations and Support, for which we currently refer to ITIL.

As the following figure illustrates, Envision kind of "folds" around projects.

An Envision process has to start somewhere, ideally within the context of a project on its own. However, in practice it is more likely that the process will be initiated within the context of the first project delivering artifacts that at some point in time are recognized as needing to be at the enterprise-level. For example, during some project a technical architecture might be created that turns out to become the start of a reference architecture for following projects.

Finally, Envision also provides the glue between processes or methods that already exists within the enterprise and a project. I hope this glue is strong enough to let this message sticks with you.

Check Your SOA

2007-11-30T15:27:00.000+01:00

Finally found the time to finish this posting which I started to write like three months ago! It's about time, as it might not take too long before it's content will start to get more and more obsolete.

Some time ago I was asked to check whether the SOA Suite had been properly installed at a customer site. So what do you do when you are asked to check something? You go and look for a checklist! At least that is what I do, and not verify surprisingly, found nothing. As you might recall from an earlier article I have the memory of a gold fish, so to prevent that I need to reinvent that wheel some next time, I thought I write it down. If not for you, than for myself.

The following describes a checklist regarding the installation of the Oracle SOA Suite 10.1.3.0 or 10.1.3.1 and with the patch to version 10.1.3.3 on top of that.

By the way, the current 10.1.3.1 version you can download from OTN. You should also be able to find the patch on OTN, but don't ask me how. Using the regular way it pointed me to MetaLink. After some inquiries I got a direct link the Windows version as well as the Linux version. Don't get confused by the fact that you actually are downloading files to install the Oracle Application Server. Installing the SOA Suite is part of that and one of the installation options.

Unless you are running the SOA Suite just for reasons of doing some howto's or create some "look mammy, my first BPEL-process!", you rather not install the ORABPEL, ORAESB and ORAWSM schemas in the Oracle Lite instance that comes with the default installation because of it's limitations.

Schemas Log File

One of the first things you can check is the log file of the creation of the schemas. Installation of the schemas is a separate step and will have been done before installation of the Application Server / SOA Suite. The irca[yyyy-mm-dd]_hh-mm-ss[AM|PM].log file will reside in the directory from which the irca.bat or .sh has been run, e.g. [SOA_Oracle_Home]\install\soa_schemas\irca\ There should be no error reported in this file other than errors related to dropping non-existent database objects (in case of first-time installation).

Installation Log Files

The next things you might want to check are the log files of the installation of the Application Server / SOA Suite itself. In case of installation on a Windows machine there will be two bigger log files in the in the folder C:\Program Files\Oracle\Inventory\logs\ (on Windows, dunno where they are on Linux) with names like installActions[yyyy-mm-dd]_hh-mm-ss[AM|PM].log. One is of the installation of 10.1.3.1 and the other one of the installation of 10.1.3.3. As they are big you better scan them for strings like "ERROR" (in uppercase).

Finding out the Ports

To check the installation you need to know the OPMN and http port. By default the OPMN (request) port of the Application Server will be 6003. The Integration Server will run on the http port, by default 8888. Both ports might have been configured differently during and after installation.

The ports that have been used during installation are configured in the [SOA_Oracle_Home]\bpel\utilities\ant-orabpel.properties file. Make a note of the values of the opmn.requestport and http.port properties.

In case the port numbers have been changed after installation, the actual OPMN port can be found in the [SOA_Oracle_Home]/opmn/conf/opmn.xml file. Search for a line like .

Whenever the http port has been changed after installation, the actual value can be found in the [SOA_Oracle_Home]/Apache/conf/httpd.conf file using the Port and Listen properties.

Checking the Consoles

The next thing you can verify is if every component works properly. The following assumes that the Application Server / SOA Suite has been installed on a machine which an IP address that can be resolved to the host name "localhost", and using the (default) http port 8888.

The components that you should check are:

HTTP Server
Application Server Control
BPEL Console
ESB Console
Web Service Manager Control
Rule Author

Except for the HTTP Server, every other component will require you to log in. By default the user name will be oc4jadmin. The actual user name can be found in the [SOA_Oracle_Home]/j2ee/home/config/system-jazn-data.xml file. Look for a user with display-name OC4J Administrator. The password will be obfuscated so if you don't know, ask your customer.

HTTP Server
Check if the HTTP Server works using the URL http://localhost:8888. This should show the "Welcome to Oracle SOA Suite 10.1.3.3" page, from now on referred to as the Application Server Control. From there you can browse to the Application Server, BPEL, ESB and Web Services Manager Control.

Application Server Control
Clicking the link to the Application Server Control (or typing in http://localhost:8888/em in your browser) should bring you to the login page of Application Server Control. When logged-in the Cluster Topology page should show, displaying the Application Server itself with at least two OC4J instances in it, one normally called home and the other one normally called oc4j_soa.

When the Application Server Control is not up, it might be that you have ran into the issue as described in section "7.9 Accessing Oracle Enterprise Manager after Applying Patch". It states that this happens in rare cases, of which my customer apparently was one.

When the control is up, click on the home instance and then on the Applications tab. From there expand the Middleware Services and then Other Services. That should show a WSIL-app application deployed on it. Click that and go to the Administration tab and on the row that reads View Proprietary Deployment Descriptor tab click on the Go to Task icon. Somewhere at the right it should read deployment-version="10.1.3.3.0".

When patching a 10.1.3.0 installation, one of the (manual) post-application tasks is to redeploy this web application. In the list with post-application tasks it is described that you have to deploy it manually only when patching 10.1.3.0. My customer patched a 10.1.3.1 installation, but for some reason the WSIL-app wasn't. In this way I discovered my customer had not yet done any of the post-application tasks.

I therefore suggest to check the deployment-version of each web application, just to make sure. It's not that much work any way. The other web applications you can check are deployed on the oc4j_soa instance and can be found by going to the Applications tab and expand the Middleware Services. That should list all the SOA Suite components, being:

Rules
ESB
WSM
BPEL

Mind that all these SOA Suite components are just web applications that have been deployed on the oc4j_soa instance. You can expand them all, check if they are up and have the correct version number in the same way as I did for the WSIL-app.

BPEL Control
Clicking the link BPEL Control from the Application Server home page should bring you to the login page of the BPEL Control. Normally you log in using the same credentials as for the Application Server Control, but these might have have been changed after installation.

When logged-in you should see the home page of the BPEL Control. Probably there is not much to see yet, so we move on to the next step. Later on we will deploy a simple BPEL process on the oc4j_soa instance and then you will use this control to test it.

Log out of the BPEL Control. This will bring you back to the login page. Follow the link on the login page to the BPEL Admin console and login. If you have never seen this before, it might be interesting to review and see what is being configured in there. Close the window once you're done.

ESB Control
Clicking the link ESB Control from the Application Server home page should bring you to a login page of the ESB Control home page. As with the BPEL Control, normally you use the same credentials as for the Application Server Control.

When logged-in you should see the home page of the ESB Control. There probably is not much to see here either, so we move on to the next step. Later on we will deploy a simple ESB service on the oc4j_soa instance and then you will use this control to test it.

Web Service Manager Control
Clicking the link Web Services Manager Control from the Application Server home page should bring you to a login page of the Web Service Manager Control. I discovered that it failed to show on my computer and can't replay what I saw at my customer's site so I leave that to your own imagination what to do here.

Rule Author
There is no link on the Application Server home page for the Rule Author (at least not in my case) so you have to check that by typing in http://localhost:8888/ruleauthor in your browser. This should bring you to the login page of the Rule Author.

Once logged in you should see the "Welcome to Oracle Rule Author!" page. As the Rule Author help is deployed using a separate web application, follow the link to "Help" in the corner right at the top and check that the help for Rule Author is coming up.

Creating JDeveloper Connections

After installation of the SOA Suite and having verified it runs you should be able to connect to the Application Server and the Integration Server from JDeveloper in order to be able to deploy BPEL and ESB processes on it.

If the connections have not been made already, start JDeveloper (10.1.3.x) and create an Application Server connection using the OPMN port you found out earlier (default 6003). After that creating an Integration Server connection using the http port you found out earlier (default 8888).

Testing BPEL Deployment

The next thing you can do is checking if you can deploy BPEL and ESB processes. I brought a prepared simple Hello World BPEL process with me, as shown below.

The assign takes some string (a name) as input and replies "Hello [name]. The time is now [date + time]", as shown in the next picture:

You should be able to deploy that on the Integration Server by right-clicking on the project and choosing Deploy and follow the link to the Integration Server. You probably deploy on the BPEL domain called "default," but your customer might have created other domains as well (which you can find out by going to the BPEL Admin console discussed earlier).

Once the Hello World project has been deployed, using the BPEL Control you can navigate to that process which will bring you to the Initiate tab. On that you can enter a value in input and press the Post XML Message button. That should show you a result similar to the following:

^{Hello jan. The time is now 2007-11-30T11:54:37+01:00}

Testing ESB Deployment

Now the Hello World BPEL process has been deployed, you might want to test the deployment of an ESB service. What I did was creating a new ESB project that I called HelloWorldESB and using the ESB diagrammer executed the following steps (unless specified otherwise left everything default):

Create a directory on the application server, for example d:\input and put in there a plain text file called name.txt with in it one string (the name of your customer or whoever is watching every step you take, for example).
Make a copy of the name.txt file. You'll find out why.
From the Component Pallet -> Adapter Services drag a File Adapter to the diagram and call it readName.
Click on the icon next to the WSDL File field and using the Adapter Configuration Wizard create an adapter that reads a file and polls the directory you just created for the file named name.txt with a frequency of 5 seconds. Define a schema for it using the Define Schema for Native Format button and create a new native format of file type Delimited and browse to the name.txt file to sample it. Finally enter a name for the record, e.g. "name" and finish the wizard.
Double click the routing service (named readName_rs), expand the Routing Rules and using the green plus button navigate to the Hello World BPEL process you deployed before until you reach and select the process operation.
Create a new transformation mapper file by pressing the button next to Transformation Map and connect the C1 field from readName.wsdl to the input field of the HelloWorld.wsdl.

After saving an closing the routing service, the ESB diagram will look like this:

You can now deploy it on the Integration Server by right-clicking the project, choose Register with ESB and choose the Integration Server connection. After a successful registration is should show a window saying:

Registration of Services Successful

BPELSystem.default.HelloWorld.readName_RS created
BPELSystem.default.HelloWorld.readName created

If you navigate to the directory containing the name.txt file, you probably will find that it's gone. No panic, this indicates that the ESB service already is working, and will have read and processed the file!

You should able to verify this by going to the ESB Control and click on the printed-paper-page-looking button at the top-right side (what were they smoking when creating that?). At my customer's site it showed the ESB instance alright. However when replaying this at home I found out there is a problem with my ESB installation, as it failed to show instances with a ORA-00904 error. Too bad, I don't have the time to fix it. Anyway, let's be positive and assume that your case is similar to my customer's so that clicking the instance will show you it's flow.

You should also be able to go to the BPEL Control and verify that the ESB service actually called HelloWorld service. Click on the most recent instance of that process and go to the Flow tab. That should show you the flow of the executed BPEL process. When you click on the replyOutput icon, it should pop up a page displaying a message similar to the one you saw earlier when testing the service, this time with the content of the name.txt file.

This is where I concluded at my customer's site that they had installed the SOA Suite alright. Haven't heard otherwise yet, so I assume my conclusion wasn't premature!

Demystifying Business Process vs Use Case Modeling

2007-10-24T20:16:00.000+02:00

The other day I was part of an interesting discussion that started with the statement that there is a problem with OUM (Oracle Unified Method), or to be more precise the Unified Process being use case centric, while these days much development is based on business process modeling. The problem being that because of this people involved with business process modeling might think that OUM does not properly fit their needs.

I often hear people talk about use cases, and too often find out they actually do not know what a use case is. I’m convinced that anyone that does know both business process modeling and use case modeling, would not say such a thing as they would realize that a business process model is just another representation of the same thing.

Let me explain and convince you that OUM supports business process modeling quite well and does so for as long as it is there.

Now, I’m not going to explain what use case modeling all is about, but rather point you to the white paper. I wrote about that subject. However, what I probably do need to explain is that you can have use cases at different levels. My paper is based on OUM and the original work of Alistair Cockburn, who presents the following levels:

Before I continue I need to make the distinction between the conceptual notion "use case", being the interaction of an actor with a system to achieve a specific goal, and a "use case description", being a narrative description of that interaction.

The key point that I’m trying to make here is that a use case should be specified as a scenario. You might realize that at every level you can describe that scenario either by using a use case description, as an activity diagram or both, whatever suits your needs. And you might also be aware that at any level above the user goal use cases, an activity diagram actually is a description of a business process at some level. So there you are ...

What some of you folks that are into the Business Process Modeling Notation (BPMN) might not be aware of, is that UML activity modeling and BPMN just are two different schema techniques for doing the same task, as is clearly explained in a white paper by Stephen A. White. OK, granted, some patterns are more effectively handled by BPMN (like the concept of ad-hoc process to support the Interleaved Parallel Routing pattern), but that concerns minor details only.

So now I proved that a business process model and a use case description can describe the same thing, I hope with that you realize that the only thing you need to do to transform a business process model into a use case description, is by creating a narrative description out of that. But let me bring this academic discussion down to a practical level and discuss how to get from business process models to use cases. Otherwise, why bother, right?

Mind that narrating a business process results in a summary use case description. Many people not being aware that there are different levels of use cases, probably will use the notion "use cases" only to mean the user-goal level use case descriptions, a use-goal use case being defined as a use case for which the primary actor can go away happily after finishing it. If you are not aware of this it is likely you will have a hard time working with use cases. Just to warn you.

When going from business process models to user cases, you will be aiming at a model of user-goal use cases (and a couple of subfunction use cases going with that) as normally that should be the lowest level at which you capture requirements. Before you can do so, you need to make sure that the lowest level business process models contain activities at the level of user-goal use cases only. If that is not the case, fix that first.

Once that has been done you need to take just one more step from there. How you do that is up to you of course, but you could create a use case description for each activity in that diagram that is a candidate for being implemented and start detailing from there. Whenever useful, you can add an UML activity diagram to that. To prevent confusion, you probably better not use BPMN for that.

How to Dump Subversive Revisions

2007-09-27T11:10:00.000+02:00

Most things in life you can more easily get rid of than an unwanted revision in Subversion. As Simon and Garfunkel sang, there are even 50 ways to leave your lover!

Now why is that so hard? It starts with the principle that revisions should build upon each other, and when necessary can be reversed with another revision, so there should be no need to do so. Subversion therefore is designed never to lose information. What this philosophy does not take into account is that sometimes people make mistakes, or even worse are clueless, and for example start to make changes and commit that to a branch instead of the trunk. Sometimes that happens not only once, but twice!

Of course you can argue that people should not make mistakes and the clueless should never be allowed to work with Subversion. Hey, I'm with you, but you need to start somewhere, and I found that many people find it hard to really understand how tagging, branching, switching and all works, and sometimes need considerable time to get there. So let's face it, shit happens and then we feel a need to dump that in the sewer.

If you ever need to do so, there are two ways. The first way is the official way, which works as follows. By the way, in my example I'm using TortoiseSVN as a client:

Make sure you are in the right trunk, tag or branch for which you want to undo some revision.
Using "Show Log" find the revisions you want to undo. You can multi-select revisions.
Right-click the selection and choose "Revert changes from this revision". This will revert all committed changes locally.
Commit that changes (make sure that this time you are committing to the right URL!)
If you switched before you started, don't forget to switch back to the HEAD of the trunk afterwards.

The second way is the "hard way", which consists of making a dump of the repository using the svnadmin dump command of the command line tool. When using that, you can make a dump of a specific range of revisions and leave out the revision(s) you don't want using the -r [lower rev[:upper rev]]switch. After that you create a new repository and load the dump into that using the svnadmin load command.

There are a couple of issue with the hard way. First of all, unless you physically remove the folders of the old repository, the old repository will still be there. And people need to be aware that they need to go to a new repository instead. Of course you can tweak that manually by first deleting the folder of old repository and then create a new one using the same name and only after that load the dump back again.

Secondly, the hard way as described, provides no solution for the situation in which you want to remove a revision from some branch while in the meantime a new revision has been created in the trunk, because you cannot provide a set of ranges while dumping. That can be tweaked by making another dump containing the latest revisions and manually "merge" that with the other dump, but that can get complex.

Finally, the repository might have become a big mama and dumping and loading that can take considerable time.

So my conclusion: you need to have a damn good reason not to do it the official way. I'm therefore most interested in the rumor I've heard, that they plan to introduce the svnadmin obliterate command. Hopefully does that allow for removal of a complete revision as well.

Business Rules as Usual

2007-09-18T10:01:00.000+02:00

There will be some enhancements in the Oracle SOA Suite 11g regarding versioning of rules for Oracle Business Rules. Until then we will have to do with versioning of dictionaries and repositories. This topic will address some best practices I have with versioning and Oracle Business Rules 10.1.3.x.

Versioning of Dictionaries

Versioning of dictionaries aims at rules administrators, which can either be a business analyst or a developer. Versioning of a dictionary is done by saving a particular dictionary using another version number. Sounds trivial, not? Think again. The dictionary probably will work in combination with some application calling the rule engine, providing it with a particular version number to work with. So if you want to change something that should be effective immediately, you should save the dictionary using that particular version number. But what if there is a problem and you want to revert to the previous version? Or would you not rather want to be able to test your changes first before disrupting production? In other words, you want to be able to use at least two different versions: one to be used for testing and one for real.

What I always have is two dictionaries, one version for example with number 2.1.0 for production and another one with number 2.1.x for testing purposes. The test version I can always recognize by the 'x' at the end. Which version the application is supposed to work with, is read from a properties file that I can change on the fly. What I also could do, is create some preference screen through which the rules administrator can select the version to be used during a particular session, the one from the properties file being the default.

As property files can be written, that screen could easily be enhanced to allow changing the default. In this way the rules administrator will have a way to change existing rules or add new ones and test them first, before bringing them into production. The latter can be done by either overwriting the 2.1.0 dictionary with the 2.1.x version, or by saving the 2.1.x dictionary as 2.1.2 and set that as default. The rules administrator will also have the option to activate particular rules for a specific period of time, by putting them in a specific version, and set that as default only during that period.

Versioning of Repositories

Versioning of repositories aims at developers. The prime reason you might want to version repositories, is that during maintenance of the application the fact definitions may change. Fact definitions are dictionary specific, so why not version dictionaries instead? Of course you also use a particular dictionary to make the changes, but obviously you want to make sure you keep a clear track of what version of a repository works with what version of the application. And of course you are using some proper source control system like Subversion to manage you configurations, right? Right! And as that system probably is file based, you want the dictionary to be in a file you can put under version control, obviously.

So what I do is that with every change (or set of changes) I make and that have been properly tested, I make an export of the rules repository and commit that to the repository of the version control system together with the corresponding sources (XSD's or Java classes).

And business rules as usual.

Analize This Business Rule!

2007-09-11T18:14:00.000+02:00

Last week I attended a session in which the new-to-come SOA Suite 11g was being presented by Clemens Utschig-Utschig to Oracle Partners as well as internal employees.

Many great new things to come, like:

One integrated SOA Suite
Regarding the versions up to 10.1.3 the SOA Suite consists of (at least) four different products (BPEL, ESB, BAM, OWSM) that happen to be on one installation CD and 'coincidentally' get all installed during one installation. With 11g this will have changed, from an architectural point of view as well as presentation-wise. For example, ESB will have become part of the infrastructure of the SOA Suite (taking care of routing requests to services), rather than a component on its own. And instead of four different consoles (that not only look like they have been created by independent teams, but probably are as well) there will be one integrated console.
Many enhancements regarding versioning, deployment and unit-testing of services.
Multiple BPEL processes in one project (and the possibility to drill down from one BPEL process to subprocesses)

Just to name a few.

One thing that I have not mentioned but for some reason want to point out in particular, is the fact that JDeveloper 11g will have the Rule Author of Oracle Business Rules integrated in the IDE. That's right folks, with 11g you will be able to connect to and maintain a rules repository using a Swing client rather than the current web client. Clemens did not demo that, so I can't tell you much about how that works, but at least it looked promising. Can't wait to get my hands on that!

But at the same time it raised the question what this means for having a 'stand-alone' Rule Author aiming at business analysts that (given the promise of rule engines and introducing agility to business rules with that) you would expect to be an important user group of rules engines. Not that I have seen that work in practice yet, but that is the promise. Will that disappear, meaning that we have concluded that we want business analysts keep their hands off rules repositories, at least not as long as Oracle Business Rules is concerned?

Fortunately, we seem to be working on a browser-based rule authoring tool as well, aiming at business analysts or any other user that for some reason you do not want to get starting to use JDeveloper. Off course, priorities, hurricanes or Bush eating a pretzel can change that any second, but I have not given up hope for business rules and agility yet!

Oracle Rule Author on Stand-Alone OC4J 10.1.3.x

2007-09-10T14:58:00.000+02:00

For quite a while I could not use Rule Author anymore on my stand-alone OC4J, the reason being that I was not able to connect because of the error '[code=CANT_CONNECT_LOOPBACK] Cannot connect due to potential loopback problems'. I thought, as the loopback problems are only potential, why not give it a try and see how far we get. But my computer thought differently and refused to cooperate.

So I Googled this error, found quite a few links, none of them useful to resolve my issue. As issues like this can easily take a lot of time, and as I had the SOA Suite with Rule Author already running in a virtual machine, I decided to leave it as it was and used that instead. That was, until my virtual machines started to freeze every now and then, especially in the middle of the heat. You can run, but you cannot hide, can you?

For some stupid reason it never occurred to me to search for the loopback problem on OTN, probably because I never associated this error message with Oracle Software. As such that assumption appeared to be correct, but a colleague of mine did nevertheless and found this topic on one of the forums. Boy, how silly I found myself to learn that it was related to the proxy exception list of my web browser! I must have checked that at some stage, but probably at the wrong moment. Arrgggh!

Anyway, as I now can connect again to my stand-alone OC4J again, I thought it would be nice to be able to avoid crashing virtual machines and all that. So I redeployed Rule Author like I was used to, that is using the ruleauthor.ear and rulehelp.ear files from my SOA Suite installation, only to discover that this did not work at all. Right. So, being a clever guy I searched OTN to see how to deploy Rule Author on a stand-alone OC4J, only to find ... nothing! But no panic, Google is still in the air, helping me to a topic that offered me the missing pieces on the IT-eye blog.

Difference with my situation is that I didn't want to install it on the embedded OC4J that comes with JDeveloper, as I want to be able to upgrade JDeveloper without needing to redeploy Rule Author again. So the following instructions are somewhat different to those of the IT-eye version:

Prerequisites:

ruleauthor_s.ear and rulehelp_s.ear
rl.jar, rulesdk.jar, webdavrc.jar, jr_dav.jar

The ruleauthor_s.ear and rulehelp_s.ear I got from the /rules folder of the SOA Companion CD 2, the jar files can be found in the [JDEV_HOME]/integration/lib folder of that same CD.

Steps to deploy (all paths relative to /j2ee/home):

Using the Application Server Control, (re)deploy both the Rule Author and the help file with that.
Create a /rules/lib folder and copy the jar files to that

Configure a rules library in the /config/server.xml file of your OC4J instance, by adding the following code snippet to the shared libraries already configured:

<shared-library name="oracle.rules" version="10.0" 
    compatible="true">
  <code-source path="../rules/lib/"/>
  <import-shared-library name="oracle.http.client"/>
  <import-shared-library name="oracle.xml"/>
</shared-library>

Add the oracle.rules library just created to the default set of shared libraries available, by adding the following line to the /config/system-application.xml file:
```
<imported-shared-libraries>
  ...
  <imported-shared-library name="oracle.rules">
<imported-shared-libraries>
```
(Re)start your OC4J instance

Rule Author should now be up and running!

By the way did I already told you that I was able to fix the problem with my virtual machines by upgrading to the latest Workstation 5 version? No? OK, I was able to fix the problem with my virtual machines by upgrading to the latest Workstation 5 version. I hope.

The End Is Near ...

2007-09-06T18:58:00.000+02:00

... of our customers needing to wait for the Oracle Unified Method (OUM for short)!

Last week version OUM Release 4.4.0 has been made available internally, and I happen to know for a fact that this version will become available to customers. How exactly is yet to be determined officially, but it has already been packaged to be released, so that should not take very long.

And as you have been waiting for more than a year for this, you can bear another few weeks, wouldn't you say?

White Paper Business Rules in ADF Business Components Revamped!

2007-08-23T10:18:00.000+02:00

Finally, the white paper Business Rules in ADF Business Components has been revamped!

As usual it took much more work than anticipated, especially as I made the 'mistake' to ask two subject experts (Steve Muench and Sandra Muller from the JHeadstart Team) to review the first draft. If I had not done that, the paper could have been published a month ago! But no, I could not help myself, I had to be thorough, I had to be me, and of course they provided me with insights that have had a significant impact on its contents. But all for the better, otherwise some of it already would have been obsolete the minute it hit the street.

"So what?", those of you who have not seen it before, might ask yourself. Well let me try to explain without copying too much of what already is explained in the paper itself.

First of all it is our (Oracle Consulting's) experience that analyzing, and implementing business rules takes a significant part of the total effort of creating the average application. Although being a very productive persistence framework as it is, this still holds true for ADF Business Components (which is part of Oracle's ADF), or ADF BC for short. Moreover, despite all our efforts trying to create the ultimate application, most resources still are being put in maintenance rather than in creating the original. So there is a lot to be gained for business rules in this regard, and that is what the white paper intends to address.

Does the paper present 'the right way' for ADF BC? Well, perhaps there is a better way, I don't know. But it is a good way, as it is consistent and makes use of the best that ADF BC has to offer. And because it is consistent (and well documented) maintenance also becomes easier, as when the developers that created the original application used it (and then ran away like lightning to build the next hip thing), they will not have left behind a maintenance nightmare. At least not what the business rules are concerned.

"So, what's new?", those of you who have sleep with the previous version of the paper under their pillow, might ask yourself.

Let me give you this short list and for the rest of it refer you to the paper itself:

Capturing business rules using UML class model
Why? Because plenty of people still want to capture requirements (including business rules) before there are tables and entity objects and therefore cannot make use of an ADF Business Components diagram (see also the article UML Rules! I posted some time ago).
Setting up framework extension classes
Doing so makes introducing generic functionality later on so much easier and therefore is strongly advised in general.
Deprecated custom authorization in ADF BC
This in particular concerns the horizontal authorization rules (restricting the set of rows of an entity you are allowed to insert, update or delete). The reason to do so is that you probably rather use Virtual Private Database for that.
'Other Attribute Rules' dropped
As I discussed in the article How to Pimp ADF BC Exception Handling, there are no compelling arguments anymore for not using built-in validators or method validators, making that the category Other Attributes Rules could be dropped, and we now suggest implementing them using these validators.
New built-in attribute validators
ADF BC provides the Length, and Regular Expression validators for attributes.
'Other Instance Rules' dropped, 'Delete Rules' added
The category Other Instance Rules is dropped for the same reason the Other Attribute Rules category has been dropped. This with the exception of rules that make use of the delete() method, which rules are now in the new category 'Delete Rules'.
Registered Rules
Using Registered Rules you can create generic method validators you can use for multiple entities. An example is a reoccurring validation of an end date that must be on or after a begin date.
UniqueKey Validator
Compared with just checking 'Primary Key' for all primary key attributes, this one and only entity-level built-in validator helps to make validation of the primary key predictable and consistent, and also supports providing a user-friendlier error message.
'Change History' and 'Cascade Delete' added
These two categories are subcategories of Change Event Rules with DML. When using JAAS/JAZN ADF BC offers built-in support for recording date/user created/updated information, which has been documented in the Change History category. As a result of introducing this category, the 'Derivation' category has been renamed to 'Other Derivation'. Furthermore, ADF BC also supports Cascade Delete by defining an association as being a 'composition'.
Sending an email using JavaMail API
The previous version of the paper here and there referred to some 'clex' library which was part of the now long-gone Oracle9iAS MVC Framework for J2EE. If you remember that you really are an Oracle veteran! Anyway, the classical example of a 'Change Event Rule without DML' is sending an email when somebody changes something in the database, which example now is based on the JavaMail API
Message handling
Especially this subject has been revised significantly. Among other things you can specify error message together with the validator, which message will end up in an entity-specific message bundle. By creating custom exceptions you also can use one single message bundle, as I explained in the article How to Pimp ADF BC Exception Handling.

Do you need more in order to get you to download the new white paper? I can hardly imagine.

How to Pimp ADF BC Exception Handling

2007-08-03T18:28:00.000+02:00

When you have been doing things in a particular way for a long time, you sometimes find that in the mean time your way has become the slow way. Not necessarily the wrong way as it works (otherwise you would not have kept doing it all the time, would you?), but you're just not fashionable anymore. Like wearing tight pants in the 60's or wide pants in the 80's. Or using vi instead of a state-of-the-art IDE like JDeveloper or Eclipse for that matter (oh yes, I dare!).

I recently discovered that I became unfashionable by using the setAttributeXXX() and validateEntity() method for implementing business rules in ADF BC instead of using Validators. Not that I didn't know of Validators, I just thought that they would not give me proper control over exception and message handling. Because one of the things I would like to have, is one single message bundle in which I could use a consistent coding of my messages, like APP-00001, APP-00002, etc. Even more important, the other thing I would like to have is that I can translate all messages to Kalaallisut the minute I need to make my application available for the Inuit people of Greenland.

As you might know, up till JDeveloper 10.1.3 ADF BC will create an entity-specific message bundle to store the messages you provide with Validators. So with many entity objects big chance you end up with many message bundles, making that keeping error codes consistent becomes a nightmare. And what about all the separate files you need to translate! You might find yourself no longer to be able to tell the messages for the bundles.

But as Steve Muench was very persistent in trying to convince me using Validators I finally gave in and tried finding a way to tackle my problem, and succeeded! No worries, don't expect rocket science from me. I hate complex or obscure code, as building maintainable Information Systems already is hard enough as it is, and therefore always try to practice Simple Design.

I assume that you have created a layer of framework extension classes as described in section 2.5 of the ADF Developers Guide and that your entity base class is called MyAppEntityImpl. If you have not yet created such a layer, do that first and come back after you finished. Chop chop!

Basically the steps are as follows:

Create extension classes that extend the exceptions that ADF BC will throw when validation fails
Override the constructor of the super class and pass in the message bundle in the call to super()
Override the setAttributeInternal() and validateEntity() methods in the MyAppEntityImpl.java entity base class and make they thrown your exceptions instead of the default ones.

Does that sound simple or what? No? OK, let me show you how I did it.

Attribute-level Validators will throw the AttrValException. So what I did was create a MyAppAttrValException as follows:

package myapp.model.exception;

import oracle.jbo.AttrValException;
import myapp.model.ResourceBundle;

public class AttrValException extends AttrValException
{
public MyAppAttrValException(String errorCode, Object[] params)
{
super(ResourceBundle.class, errorCode, params);
}

/**
* When the message contains a semicolon, return the message
* starting with the position after the semicolon and limit the
* text to the message text from the resource bundle.
*
* @return the stripped message
*/
public String getMessage()
{
String message = super.getMessage();
// strip off product code and error code
int semiColon = message.indexOf(":");
if (semiColon > 0)
{
message = message.substring(semiColon + 2);
}
return message;
}

And this is how I override the setAttributeInternal in the MyAppEntityImpl:

/**
* Overrides the setAttributeInternal of the superclass in order to
* pass in a custom message bundle to MyAppAttrValException subclass
* of the AttrValException. To be able to uniquely identify the
* entries in the message bundle, the error code is extended with the
* fully qualified class name of the Impl.
*/
protected void setAttributeInternal(int index, Object val)
{
try
{
super.setAttributeInternal(index, val);
}
catch (AttrValException e)
{
String errorCode = new StringBuffer(getClass().getName())
.append(".")
.append(e.getErrorCode())
.toString();
throw new MyAppAttrValException(errorCode, e.getErrorParameters());
}
}

In a similar way you can handle entity-instance-level Validators and Method Validator by extending the ValidationException and overriding the validateEntity() method to throw your MyAppValidationException.

As said, the message you provided when creating build-in Validators and Method Validators end up in an entity-specific message bundle. I've been told that this is going to change in JDeveloper 11g, but currently there is no stopping ADF BC from doing that.

So, at a convenient point in time (for example when most of your Validators have been implemented and tested), what you need to do is copying the error messages from the entity-specific message bundles to your custom message bundle.

To prevent duplicates in the keys of the custom message bundle, you should extend the key of each message with the fully qualified class name of the Impl.java file of the entity object where it's coming from. Otherwise duplicates might occur whenever you have two different entity objects, both with an attribute with the same name and a build-in Validator specified for them. The overriden setAttributeInternal() method assumes you did.

The entries in the message bundle would look then similar to this:

{ "myapp.model.adfbc.businessobject.EmployeeImpl.Salary_Rule_0",
"APP-00001 Employee's Salary may not be below zero" },
...

Well that wasn't rocket science, was it?

We Are Proud of Borg

2007-07-13T13:35:00.000+02:00

I case you wonder why I don't write that much lately, my wife just got out of the hospital with a so-called Ilizarov frame, with the purpose to try and fix an arthritis in her ankle (caused by an ill-treated complicated fracture many, many years ago).

Believe me, anyone who've seen her was impressed! She looks like an unfinished Robocop, a kind of Transformer that just got out of it's egg. When she wants something from you, you just know that resistance is futile. Every time she walks I have a hard time suppressing the urge to make 'uhnk-chuck uhnk-chuck' sounds. Fortunately she doesn't read my blog (I hope) and if she does, I'm just too fast for her as long as I don't let myself get cornered.

What also is impressive is the amount of time you need to spend in the beginning to take care of some one in her condition. People with children might remember the first couple of weeks, how chaotic things can be and you trying to get a grip on the situation and finding a modus operandi to get you through the day without going wacko. Picture that, plus a full-time job and the Dutch health care system of today and you cannot help feeling sorry for me. Fortunately there are plenty of people around me willing to help out, so forgive me for making things look worse than they are. I needed an excuse for not writing that much and thought a little exaggeration could do some good here.

But what is most impressive is the courage of my wife, to go through the operation knowing what she nows about the frame that she will have to suffer that for three months. That really makes me proud!

How to Prevent Your Rule Gets Fired in ADF BC?

2007-07-04T15:32:00.001+02:00

When using the Oracle ADF (Application Development Framework), implementing data-related business rules in the persistance layer is a good practice. In ADF this business layer is called ADF Business Components (aka BC4J, or Business Components for Java). This article will go briefly into this subject, just enough to let you get an appetite for the upcoming revised white paper Business Rules in ADF BC. So don't eat too much of this apetizer, to leave room for the main course to come!

Implemening business rules in ADF Business Components means implementing them in so-called entity objects. For an application that uses a relational database, to a certain extend you can compare an entity object with an EJB entity bean, as like an entity bean the purpose of the entity object is to manage storage of data in a specific table. Unlike EJB entity beans, ADF entity objects provide hooks to implement business rules that go way beyond what you can do with EJB entity beans.

I won't go into detail about these hooks. When you're interested, enough documentation about the subject can be found on the internet (to begin with the Steve Muench's web log) or read the white paper! I will let you know when it is available and where to find it.

Where I do want to go into detail is one aspect, being how to prevent that rules get fired unneccessarily, for example for reasons of performance. I assume some basic knowledge of ADF Business Components, so if you don't have that, this is where you might want to stop reading.

There are the following typical options, provided as methods on any EntityImpl:

The isAttributeChanged() method can be used to check if the value of an attribute has actually changed, before firing a rule that only makes sense when this is the case.
Furthermore there is the getEntityState() method that can be used to check the status of an entity object in the current transaction, which can either be new, changed since it has been queried, or deleted. You can use this method to make that a rule that only gets fired, for example when the entity object is new.
There is also the getPostState() that does a similiar thing as getEntityState() but which takes into consideration whether the change has been posted to the database.

Sometimes knowing whether or not something has changed does not suffice, for example because you need to be compare the old value of an attribute with the new one. That typically happens in case of status attributes with restrictions on the state changes. Normally you would be able to do so using the getPostedAttribute() method, that will return the original value of an attribute as read from or posted to the database. However, that won't work when you are using the beforeCommit() method to fire your rule, as at that time the change already has been posted, so the value getPostedAttribute() returns will not differ from what the getter will return.

"So what", you might think, "when do I ever want to use the beforeCommit()?". Well, you have to as soon as you are dealing with a rule that concerns two or more entity objects that could trigger the rule, as in that case the beforeCommit() is the only hook of which you can be sure that all changes made are reflected by the entity objects involved.

Suppose you have a Project with ProjectAssignments and you want to make sure that the begin and end date of the ProjectAssignments fall within the begin and end date of the Project. A classical example, I dare say. Now the events that could fire this rule are the creation of a ProjectAssignment, the update of the Project its start or end date or the update of the ProjectAssignment its start or end date.

Regarding the creation of the ProjectAssignment, that you can verify by using the getEntityState() which would return STATUS_NEW in that case. Regarding the change of any of the dates, you can check for STATUS_MODIFIED, but that also returns true when any of the other attributes have been changed.

Now suppose that, as there can be multiple ProjectAssignments for one Project, you only want to validate this rule when one of those dates actually did change. The only way to know is by comparing the old values with the new one. As explained before, as the hook you are using will be the beforeCommit() method, the getPostedAttribute() also will return the new value as that time the changes already have been posted to the database.

Bugger, what now? Well, I wouldn't have raised the question unless I would have some solution to it, would I? The solution involves a bit yet pretty straightforward coding. I will only show how to solve this for the Project, for the ProjectAssignment the problem can be solved likewise. The whole idea behind the work-around is that you will use instance variables to store the old values so that they are still available in the beforeCommit().

Open the ProjectImpl.java and add the following private instance variables:

private Date startDateOldValue;
private Date endDateOldValue;

In general you can use a convention to call this specific type of custom instance variables [attribute name]OldValue, to reflect their purpose clearly. Now you need to make that these variables are initialized at the proper moment. This will not be the validateEntity(), as that can fire more than once with unpredictable results. No, it should be the in the doDML() of the ProjectImpl, as follows:

protected void doDML(int operation, TransactionEvent e)
{
  // store old values to be able to compare them with
  // new ones in beforeCommit()
  startDateOldValue = (Date)getPostedAttribute(STARTDATE);
  endDateOldValue = (Date)getPostedAttribute(ENDDATE);

  super.doDML(operation, e);
}

Now in the beforeCommit() you can compare the startDateOldValue with what will be returned by getStartDate(), etc. Although it might not look like it at first sight, this might be the most trickiest part as you need to deal with null values as well. In JHeadstart the following convenience method has been created to tackle this, in the oracle.jheadstart.model.adfbc.AdfbcUtils class:

public static boolean valuesAreDifferent(Object firstValue
              , Object secondValue)
{
  boolean returnValue = false;
  if ((firstValue == null) || (secondValue == null))
  {
    if (( (firstValue == null) && !(secondValue == null))
        ||
        (!(firstValue == null) && (secondValue == null)))
    {
      returnValue = true;
    }
  }
  else
  {
    if (!(firstValue.equals(secondValue)))
    {
      returnValue = true;
    }
  }
  return returnValue;
}

This method is being used in the beforeCommit() of the ProjectImpl, as follows:

public void beforeCommit(TransactionEvent p0)
{
  if ( getEntityState() == STATUS_MODIFIED &&
       ( AdfbcUtils.valuesAreDifferent(startDateOldValue,
              getStartDate()) ||
         AdfbcUtils.valuesAreDifferent(endDateOldValue,
              getEndDate())
       )
    )
  {
    brProjStartEndDate();
  }
  super.beforeCommit(p0);
}

Finally, the actual rule has been implemented as the brProjStartEndDate() method on the ProjectImpl as follows:

public void brProjStartEndDate()
{
  RowIterator projAssignSet = getProjectAssignments();
  ProjectAssignmentImpl projAssign;
  while (projAssignSet.hasNext())
  {
    projAssign =
      (ProjectAssignmentImpl)projAssignSet.next();
    if (((getEndDate() == null) ||
         (projAssign.getStartDate().
              compareTo(getEndDate()) <= 0)
        ) &&
         (projAssign.getStartDate().
              compareTo(getStartDate()) >= 0)
       )
    {
      // rule is true
    }
    else
    {
      throw new JboException("Project start date must " +
        "be before start date of any Project Assignment");
    }
  }
}

Well that wasn't to difficult, was it?

Did You Found Your Balance With Java?

2007-06-21T12:18:00.000+02:00

When I was young I practised judo for a couple of years. Got myself a brown belt and nothing after that, as I wasn't the competition kind of guy. I could have managed getting the black belt on technique alone (by doing so-called kata's), but that would require a lot of practise and time, which I did not had. One of the reasons being that at the same time I also practiced jujutsu (got myself a green belt for that).

When I went to the university I visited a different dojo, again only for a couple of times because of lack of time, but long enough to understand what in the previous dojo the sensei failed to teach me: proper balance. Imagine that during a randori (that is sparring) you jump around too much, giving the opponent many opportunities to show you every corner of the dojo. Well, that was me.

I have not been doing martial arts for many years now, so I was way beyond any frustration about this. That was until on the recent J-Spring of the NL-JUG (spring conference of Dutch Java User Group) when I learned about the JavaBlackBelt community. As is described on the home page of their web site: "JavaBlackBelt is a community for Java and related technologies certifications. Everybody is welcome to take existing exams and build new ones." I couldn't help it, I had to check it out. Am I still jumping around, or have I found my balance with Java?

Well, I expect it will be a long way before I know, as again I don't have much time. But I managed to get at least a yellow belt! So for now I find some comfort knowing that at least I'm beyond the level of Java-newbie.

Cooking with OUM

2007-06-15T11:25:00.001+02:00

The other day I had a really challenging discussion about how to apply OUM (Oracle Unified Method) / UP (Unified Process) on a running project. As with all projects, it had to be decided what needed to be done to deliver a system, no surprise there. The challenging part first of all was in that we tried to fit an incremental approach into a waterfall-like offering. Secondly the approach had to be supported by people that had no previous exposure to OUM nor UP. So how can you bring that to a success without boiling peoples brains?

Let me start with the good news, which is that we reached a point that people have an idea about what they are going to do, and made that plan themselves rather than me telling them what to do. That is exactly how you want things to be in the end, so what more do you want? I should be happy!

The bad news is that something 's nagging the back of my head, being that I made a couple of mistakes along the road, resulting in me not having the feeling the message was understood properly. But as a famous Dutch philosopher once said: "every disadvantage 'as it's advantage", in this case being that that me feeling miserable is not your problem, and we all might learn from it.

In retrospect I think the one and only real mistake was skipping the part in where you explain the principles behind OUM and UP, and being so naive to think I could do that along the road. OK, I jumped on a running train and the approach needed to be there yesterday, so maybe I'm excused, but boy, what a mistake that was as it made discussions so difficult. But let's skip the nag whine part and let me explain what I would do differently next time.

First of all I would explain very well the difference between a task and a deliverable. When that is clear, I would be ready to explain how iterations go in OUM. Pretty important when you need to explain how exactly you do high-risk activities first (a principle that I luckily did not forget to explain up-front).

Unfortunately, I presented all tasks using the name of what OUM calls 'work products', giving the impression that every task results in a deliverable that actually is going to be presented as such to the customer. As in my proposal there were quite a few tasks to do, it looked like I proposed creating a pile of deliverables, scaring the hell out of people.

In OUM the term deliverable is reserved for a "product" that we actually present to the customer and that you likely will find in the project proposal or project plan. In order to get to that deliverable, you sometimes do multiple tasks, like creating an initial version of some document, reviewing that with the customer, in a next phase adding details using some diagrams, reviewing it again, etc., before you finally come up with one concrete deliverable.

Every (intermediate) task results in a "work product" that might or might not be presented to the customer as a deliverable. The important thing to notice is that, although the tasks might have different names and associated codes, you actually work on multiple iterations of the same deliverable.

Below an example of how the Use Case Model is created in three iterations (I left out any review tasks between each iteration). The rounded boxes are the tasks, the iterations of the work products are the square ones. By drawing a box around the three Use Case Model iterations I express these will be combined into one single deliverable.

How many work products you plan to combine into one deliverable, is up to you. Three words of advice, though.

First try to prevent multiple people working on the same document at the same time, because that unnecessarily introduces the risk of people needing to wait on each other. When you anticipate this might happen, that’s a strong contra-indication for combining work products.

For this reason, in the example the Use Case Realization has not been combined with the Use Case Model, as in this case the assumption is that the Use Case Realization will be worked out by a Designer, while the Use Case Model has been worked out by a (Business) Analyst, both having different responsibilities.

Secondly, do not iterate the same work product over a long period of time, because you might get the feeling it never finishes. You definitely should not let deliverables iterate cross phases that need to be signed-off. Not many customers want to put their signature on a document in a state of “draft”. I always try to prevent this kind of signing-off way of working as it can be killing for agility, but when there is no other option, be aware that after signing-off normally a deliverable can only be changed after a formal change request procedure (and that takes valuable time).

In the example, this is one of the reasons the MoSCoW-list has not been combined with the Use Case Model, as the assumption is that the MoSCoW-list has been used to define the scope and priorities of high-level requirements, providing the baseline for the activities during the Elaboration phase.

Finally, wherever you combine work products, keep track of the link to the tasks that create them, and make explicit in what iteration the deliverable is, for the following reasons:

When keeping track of the original tasks you can make use of the work breakdown structure the method offers, supporting estimating and progress tracking (for some silly reason some managers like to know how far you are from time to time).
Knowing what task you are performing facilitates using templates and guidance the method offers for that task.
It allows for people outside the project to understand what you're trying to achieve with a specific work product, enabling them to have the right perspective when reviewing it. Otherwise there is a big risk of expecting the wrong thing and not talking the same language. I have seen this going nasty a couple of times when the customer brought in their own experts, believe me.
Whenever you want to know how a specific workproduct should look like, you can ask questions like: "does anyone have a good example of a RD.011 High-level Business Process Model for me?" and surprise everyone around you. You might even get what you asked for rather than some document that you have to study for an hour or so to finally conclude it's not what you need. Have you been there, done that?

These are benefits you directly start to profit from right here right now. But what about some next time? Would it not be nice when you could reuse your deliverables as an example for one of your next projects? Or even better, would you not be proud when one of your deliverables ends up in the company-library of outstanding examples, giving you the feeling you have achieved all there is to achieve, quit your job, sell your house and go and live like a king in France, as we Dutchmen say?

Hmmm... Suddenly I start to understand why sometimes it is so hard to get good sample deliverables in some organizations. It may be because of some company policy forbidding such a library.

UML White Papers Released

2007-06-05T22:00:00.000+02:00

Remember the UML white papers I talked about updating them? Well they got published today!

So when you are interested in getting started with:

UML use case modeling
UML class modeling
UML activity modeling

go to the JDeveloper Technical Papers section on OTN and download them from there. I expect the average paper to take not more that one to two hours to read and digest. Happy reading!

Recipe for Stress

2007-06-01T19:40:00.000+02:00

When capturing specifications, it is always a challenge to weight effort against risk. The more you specify, the less risk there is of missing requirements. At least that is what you hope. Unfortunately, experience shows that no matter how hard you try, you never get them all. So where do you stop? When is it time to go home and grab a beer?

The other day I reviewed some use cases. Some of the use cases were worked out pretty detailed. There even was a log-in use case with a (conceptual) screen layout. But as this was only the start of the project no supplementary requirements had been defined yet. As an answer to of the question of how detailed one should get while capturing requirements, I provided the following example based on the log-in use case (which by the way is subfunction rather than a user goal).

Your customer might agree that you can leave the functional requirements for a log-in use case to "the user can log in by providing either a user name and a password or a customer number and a password", and not ask for a screen layout as for most people that is pretty straightforward.

However, at the same time supplementary requirements to this use case might be something like:

At a minimum passwords must be changed every 2 months
Passwords must be at least 8 characters in length, and a mixture of alphabetic and non-alphabetic characters
Passwords may not be reused within 5 password changes.

Notice how the functional requirement is very brief, only one short sentence. However, the supplementary requirements are and should be pretty specific, as often it is far from trivial how to implement them.

Thinking about something "simple" as securing passwords might also trigger the users to define other security requirements as well, like securing web services. When doing a mapping from sequirity requirements to an initial technical architecture, you might decide to use Oracle Web Server Manager. That might imply the need for some extra setup and expertise to do so. And it all started with a simple screen with only two items on it!

So what do we learn from this?

Failing to recognize supplementary requirements in the beginning, is a good recipe for a lot of stress later on. That is nice to know for the masochistic project managers out there. For those who are not like that you better make sure you have discussed the supplementary requirements with your customer to a sufficient level to be able to baseline at least is a little bit reliable.

Not discussing supplementary requirements and think it just will blow over is a mistake. Most customers have only a vague idea if any, and will expect us to tell them what their supplementary requirements should be. Failing to do so might jeapordize your relationship with your customer.

UML rules!

2007-05-25T16:07:00.001+02:00

Currently I'm updating the white paper Business Rules in ADF BC. Doing so it occurred to me that there are still many questions to answer what to do when you want to record business rules in UML. With this posting I will share some of my idea's. JDeveloper 10.1.3 will be my tool and ADF BC4J the targeted persistence layer, but I expect the idea's to more generic and also applicable to EJB 3.0 and POJO's for that matter.

In "the old days" when we were young and still using the function hieararchy diagrammer, business rules were either an intrinsic part of the entity relationship diagram or recorded in function descriptions. As this sometimes resulted in the same business rule being recorded more than once (in different functions) and too often in an inconsistent way, we found it to be a neat idea to record them explicitly, that is as functions of their own, and link them to the functions they applied to. This has been a very succesful approach for quite a few years.

At the same time UML arrived as the modeling language for object-oriented analysis and design, and more recently BPMN for business process modeling. As I have never come upon any information system for which there were no business rules whatsoever, you can imagine that it was quite a shock for me to discover that there was no similar thing in place for UML at that time. Of course, a UML class diagram as such allows for expressing more business rules than an entity relationship diagram, but that covers only part of it, doesn't it? As far as the rest is concerned, we were kind of back to square one, meaning that we had to record business rules as part of use cases, for example. The BPMN specification even explicitly excludes business rules from their scope, so what do you do when using BPM?

Some improvements have arrived since the beginning of UML. At some point (not sure since when) constraints have been added (as a stereotype). Also OCL has emerged as a formal language to capture business rules. You can use OCL to record the business rule in a constraint. I don't know about you, but I still have to meet the first person that speaks OCL, let alone the kind of persons we call customers. So in most cases I expect constraints to be recorded using some natural language.

Now let's asume you are using UML to capture functional requirements. Business rules can be found during any stage, for example while doing use case modeling. When starting to create use cases it perfectly makes sense to record rules as part of that. Use cases have some properties that can be used for recording business rules, being pre-conditions and post-conditions. Pre-conditions must be met before the use case can be executed. A typical example would be "the user must be logged on", but could also be something like "the user must be a manager to view detailed employee information".

Post-conditions can be devided in minimal guarantees and success guarantees. Minimal guarantees indicate what must hold true when none of scenarios of the use case have finished successfully. A success guarantee describes a succesfull execution of the use case. Examples of success guarantees are:

The employee information has been recorded successfully
A change of the employee salary must have been logged
When the customer has boarded 15 times or more on a flight, its frequent flyer status must be set to silver elite.

Then there are so-called invariants, being conditions that always should hold true (before, during and after execution of any use case). Examples would be

End date must be on or after begin date
There cannot be a non-vegetarian dish in a vegetarian dinner.

Your use case template might not have a property called invariants. But hey, it's your use case, so why not add it?

By now you might wonder how to deal with the problem I pointed out at the beginning, being that you captured the same business rule more than once for different use cases. You might also find yourself recording business rules that relate to each other, or even contradict. Don't worry to much about that in the beginning (using the Unified Process that would be while doing Requirements Modeling). Just record the business rules when and where you find them. You only risk a confused customer when you start record business rules as artifacts on their own.

During some next iteration, you detail the use cases. By that time you might have a class model or will start to create one (using the Unified Process that would be during Requirements Analysis). At this point it makes sense to pull business rules out of the use cases and into constraints. Doing so you are able to remove duplications, and inconsistencies.

If your tool supports publishing use cases together with the artifacts linked to that (like classes and constraints ), you might consider to remove the business rules from the use case to prevent duplication and inconsistencies.

Using JDeveloper you normally add constraints to UML class diagrams and link them to the classes they relate to. But you can also include them in use case diagrams and link them to use cases. Finally, you can link related business rules together, which is convenient when you want to decompose business rules.

Having done all this you are not only able to track what rules apply to what classes but also to what use cases. Handy for example for testing purposes.

Finally, you can include the same contraints in (for example) ADF Business Component diagrams, link them to entity objects, classify them or whatever fancy stuff you like to do. Check out the new version of the Business Rules in ADF BC for ideas about classifying business rules when using BC4J. I will let you know when it get's published.

Got to stop here folks. A long and sunny weekend just knoked at my door, and I intend to open ...

Yet Another Useless Teaser

2007-05-16T20:00:00.000+02:00

Yes! Today the Oracle Unified Method (OUM) Release 4.3.0 has been announced!

Ermmm. That was an internal announcement, as we do not yet have it available for customers. Me and my big mouth! Well, now you know I should say some more. There is no way back, is there?

To begin with I can tell you that it will not take long before OUM will be made available to customers, as the conditions for that are being discussed almost as we speak. I cannot tell you the exact details yet, as otherwise I would have to kill you. Also, I do not know the details myself so I have to wait as well. So now you still know nothing, do you?

Well, I did write about OUM before, so if you haven't read that article yet, what's keeping you? Furthermore, if it is any consolation to know (which by the way also was the name of a Dutch death metal band that split in 1999), I will tell you what and how as soon as we can deliver. So stay tuned!

JDeveloper Use Case Modeler Revisited

2007-05-14T14:12:00.000+02:00

The other day I wrote about the UML Class Modeler of JDeveloper 10.1.3 and concluded that not much has been changed since JDeveloper 10.1.2. I cannot say the same for the UML Use Case Modeler of 10.1.3. Well, number-wise there still are not that many enhancements, but importance-wise the more.

First of all two new use case objects have been added, being the System Boundary and Milestone. The System Boundary can be used to organize use cases by subject, and typically will be used to organize use cases by (sub)system. The Milestone boundary can be used to organize use cases, for example by iteration in case you develop the system incrementally. You can put all use cases that are within the scope of a specific increment in the Milestone of that increment.

In the example I have used System Boundaries to organize use cases by subsystem. One subsystem being the front-office and the other one the back-office of some service request system.

System Boundaries and Milestones can be used together and a use case can be in more than one of each. For example, the use case Enter Service Request can be in both the "Service Request Front-Office" System Boundary as well as in the "First Increment" Milestone.

The most appealing change concerns user-defined use case templates. The JDeveloper UI has been changed and now allows you to create new pallet pages with user-defined components. There are four different type of pallet pages you can create, one for cascading style sheets, one for Java, one for code snippets and one for use case objects. The latter is the one I'm interested in right now.

You can create your own version for every type of use case objects (actor, use case, system boundary, or milestone), the most interesting one being that for use cases. Normally on a project you will have two different types of use cases, so-called "casual" ones and "fully dressed" ones. The difference is in the number of properties you specify, where a fully dressed one has more properties. The casual one is typically used for simple use cases for which "just a few words" will suffice to describe them. For more complex use cases you will use the fully dressed version.

To my taste both use case templates that are shipped with JDeveloper lack a few properties, being:

Use Case number (it is a good practice to number use cases for reference)
Priority (in general it is good practice to prioritize everything you do in a project)
Status (is the use case just started, draft, or more or less stable)
Brief Description (describe what the use case is about in just a few sentences).

To adjust the templates to include the above properties I normally had to adjust the original ones in the appropriate JDeveloper folder (after making a backup copy, of course). But now I can make a copy to a project specific folder (which I can put under version control) rename them to whatever I want, and add them to JDeveloper as components in a newly created pallet page.

There is one caveat though. When you start creating use cases, it might not always be clear if a use case is simple or complex. Normally you will create use cases in iterations, starting with just a Brief Description and add the scenario later. It then may turn out that the use case is a lot more complex than you initially thought. Unfortunately you cannot transform a casual use case to a fully dressed one (or visa versa).

But rather than creating every use case as a fully dressed one to be on the safe side, I choose to live dangerously and add extra properties when needed. Under the hood every use case starts as a copy of one of the templates and consists of XML/HTML tags. Using the Source view you can simply copy the extra fully-dressed properties from a fully dressed use case to a casual one and fill them out.

Last thing to say is that the use case editor has become a lot more userfriendly. Also nice to know, wouldn't you say so?

JDeveloper Class Modeler Revisited

2007-05-10T11:42:00.000+02:00

About two years ago I published a couple of white papers about UML, unit testing, and source control, and how to do that using JDeveloper 9.x / 10.1.2. These white papers still can be found on the white paper archive on OTN and are:

Getting Started with Use Case Modeling
Getting Started with Class Modeling
Getting Started with Activity Modeling
Getting Started with Unit Testing
Getting Started with CVS

As since then the world has turned around a couple of times I took the time and started revisiting them and bring them up-to-date with JDeveloper 10.1.3.

The first paper I have revisited is the one for UML class modeling. To be honest, not much has changed. The only thing I find worth mentioning is that for a class attribute you now can specify if and when it can be updated, using a new property called Changeability. Do I hear a big "Wauw!"? No? Hmm.... Well, you know, of all the UML modelers it is the most mature one, as it also is the base of the Java class, ADF business components, and database modelers, which are (implementation specific) specializations of the UML class modeler.

Only thing that really bugs me is that you still cannot specify unique identifiers for a UML class. I must admit, it's not in UML 2.0 as well, but we are a company that happens to sell relational databases (among other things) and for many of us specifying a unique identifier is like riding a bike is for Lance Armstrong! One of the consequences is that it still is a bad idea to transform a UML class model to ADF business components, because (as I explain in the white paper) ADF entity objects require primary keys and the transformer therefore alphabetically picks the first attribute, and that is not necessarily the right one. The result is a lot of (error prone) work fixing the ADF model, making that I advice against transforming a UML class model to ADF business components. And that's too bad as most people I know use ADF.

"But hey, you are talking JDeveloper 10.1.3, and is that not already an archived version?", I hear you say. Well, almost folks. Recently the JDeveloper 11g - Technology Preview has been released, and YES! the new features overview tells me there is a major change regarding the UML class modeler.

From a feature point of view there still is not much to tell, although one of my enhancement requests from two years ago has been implemented, being that attributes can now also visually be order logically (instead of alphabetically) allowing you, for example, to put the most important ones at the top and attributes that logically belong together above each other. Is that a big deal? Believe me, you want to review a big UML class model it is!

But yet I do have the impression the UML modelers of JDeveloper are going to be taken more seriously than before, as the new features page also state that the UML class modeler has been rewritten on a new graphical engine that provides better better performance and scalability. And not only that (and I quote): "Future releases will see other modelers being re-hosted on this new framework". Initially we will most benefit from this as it will greatly improve the usability of the modelers, not only from an editing point of view but also regarding publishing the models. And whoever has been on a project that involved more than just a few tables knows how important that can be.

Can't wait to get my hands dirty on that! But first I need to revisit the other modelers as well, so watch out for new postings about that. I already can tell you that the UML use case modeler of 10.1.3 definitely has more seriously been changed!

My Agile Project (based on a true story)

2007-05-07T15:59:00.000+02:00

What makes a project 'agile'? Are you only doing an agile project when you apply all principles of eXtreme Programming for example? From the way I phrased these questions you probably already concluded that I think differently.

What agile projects really make agile is best formulated in the Agile Manifesto. Among others the following principles are included:

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.

Not specifically included (probably assumed to be intrinsic) is the following principle:

Address the biggest risks first.

I think these principles best explain why my latest project has been such a success, meaning that it as delivered in time, within budget, with a happy customer, and last but not least, happy end-users!

But before I explain, let me first tell you a little bit about the project. It concerns maintenance of an existing system that is over more than five years old. The system has been build using Oracle Designer / Developer, with a lot of PL/SQL code in the database of which a considerable part enforces rules and regulations. Rules and regulations being changed significantly was the reason to start the project and make the system adapted accordingly.

The major challenges we focussed were these:

A first increment should go into production before January (it was end of October when I got first involved so we had only two months).
The PL/SQL code was set up using a lot of (what we in retrospective may call) bad practices, like one huge package that 'does it all' with tightly coupled procedures of which many had no clear responsibility.

Although we talk about rules and regulations, many things were up for interpretation. When you read one of my previous posts you know I try to avoid a Big Design Up-Front where feasible. Normally you have to justify this to people that have not yet bought into agile development, but in this case the time constraint made it even impossible to create such a thing.

So what you gonna do? Well, first of all we (meaning a team of 3 developers, a client-project manager and ambassador user) decided to meet on a daily basis to monitor the status, exchange information and address issues as soon as they emerge. In this setting we made up a high-level MoSCoW-list, determined the dependencies between the requirements, and decided upon their prorities (primarily based on dependencies and biggest risks first). Using this as input, we then thought of an approach how we best could adapt the existing system in an incremental way. We created a low-level MoSCoW-list with a scope of two weeks, delivered what we produced and based on our experience created the next low-level MoSCoW-list.

Sounds easy, wouldn't you agree? Yeh, right...

We are dealing with people here and in general people have a few pretty persistent habitst that must be dealt with. Like many people find it counter-intuitive to design and implement functionality just to achieve a foreseable goal, knowing that in some near future they probably have to change it as a result of evolving insight. Main reason is that we have a tendency to try to do it right the first time to prevent needing to change code later on which just seems like overhead. Unfortunately experience shows almost none of us is capable overseeing everything that needs to be taken into account upfront. Discrepancies between vision and reality emerge the minute we start to build the real thing and especially become apparent when end-users are confronted with it.

Another tendency we have, is starting with a couple of simple things just to get a sense of achievement, while postponing the more complex issues, hoping they kind of solve themselves as time goes by. But those issue are the ones with the biggest risks associated with them and when they do not solve themselves, it often is too late and you find yourself being in big trouble. I have never done research to find out, but I do believe that addressing high-risk issues too late is the major cause of budget overrun, deadlines not being met or deliverance of considerable less functionality than planned for.

So what agile methods therefore promote is developing small increments that are delivered frequently. This will prevent us from going the wrong direction for too long as we will notice it soon after that happens. To prevent that changing things later indeed results in a lot of overhead, we maintain a simple design by creating the simplest thing that could possibly work. Whenever the design starts to get complex, we refactor to make it simple again. To allow for refactoring without breaking the functionality, we apply unit testing to prove the system still works afterwards. To reduce risk we ensure that the code one person creates works with to code of others, by continuously integrating everything we produce. And wherever feasible, we start with those aspects that are the least clear to us how to proceed with, to find out as soon as possible how they can be resolved and what the impact on other aspects is.

All this is not just a nice theory. We did just that and proved it works!

We consulted each other at least once a day, keeping eachother on track with maintaining a simple design and refactoring when necessary. On an average we delivered every two weeks and from increment to increment added new functionality while at the same we improved the existing poor architecture. The result is a system that not only does what it is supposed to do, but which also is based on a solid architecture with excellence through simplicity!

Only time will tell, but I have a strong feeling that we considerably reduced the costs for maintenance and perhaps made that the system will survive for yet another five years.

Can I take you(r method) home with me?

2007-05-01T20:26:00.000+02:00

Already a couple of years I'm a member of a group within the Oracle Methods Group that is responsible for the Oracle Unified Method, or OUM for short. OUM is based on the Unified Process (UP), as is the Rational Unified Process (RUP) and as such has many similarities with RUP.

"So why yet another method?", you might ask. Well, there are plenty of reasons, but to mention a few important ones:

Years ago Oracle created the Custom Development Method. That method has been a great success for ourselves as well our customers, and many people are used to it. While creating OUM we are able to make it relatively easy for people to "migrate" from one method to the other by using the same kind of structure.

RUP is optimized for being used with Rational/IBM tooling. And as Oracle has it's own extensive set of tooling it makes sense to have a method taking that into consideration. Of course, to a certain extend RUP is configurable and extendable, but that has its limitations and would not facilitate the first point.

Maybe you do not know Oracle for running IT projects, but we do have a lot of expirience, especially regarding projects that involve a database or a Service Oriented Architecture and found that there is a lot of room for improvement in how the (Rational) Unified Process addresses those issues, if at all.

Last but not least, the Unified Process does not address cross-project/enterprise-level issues very well. Nowadays there is also the Enterprise Unified Process. Although a valuable extension to UP for us it still has a few shortcomings. Among them that it still does not address all aspects we found to be important to cover, like IT Governance.

On a very short notice the Oracle Methods Group will make OUM 4.3 available. That version will have a first cut of what we consider to be necessary to add to UP to make it an enterprise-level development method. For that we added an extra "dimension" to UP which we call Envision, as well as a couple of enterprise-level processes.

OUM 4.3 will also be the first release that (to a limited extend) is available for our customers as well. As the specifics are yet to be determined, I cannot tell you more yet, but I will keep you posted.

I know, going to the pub telling people that I'm working on OUM "which now has Envision as well!" won't get me the hottest chicks wanting to take me home. But hey, to most girls telling you are in the IT business is a turn-off anyway, so now you know I dare to tell you I'm quite exited about this release going to hit the streets!

And as soon as it does, I will let you know ...

When the going get's tough ...

2007-04-25T10:36:00.000+02:00

Man, have I been busy lately!

Actually that is a very lame excuse for not having posted for a long, long time. The excuse even gets lamer, when considering the reasons for being busy, namely being lead developer / project manager for two projects at the same time. So what happened to me using agile development princples and being in control and all that crap? Do I not only talk the talk but also walk the walk?

Let me put it this way. If I hadn't had done so I probably would have had a complete burn-out, as just like in real life both projects became the busiest at exactly the same time. And just like the average male, I'm very bad at doing two things at the same time. So I had to go back to my manager and say: "Hey I only have one brain (and what a fine brain that is, ahum) but you are asking me to do a two-brain's job. So if nothing is going to change, I need to make one of those jobs a no-brainer so that I can concentrate on the other one." And as nothing changed, that was what I did.

This made the other project manageable again, but for someone like me it did not make my job much fun anymore as I like to do things properly. So that was when I decided to go to Costa Rica for holidays. When I got back, I felt soo much better again. I even recovered the energy to start thinking again about reviving this blog. It still took two months to actually do it, but what the heck.

So, all you nice people out there, my lesson for today is: when the going get's tough, just walk away and have a vacation!

Well that's not true of course, but it does help. So next time I hope to write about my experience on the better of the two projects that I dare to qualify as a really 'agile' project.

How to get agile in an agile way

2006-10-03T21:46:00.000+02:00

For some peculiar reason I'm involved with three customers at the same time wanting me to assist them in changing their way of doing systems development. And I say peculiar, because although this is my expertise for many years already, there has not been even one customer asking me to help them with this for almost five years. Maybe it's the sign of the time, or maybe it is because I stopped dying my hair letting my gray hairs come through and customers finally start to take me seriously. I don't know.

Anyway. It is at the same time I see many people turning away from a Big Design Up Front, plan-based methodologies or (at the other side of the spectrum) from working without any formal method at all. Both have in common that they have an urgent need to get a better grip on the (planning of) the development process, and it is likely they will meet each other somewhere 'in the middle' where the agile methodologies are.

It's funny to notice that getting started with agile methodologies in itself also is best done in an agile way. With that I mean starting with a minimal set and from there, discover where and when you need to scale up. But although you start with a minimal set it still is important you are aware of which direction you want to go. It is even more important to ensure that the people that will be doing the actual work, agree this is the direction you all want to go. Otherwise the risk of resistance up to a level where you will fail misserably will be unacceptable high.

For that reason many of the techniques I suggest to introduce into the development proces, I actually use for introducing the techniques themselves. Like a meta-method if you will. Depending on what I think would fit, this could be using the MoSCoW principle for prioritization the changes. But also make them aware of the similarities between the Onsite Customer practice and them being involved in changing the development process.

This has many advantages. The most important ones are that I verify early in the process if people actually understand the technique and if it will work for them. While doing so I have the opportunity to use a kind of trial and error way of implementing a technique, and securing it not only in a formal way but also by creating enough support for it to last after I have left the building.

Practice what you preach

2006-09-25T22:12:00.000+02:00

Man, I could not have imagined in my wildest dreams that I would write about time boxing and prioritization again that soon. Or couldn't I? Whish it were true, as knowing myself it would be the mother of all lies to say so, as it was almost the day after my last posting that faith struck again. Read on and you will understand why I haven’t been updating this blog for more than two weeks now.

What was the case? I was just about to move in two weeks and still had a lot of things to do. One of them visiting a SOA Suite 'code camp' in Slovakia on which I was to present about Oracle Business Rules, and I still needed to prepare myself. But at the same time many things needed to be done privately to prepare for the move. I already had some extremely busy weeks before, so I saw going to Slovakia as an opportunity to escape and get some rest before the move itself. And boy, what a mistake that was.

As always it started with too many things to do in too little time. We have a parquet floor in our new house that looked so dull that I nearly fell asleep only by looking at it. So we decided that it needed a different colour. But also the ceiling, walls, casing, etc needed to be repainted. I hoped that during my stay in Slovakia the painting could be done and the floor soured. Painting alone would have taken two weeks the least when I would have done it myself, but as my brother in law was doing it and since he is a professional I hoped for some magic here.

And magic he did, but when I got back on Friday the floor still needed to be soured. I hoped (but was not really expecting) to be able to sour and clean in one day. Two of my best friends helped me out on Saturday, but on Sunday I was on my own. And despite my best efforts it took me until Monday before souring was finished and then I still had to clean, paint and enamel. So despite the fact I was supposed to go to the office I was pushing myself to the limit while trying to get the floor to the stage that it was painted, working for eight hours in a row, no rest except for a quick coffee.

I had to compensate Tuesday evening to get some urgent work done, so on Wednesday I still had not packed even one cardboard box. And what a contrast that was with my original planning that assumed that I would start packing somewhere during the Sunday! As a result I had one Wednesday evening and one Thursday left to pack before Friday, which was the day the remover would come. And I only had that Thursday because I could switch it with Friday, which originally was my day off.

So what you then do is stop thinking, start packing like a madman, and see how far you get. And at 01:00 we were almost done! But not quite, so when the remover came at 9:00 it was an utter chaos in our old house, and a couple of boxes were carried away while I was still packing them. There was only a small comfort in hearing that the removers were used to this because they saw this happening all the time. Despite my wife pledging me not to do so, I had to leave at 11:00 to the office. Later on my wife told me that that was the moment she knew for sure she was married to a lunatic.

Now what has this all to do with time boxing and prioritization? Everything! It perfectly illustrates the situation that we encounter very often being a customer wanting every requirement to be implemented, which in theory makes time-boxing using the MoSCoW principle and leave out Should Haves if they don’t fit anymore in the time-box (when there is a temporary work-around) almost impossible. In my case the requirement was that everything should be packed before the removers would come. The time-box is obvious: Friday 9:00 the removers are there, period.

However, no matter what kind of customer you have in practice MoSCoW can at least be used to determine a best order in which things should be done, doing the most important things with the biggest risks first. In my case that was preparing big and heavy things for removal, like securing the washing machine. I got that done. And even though customers say they want every requirement implemented, when no time or budget is left they suddenly are capable of making compromises and drop Should Haves like they couldn't care less. Just like that. In my case the compromise was that the removers left enough stuff behind so that I needed to drive myself two more times to get the old house empty as a work-around.

Maybe some next time I will tell you about the similarities between something that all people understand, being that you don’t leave your garbage to the new owners of your old house, and the principle of refactoring, which only a few software developers seem to understand.

Excuse me, but what are you talking about?

2006-09-06T18:20:00.000+02:00

You don't even need to watch carefully to discover that often two people that seem to be talking about the same, in fact could not be further from that. "Tell me something new", you might react. And you're right, I'm not telling something new, it happens all the time. And that is the saddest thing in my business, because it costs us all a huge amount of time, money and frustration.

Knowing all this it always amazes me how apathetic we seem to be about it, because it is so easy to do something about it. Try asking people what exactly it is they mean when they tell you something, that's all. Simple enough, wouldn't you say?

The other day I had a conversation with someone about preventing overrun on activities, and he said he was using the time boxing principle for that. But from the rest of the story it became clear that there still was a considerable track record of deadlines not being made. And as always the problem was that the customer kept on adding new requirements during the project and was not able to prioritize them properly, naturally. Now where have I heard that story before?

So I simply asked this guy what he meant with this "time boxing" principle. Well, not exactly like that, I must admit. In practice it is not always as that simple as that because before you know it people get the impression you think they are stupid or something. Or even worse they start to think that you are stupid, and you don't want that, do you? So what I asked instead was how they had implemented this "time boxing" principle. Do you notice how different the question is, but that in principle I ask the same?

And what he told me is that they had divided the functionality into smaller pieces that, on an average, could be implemented in a couple of weeks and that they had set a final delivery date for every piece of functionality to deliver. Plain and simple, right? "So what happens when you find out that something takes more time than anticipated?", I asked. "Well, we have planned for contingency, of course", was the reply. "And what if they customer finds out that the specifications are not complete?". "Well we create a change request for every extra thing they want and see if we can implement it in the time box. Otherwise we postpone it to the next time box or, if they really, really need it, increase the scope and set a new end date. But that will cost them, of course". "Have you ever hear of the MoSCoW principle?", I asked. "Huh?".

So what I found out with this simple question, is that the time boxing principle had been implemented only poorly, because they were not applying a proper prioritizing principle, which coincidentally will make time boxing fail without.

I'm not going to explain here what time boxing and the MoSCoW principle are all about. You can easily find out elsewhere. And I'm also not going to tell you that using them is as easy as it sounds, because it's not. Maybe some other time I will tell you what you can try to do about that. I will also not explain here that normally you don't plan for contingency when you do time boxing and what a waste of time making change request forms is in that case.

But I do hope I gave you an example how asking simple questions can help you in
preventing people from wasting each others and especially your precious time.

Live longer with source control!

2006-08-30T19:56:00.000+02:00

I confess, I made a bad mistake some time ago. I started to work on a project that had no proper source control system in place. Instead, zip files with source code were copied back and forth and merging was done manually using WinMerge. And I did not take a stand and say: "I want a source control system or I'm outta here!". Not that it would have helped, but it is the principle that counts.

As a result I spent at least two hours every week merging code. And then the risk that something would go wrong and I would need to spend at least another hour to find out what was going on and correct the problem! As I have the memory of a gold fish (*) those were the two hours I picked up the phone saying "I call you back!" only to put it down without taking the effort to notice who was calling. People could have said my boss was standing outside giving free money to whoever asked for it without me hearing them, let alone being interested. By Turing, what a stress!

Now I'm in a similar project situation where we do use a proper source control system, in this case Subversion. If they are using a source control system in Heaven, I'm sure it is Subversion. Although not flawless, but what a blessing it is! But if you are using CVS, that's OK too. And there are probably more cool source control systems that I don't know of, but since I don't know them, who cares.

By the way, did I already tell you I wrote a white paper about how to use CVS in combination with JDeveloper? No? OK, I wrote a white paper about how to use CVS in combination with JDeveloper. You can download it from OTN. It is based on JDeveloper 9.x and perhaps a little bit outdated, but it still explains the principles of source control and what it can do for you. JDeveloper 10.1.3 already offers much better integration, also with Subversion. I hope to be able to write a white paper about that too. Maybe it helps when you call my manager and tell him you want me to write it. Just say the word and I'll give you his phone number!

Anyway, the point I try to make here is this. The effort of installing a source control system like Subversion and maintain it, is only a fraction of the time people spend on things like manual merging. And then the risk of things going wrong. Come on project managers, you can do without that! It is hard to give you exact metrics, but based on experience I dare to say you already crossed the break-even point when two people worked together for a month. After that, you definitely keep money in your pocket. And the developers are happier too!

And once you have it, you suddenly realize it offers opportunities you were not aware of they existed before. Like going back to a previous version (for example the one that is in acceptance test) on instant, fix an issue there and almost with one click of the mouse go back to the current version and move on. Or find out who implemented that one great feature so you can give him or her the medal of honor of the programmer of the week!

I promised myself that next time they send me to a project where no proper version control is in place, I simply refuse saying that it's bad for my health. I advice you to do the same and tell your project manager I said so. I'm sure no further explanation will be needed.

(*) Actually, the saying 'having the memory of a gold fish' appears to do great injustice to this fellow creature. If it's true what they say at Goldfish pass memory test I would praise myself fortunate when I would have the memory of a gold fish.

Don't ever let me hear you say "non-functional" again

2006-08-28T10:01:00.000+02:00

The other day I had I discussion with a colleague about what many people call "non-functional" requirements (as opposed to functional requirements), being requirements regarding performance, security and so on. This colleague argued that “non-functional” exactly expressed what these requirements are.

“OK”, I said. “Suppose you want to buy a car. Can you agree with me that its top speed is a non-functional requirement? Especially since normally any car can easily drive over 120 km/hour, which is the speed limit in Holland anyway, right?” “I agree that the top speed is a non-functional requirement, yes”, he replied. “So what you are saying here is that you might not even take a look at the specifications, but just assume that it can drive fast enough. But at the same time you would not take the car when it would not meet functional requirements like having a CD player, enough space to stow your kids and a sun-roof?”, I asked. “Yes”, he said, “that’s what I’m saying”.

“Now what if you are a sailor that happens to live in Munich but who sails from Kiel. Mind you, the distance between these two cities is about 700 km (435 miles). But fortunately - with the exception of a few spots - there is no speed limit in Germany! You have a lovely wife and you want to spend as much time with her as possible. Would you then have a look at its top speed?”, I asked. “Ermmm…”

The point I had made here is that what some people consider to be non-functional requirements could very well be a functional one to somebody else. Like logging would be a non-functional requirement for the average user, while an EDP auditor might have a totally different opinion about that.

So please stop using the term “non-functional”, people! In real life people say non-functional to something that does not work. It is confusing and imprecise, where the development of information systems is a very precise job. Use “supplementary requirements” instead. The only disadvantage I can think of is that I still have problems saying "supplementary requirements" without twisting my tongue.

Blogito ergo sum

2006-08-24T12:42:00.000+02:00

"I blog, therefore I am". Withouth having checked this, there is no doubt in my mind there are many bloggers that have used this phrase. So why am I using it?

Blogging as a phenomena has intrigued me ever since I'm aware of it's existence. Vain as I am, I can understand people blogging. But for a long time I failed to understand why people would bother to read them. Don't they have better things to do than read other people's whimsical thoughts, like a job or something? To tell you the truth, I still don't understand.

That is to say, with a few exceptions. There are a couple of blog's that I read once and a while. I won't give you names as I don't want to hurt the feelings of people that I will fail to mention. But most of them have one thing in common, which is that they are job related. In my case that means they have software development as main topic, especially where it concerns Oracle which is my current employer. And the bloggers themselves also have something in common, being that in their field they are somebody".

So, to give you a reason to read my blog during working hours and prevent you from getting fired on instant when your boss finds out about it, I promise I will keep this blog also job related. That is, when your job concerns software development, especially when the Oracle toolstack is involved in that. Somewhere.

And occasionaly I will share some whimsical thoughts with you. Hope you will forgive me.

Cheers!