JDeveloper Class Modeler Revisited

About two years ago I published a couple of white papers about UML, unit testing, and source control, and how to do that using JDeveloper 9.x / 10.1.2. These white papers still can be found on the white paper archive on OTN and are:

• Getting Started with Use Case Modeling
• Getting Started with Class Modeling
• Getting Started with Activity Modeling
• Getting Started with Unit Testing
• Getting Started with CVS
  

As since then the world has turned around a couple of times I took the time and started revisiting them and bring them up-to-date with JDeveloper 10.1.3.

The first paper I have revisited is the one for UML class modeling. To be honest, not much has changed. The only thing I find worth mentioning is that for a class attribute you now can specify if and when it can be updated, using a new property called Changeability. Do I hear a big "Wauw!"? No? Hmm.... Well, you know, of all the UML modelers it is the most mature one, as it also is the base of the Java class, ADF business components, and database modelers, which are (implementation specific) specializations of the UML class modeler.

Only thing that really bugs me is that you still cannot specify unique identifiers for a UML class. I must admit, it's not in UML 2.0 as well, but we are a company that happens to sell relational databases (among other things) and for many of us specifying a unique identifier is like riding a bike is for Lance Armstrong! One of the consequences is that it still is a bad idea to transform a UML class model to ADF business components, because (as I explain in the white paper) ADF entity objects require primary keys and the transformer therefore alphabetically picks the first attribute, and that is not necessarily the right one. The result is a lot of (error prone) work fixing the ADF model, making that I advice against transforming a UML class model to ADF business components. And that's too bad as most people I know use ADF.

"But hey, you are talking JDeveloper 10.1.3, and is that not already an archived version?", I hear you say. Well, almost folks. Recently the JDeveloper 11g - Technology Preview has been released, and YES! the new features overview tells me there is a major change regarding the UML class modeler.

From a feature point of view there still is not much to tell, although one of my enhancement requests from two years ago has been implemented, being that attributes can now also visually be order logically (instead of alphabetically) allowing you, for example, to put the most important ones at the top and attributes that logically belong together above each other. Is that a big deal? Believe me, you want to review a big UML class model it is!

But yet I do have the impression the UML modelers of JDeveloper are going to be taken more seriously than before, as the new features page also state that the UML class modeler has been rewritten on a new graphical engine that provides better better performance and scalability. And not only that (and I quote): "Future releases will see other modelers being re-hosted on this new framework". Initially we will most benefit from this as it will greatly improve the usability of the modelers, not only from an editing point of view but also regarding publishing the models. And whoever has been on a project that involved more than just a few tables knows how important that can be.

Can't wait to get my hands dirty on that! But first I need to revisit the other modelers as well, so watch out for new postings about that. I already can tell you that the UML use case modeler of 10.1.3 definitely has more seriously been changed!

My Agile Project (based on a true story)

What makes a project 'agile'? Are you only doing an agile project when you apply all principles of eXtreme Programming for example? From the way I phrased these questions you probably already concluded that I think differently.

What agile projects really make agile is best formulated in the Agile Manifesto. Among others the following principles are included:

• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and good design enhances agility.

Not specifically included (probably assumed to be intrinsic) is the following principle:

• Address the biggest risks first.
  

I think these principles best explain why my latest project has been such a success, meaning that it as delivered in time, within budget, with a happy customer, and last but not least, happy end-users!

But before I explain, let me first tell you a little bit about the project. It concerns maintenance of an existing system that is over more than five years old. The system has been build using Oracle Designer / Developer, with a lot of PL/SQL code in the database of which a considerable part enforces rules and regulations. Rules and regulations being changed significantly was the reason to start the project and make the system adapted accordingly.

The major challenges we focussed were these:

• A first increment should go into production before January (it was end of October when I got first involved so we had only two months).
• The PL/SQL code was set up using a lot of (what we in retrospective may call) bad practices, like one huge package that 'does it all' with tightly coupled procedures of which many had no clear responsibility.

Although we talk about rules and regulations, many things were up for interpretation. When you read one of my previous posts you know I try to avoid a Big Design Up-Front where feasible. Normally you have to justify this to people that have not yet bought into agile development, but in this case the time constraint made it even impossible to create such a thing.

So what you gonna do? Well, first of all we (meaning a team of 3 developers, a client-project manager and ambassador user) decided to meet on a daily basis to monitor the status, exchange information and address issues as soon as they emerge. In this setting we made up a high-level MoSCoW-list, determined the dependencies between the requirements, and decided upon their prorities (primarily based on dependencies and biggest risks first). Using this as input, we then thought of an approach how we best could adapt the existing system in an incremental way. We created a low-level MoSCoW-list with a scope of two weeks, delivered what we produced and based on our experience created the next low-level MoSCoW-list.

Sounds easy, wouldn't you agree? Yeh, right...

We are dealing with people here and in general people have a few pretty persistent habitst that must be dealt with. Like many people find it counter-intuitive to design and implement functionality just to achieve a foreseable goal, knowing that in some near future they probably have to change it as a result of evolving insight. Main reason is that we have a tendency to try to do it right the first time to prevent needing to change code later on which just seems like overhead. Unfortunately experience shows almost none of us is capable overseeing everything that needs to be taken into account upfront. Discrepancies between vision and reality emerge the minute we start to build the real thing and especially become apparent when end-users are confronted with it.

Another tendency we have, is starting with a couple of simple things just to get a sense of achievement, while postponing the more complex issues, hoping they kind of solve themselves as time goes by. But those issue are the ones with the biggest risks associated with them and when they do not solve themselves, it often is too late and you find yourself being in big trouble. I have never done research to find out, but I do believe that addressing high-risk issues too late is the major cause of budget overrun, deadlines not being met or deliverance of considerable less functionality than planned for.

So what agile methods therefore promote is developing small increments that are delivered frequently. This will prevent us from going the wrong direction for too long as we will notice it soon after that happens. To prevent that changing things later indeed results in a lot of overhead, we maintain a simple design by creating the simplest thing that could possibly work. Whenever the design starts to get complex, we refactor to make it simple again. To allow for refactoring without breaking the functionality, we apply unit testing to prove the system still works afterwards. To reduce risk we ensure that the code one person creates works with to code of others, by continuously integrating everything we produce. And wherever feasible, we start with those aspects that are the least clear to us how to proceed with, to find out as soon as possible how they can be resolved and what the impact on other aspects is.

All this is not just a nice theory. We did just that and proved it works!

We consulted each other at least once a day, keeping eachother on track with maintaining a simple design and refactoring when necessary. On an average we delivered every two weeks and from increment to increment added new functionality while at the same we improved the existing poor architecture. The result is a system that not only does what it is supposed to do, but which also is based on a solid architecture with excellence through simplicity!

Only time will tell, but I have a strong feeling that we considerably reduced the costs for maintenance and perhaps made that the system will survive for yet another five years.

Can I take you(r method) home with me?

Already a couple of years I'm a member of a group within the Oracle Methods Group that is responsible for the Oracle Unified Method, or OUM for short. OUM is based on the Unified Process (UP), as is the Rational Unified Process (RUP) and as such has many similarities with RUP.

"So why yet another method?", you might ask. Well, there are plenty of reasons, but to mention a few important ones:

• Years ago Oracle created the Custom Development Method. That method has been a great success for ourselves as well our customers, and many people are used to it. While creating OUM we are able to make it relatively easy for people to "migrate" from one method to the other by using the same kind of structure.


• RUP is optimized for being used with Rational/IBM tooling. And as Oracle has it's own extensive set of tooling it makes sense to have a method taking that into consideration. Of course, to a certain extend RUP is configurable and extendable, but that has its limitations and would not facilitate the first point.


• Maybe you do not know Oracle for running IT projects, but we do have a lot of expirience, especially regarding projects that involve a database or a Service Oriented Architecture and found that there is a lot of room for improvement in how the (Rational) Unified Process addresses those issues, if at all.


• Last but not least, the Unified Process does not address cross-project/enterprise-level issues very well. Nowadays there is also the Enterprise Unified Process. Although a valuable extension to UP for us it still has a few shortcomings. Among them that it still does not address all aspects we found to be important to cover, like IT Governance.

On a very short notice the Oracle Methods Group will make OUM 4.3 available. That version will have a first cut of what we consider to be necessary to add to UP to make it an enterprise-level development method. For that we added an extra "dimension" to UP which we call Envision, as well as a couple of enterprise-level processes.

OUM 4.3 will also be the first release that (to a limited extend) is available for our customers as well. As the specifics are yet to be determined, I cannot tell you more yet, but I will keep you posted.

I know, going to the pub telling people that I'm working on OUM "which now has Envision as well!" won't get me the hottest chicks wanting to take me home. But hey, to most girls telling you are in the IT business is a turn-off anyway, so now you know I dare to tell you I'm quite exited about this release going to hit the streets!

And as soon as it does, I will let you know ...

When the going get's tough ...

Man, have I been busy lately!

Actually that is a very lame excuse for not having posted for a long, long time. The excuse even gets lamer, when considering the reasons for being busy, namely being lead developer / project manager for two projects at the same time. So what happened to me using agile development princples and being in control and all that crap? Do I not only talk the talk but also walk the walk?

Let me put it this way. If I hadn't had done so I probably would have had a complete burn-out, as just like in real life both projects became the busiest at exactly the same time. And just like the average male, I'm very bad at doing two things at the same time. So I had to go back to my manager and say: "Hey I only have one brain (and what a fine brain that is, ahum) but you are asking me to do a two-brain's job. So if nothing is going to change, I need to make one of those jobs a no-brainer so that I can concentrate on the other one." And as nothing changed, that was what I did.

This made the other project manageable again, but for someone like me it did not make my job much fun anymore as I like to do things properly. So that was when I decided to go to Costa Rica for holidays. When I got back, I felt soo much better again. I even recovered the energy to start thinking again about reviving this blog. It still took two months to actually do it, but what the heck.

So, all you nice people out there, my lesson for today is: when the going get's tough, just walk away and have a vacation!

Well that's not true of course, but it does help. So next time I hope to write about my experience on the better of the two projects that I dare to qualify as a really 'agile' project.

How to get agile in an agile way

For some peculiar reason I'm involved with three customers at the same time wanting me to assist them in changing their way of doing systems development. And I say peculiar, because although this is my expertise for many years already, there has not been even one customer asking me to help them with this for almost five years. Maybe it's the sign of the time, or maybe it is because I stopped dying my hair letting my gray hairs come through and customers finally start to take me seriously. I don't know.

Anyway. It is at the same time I see many people turning away from a Big Design Up Front, plan-based methodologies or (at the other side of the spectrum) from working without any formal method at all. Both have in common that they have an urgent need to get a better grip on the (planning of) the development process, and it is likely they will meet each other somewhere 'in the middle' where the agile methodologies are.

It's funny to notice that getting started with agile methodologies in itself also is best done in an agile way. With that I mean starting with a minimal set and from there, discover where and when you need to scale up. But although you start with a minimal set it still is important you are aware of which direction you want to go. It is even more important to ensure that the people that will be doing the actual work, agree this is the direction you all want to go. Otherwise the risk of resistance up to a level where you will fail misserably will be unacceptable high.

For that reason many of the techniques I suggest to introduce into the development proces, I actually use for introducing the techniques themselves. Like a meta-method if you will. Depending on what I think would fit, this could be using the MoSCoW principle for prioritization the changes. But also make them aware of the similarities between the Onsite Customer practice and them being involved in changing the development process.

This has many advantages. The most important ones are that I verify early in the process if people actually understand the technique and if it will work for them. While doing so I have the opportunity to use a kind of trial and error way of implementing a technique, and securing it not only in a formal way but also by creating enough support for it to last after I have left the building.

Practice what you preach

Man, I could not have imagined in my wildest dreams that I would write about time boxing and prioritization again that soon. Or couldn't I? Whish it were true, as knowing myself it would be the mother of all lies to say so, as it was almost the day after my last posting that faith struck again. Read on and you will understand why I haven’t been updating this blog for more than two weeks now.

What was the case? I was just about to move in two weeks and still had a lot of things to do. One of them visiting a SOA Suite 'code camp' in Slovakia on which I was to present about Oracle Business Rules, and I still needed to prepare myself. But at the same time many things needed to be done privately to prepare for the move. I already had some extremely busy weeks before, so I saw going to Slovakia as an opportunity to escape and get some rest before the move itself. And boy, what a mistake that was.

As always it started with too many things to do in too little time. We have a parquet floor in our new house that looked so dull that I nearly fell asleep only by looking at it. So we decided that it needed a different colour. But also the ceiling, walls, casing, etc needed to be repainted. I hoped that during my stay in Slovakia the painting could be done and the floor soured. Painting alone would have taken two weeks the least when I would have done it myself, but as my brother in law was doing it and since he is a professional I hoped for some magic here.

And magic he did, but when I got back on Friday the floor still needed to be soured. I hoped (but was not really expecting) to be able to sour and clean in one day. Two of my best friends helped me out on Saturday, but on Sunday I was on my own. And despite my best efforts it took me until Monday before souring was finished and then I still had to clean, paint and enamel. So despite the fact I was supposed to go to the office I was pushing myself to the limit while trying to get the floor to the stage that it was painted, working for eight hours in a row, no rest except for a quick coffee.

I had to compensate Tuesday evening to get some urgent work done, so on Wednesday I still had not packed even one cardboard box. And what a contrast that was with my original planning that assumed that I would start packing somewhere during the Sunday! As a result I had one Wednesday evening and one Thursday left to pack before Friday, which was the day the remover would come. And I only had that Thursday because I could switch it with Friday, which originally was my day off.

So what you then do is stop thinking, start packing like a madman, and see how far you get. And at 01:00 we were almost done! But not quite, so when the remover came at 9:00 it was an utter chaos in our old house, and a couple of boxes were carried away while I was still packing them. There was only a small comfort in hearing that the removers were used to this because they saw this happening all the time. Despite my wife pledging me not to do so, I had to leave at 11:00 to the office. Later on my wife told me that that was the moment she knew for sure she was married to a lunatic.

Now what has this all to do with time boxing and prioritization? Everything! It perfectly illustrates the situation that we encounter very often being a customer wanting every requirement to be implemented, which in theory makes time-boxing using the MoSCoW principle and leave out Should Haves if they don’t fit anymore in the time-box (when there is a temporary work-around) almost impossible. In my case the requirement was that everything should be packed before the removers would come. The time-box is obvious: Friday 9:00 the removers are there, period.

However, no matter what kind of customer you have in practice MoSCoW can at least be used to determine a best order in which things should be done, doing the most important things with the biggest risks first. In my case that was preparing big and heavy things for removal, like securing the washing machine. I got that done. And even though customers say they want every requirement implemented, when no time or budget is left they suddenly are capable of making compromises and drop Should Haves like they couldn't care less. Just like that. In my case the compromise was that the removers left enough stuff behind so that I needed to drive myself two more times to get the old house empty as a work-around.

Maybe some next time I will tell you about the similarities between something that all people understand, being that you don’t leave your garbage to the new owners of your old house, and the principle of refactoring, which only a few software developers seem to understand.

Excuse me, but what are you talking about?

You don't even need to watch carefully to discover that often two people that seem to be talking about the same, in fact could not be further from that. "Tell me something new", you might react. And you're right, I'm not telling something new, it happens all the time. And that is the saddest thing in my business, because it costs us all a huge amount of time, money and frustration.

Knowing all this it always amazes me how apathetic we seem to be about it, because it is so easy to do something about it. Try asking people what exactly it is they mean when they tell you something, that's all. Simple enough, wouldn't you say?

The other day I had a conversation with someone about preventing overrun on activities, and he said he was using the time boxing principle for that. But from the rest of the story it became clear that there still was a considerable track record of deadlines not being made. And as always the problem was that the customer kept on adding new requirements during the project and was not able to prioritize them properly, naturally. Now where have I heard that story before?

So I simply asked this guy what he meant with this "time boxing" principle. Well, not exactly like that, I must admit. In practice it is not always as that simple as that because before you know it people get the impression you think they are stupid or something. Or even worse they start to think that you are stupid, and you don't want that, do you? So what I asked instead was how they had implemented this "time boxing" principle. Do you notice how different the question is, but that in principle I ask the same?

And what he told me is that they had divided the functionality into smaller pieces that, on an average, could be implemented in a couple of weeks and that they had set a final delivery date for every piece of functionality to deliver. Plain and simple, right? "So what happens when you find out that something takes more time than anticipated?", I asked. "Well, we have planned for contingency, of course", was the reply. "And what if they customer finds out that the specifications are not complete?". "Well we create a change request for every extra thing they want and see if we can implement it in the time box. Otherwise we postpone it to the next time box or, if they really, really need it, increase the scope and set a new end date. But that will cost them, of course". "Have you ever hear of the MoSCoW principle?", I asked. "Huh?".

So what I found out with this simple question, is that the time boxing principle had been implemented only poorly, because they were not applying a proper prioritizing principle, which coincidentally will make time boxing fail without.

I'm not going to explain here what time boxing and the MoSCoW principle are all about. You can easily find out elsewhere. And I'm also not going to tell you that using them is as easy as it sounds, because it's not. Maybe some other time I will tell you what you can try to do about that. I will also not explain here that normally you don't plan for contingency when you do time boxing and what a waste of time making change request forms is in that case.

But I do hope I gave you an example how asking simple questions can help you in
preventing people from wasting each others and especially your precious time.

Live longer with source control!

I confess, I made a bad mistake some time ago. I started to work on a project that had no proper source control system in place. Instead, zip files with source code were copied back and forth and merging was done manually using WinMerge. And I did not take a stand and say: "I want a source control system or I'm outta here!". Not that it would have helped, but it is the principle that counts.

As a result I spent at least two hours every week merging code. And then the risk that something would go wrong and I would need to spend at least another hour to find out what was going on and correct the problem! As I have the memory of a gold fish (*) those were the two hours I picked up the phone saying "I call you back!" only to put it down without taking the effort to notice who was calling. People could have said my boss was standing outside giving free money to whoever asked for it without me hearing them, let alone being interested. By Turing, what a stress!

Now I'm in a similar project situation where we do use a proper source control system, in this case Subversion. If they are using a source control system in Heaven, I'm sure it is Subversion. Although not flawless, but what a blessing it is! But if you are using CVS, that's OK too. And there are probably more cool source control systems that I don't know of, but since I don't know them, who cares.

By the way, did I already tell you I wrote a white paper about how to use CVS in combination with JDeveloper? No? OK, I wrote a white paper about how to use CVS in combination with JDeveloper. You can download it from OTN. It is based on JDeveloper 9.x and perhaps a little bit outdated, but it still explains the principles of source control and what it can do for you. JDeveloper 10.1.3 already offers much better integration, also with Subversion. I hope to be able to write a white paper about that too. Maybe it helps when you call my manager and tell him you want me to write it. Just say the word and I'll give you his phone number!

Anyway, the point I try to make here is this. The effort of installing a source control system like Subversion and maintain it, is only a fraction of the time people spend on things like manual merging. And then the risk of things going wrong. Come on project managers, you can do without that! It is hard to give you exact metrics, but based on experience I dare to say you already crossed the break-even point when two people worked together for a month. After that, you definitely keep money in your pocket. And the developers are happier too!

And once you have it, you suddenly realize it offers opportunities you were not aware of they existed before. Like going back to a previous version (for example the one that is in acceptance test) on instant, fix an issue there and almost with one click of the mouse go back to the current version and move on. Or find out who implemented that one great feature so you can give him or her the medal of honor of the programmer of the week!

I promised myself that next time they send me to a project where no proper version control is in place, I simply refuse saying that it's bad for my health. I advice you to do the same and tell your project manager I said so. I'm sure no further explanation will be needed.

(*) Actually, the saying 'having the memory of a gold fish' appears to do great injustice to this fellow creature. If it's true what they say at Goldfish pass memory test I would praise myself fortunate when I would have the memory of a gold fish.

Don't ever let me hear you say "non-functional" again

The other day I had I discussion with a colleague about what many people call "non-functional" requirements (as opposed to functional requirements), being requirements regarding performance, security and so on. This colleague argued that “non-functional” exactly expressed what these requirements are.

“OK”, I said. “Suppose you want to buy a car. Can you agree with me that its top speed is a non-functional requirement? Especially since normally any car can easily drive over 120 km/hour, which is the speed limit in Holland anyway, right?” “I agree that the top speed is a non-functional requirement, yes”, he replied. “So what you are saying here is that you might not even take a look at the specifications, but just assume that it can drive fast enough. But at the same time you would not take the car when it would not meet functional requirements like having a CD player, enough space to stow your kids and a sun-roof?”, I asked. “Yes”, he said, “that’s what I’m saying”.

“Now what if you are a sailor that happens to live in Munich but who sails from Kiel. Mind you, the distance between these two cities is about 700 km (435 miles). But fortunately - with the exception of a few spots - there is no speed limit in Germany! You have a lovely wife and you want to spend as much time with her as possible. Would you then have a look at its top speed?”, I asked. “Ermmm…”

The point I had made here is that what some people consider to be non-functional requirements could very well be a functional one to somebody else. Like logging would be a non-functional requirement for the average user, while an EDP auditor might have a totally different opinion about that.

So please stop using the term “non-functional”, people! In real life people say non-functional to something that does not work. It is confusing and imprecise, where the development of information systems is a very precise job. Use “supplementary requirements” instead. The only disadvantage I can think of is that I still have problems saying "supplementary requirements" without twisting my tongue.

Blogito ergo sum

"I blog, therefore I am". Withouth having checked this, there is no doubt in my mind there are many bloggers that have used this phrase. So why am I using it?

Blogging as a phenomena has intrigued me ever since I'm aware of it's existence. Vain as I am, I can understand people blogging. But for a long time I failed to understand why people would bother to read them. Don't they have better things to do than read other people's whimsical thoughts, like a job or something? To tell you the truth, I still don't understand.

That is to say, with a few exceptions. There are a couple of blog's that I read once and a while. I won't give you names as I don't want to hurt the feelings of people that I will fail to mention. But most of them have one thing in common, which is that they are job related. In my case that means they have software development as main topic, especially where it concerns Oracle which is my current employer. And the bloggers themselves also have something in common, being that in their field they are somebody".

So, to give you a reason to read my blog during working hours and prevent you from getting fired on instant when your boss finds out about it, I promise I will keep this blog also job related. That is, when your job concerns software development, especially when the Oracle toolstack is involved in that. Somewhere.

And occasionaly I will share some whimsical thoughts with you. Hope you will forgive me.

Cheers!