SOA - yet another layer of complexity

In my view introduction of SOA will push IT complexity beyond a point that can be effectively managed by todays management infrastructure. Why do I say that? Several reasons:

1. SOA based applications will require management tools that will need to adapt to changing requirements much faster then can be accomplished using todays management tools
2. SOA based applications would be able to connect a vast array of disintegrated processes together creating a level of of interconnectedness that can no longer be handled by event based health monitors.
3. The number of tools and technologies required just to manage and a SOA environment in itself creates a management and complexity overhead.
4. Complexity simply will grow exponentially if not managed properly or methodically reduced and simplified.
   

So SOA infrastructure has a potential of bringing a complex environment that is out of control and much more difficult to manage then typical 3 tier, client/server or monolithic applications. So how do we deal with the benefits on one end and growing complexity on the other? Well I believe there is only 1 way to do this -- investment in SOA governance tools and practices. This means not only technologies, but also skills, processes and practices. There are several methods that look very attractive:

1. Virtualization -- one way to deal with complexity is try to simplify the environment -- do more with less -- less hardware, less network, less software. Less is more very often.
2. Create and Maintain Capability Matrix -- rather then tools matrix and map tools you have to capabilities. Obviously you need to have all critical capabilities fulfilled by at least one technology or tool. Duplicates can be eliminated. Think in terms of technology capabilities rather then tools and technologies. Use this matrix to simplify your environment.
3. Use KISS approach -- I like this approach since my college years. Keep It Stupid and Simple. Apply it in every possible aspect. Complicated setup are hard to maintain and manage.
   

If left unchecked the benefits of SOA (and there are many) will be eroded by increased maintenance, degradation of quality of service , loss of revenue and customer loyalty. So implementing a successful SOA environment is about well balanced growth -- focusing not just on adding capabilities but also looking into how to reduce the complexity.

So what is Enterprise Service Bus?

It seems like there is a confusion around the meaning of Enterprise Service Bus (ESB). To some ESB is a specific product offering from vendors such as IBM, Oracle, BEA etc. To others ESB is a set of technologies such as messaging, application server, web services, data services integrated together -- basically a technology fabric that lets users unlock the value of SOA. I tend to a agree with the later definition.

ESB is a specific implementation of SOA and would have to be combination of various technologies integrated using SOA paradigms and standards.

Once we have the definition right, we can start asking ourselves questions: "What does it mean to manage an ESB?" In my view, based on the later definition of the term ESB, it means that all components of ESB must be managed in a single inform fashion -- not just an individual product offering. "How do we measure the quality of service provided by ESB?" This is a much more complex question to answer and not just due to the sheer complexity of technologies involved but also due to the different perceptions/definitions of what QOS actually means for different users and stakeholders of the SOA/ESB environment.
At any rate, proper definition of ESB is essential before more complex questions can be asked and answered.

Essential building blocks of a sound SOA environment

Many SOA implementations are evolutionary projects, usually starting with application integration and then moving up the food chain -- web services, ESB, business service and workflow management. Few shops have actually implemeted all layers of SOA, but works under way. Below are the basic building blocks for successfull SOA implementation:

• Service Registry (UDDI)
• Common Message Bus (JMS, MQ, others)
• Common Application Service Platform (Application Servers)
• Common Service Orchestration and collaboration engine (Enterprise Service Bus)
• Data transformation and integration services (Message borkers)
• Common Management and Monitoring Services

Obviosly these technology peices must be complemented by proper orghanization and skillset to put into and effective enterprise integration fabric. So SOA is not just about technology but also about people, processes and best practices to make it successful. And that is why SOA can not be bought or aquired like other products. It must be built, nurtured and refined every step of the way. There is a term that sums it all up -- SOA governance.

Improving performance of Java applications

During many implementation of high performance java applications, I found that the following strategies lead to be better performaning java apps -- especially better performing server apps.

• Limit the number objects instanciations. This might actually be difficult to do, however with the right design and approach tremendous saving are possible. Object pools is one way to go, but may not be applicable in every situation.
• Avoid or improve object serialization if possible. Object serialization (accross networks) happens to be very slow. It offers ease of use, but you pay in performance. Analyze the classes that are serialized and make sure only required attributes are serialized. Hashtables and complex structures are the biggest hit.
• Analyze and optimize string related operations -- especially concatentations, string buffer extensions are usually expensive.
• More threads does not mean better throughput -- thread management adds overhead and after a certain number may actually degrade performance. Multiple CPUs might add marginal improvement especially when threads are dependent and sychronized with other threads. Revise your thread model to take advantage of multiple CPUs.
• Avoid synchronous IO -- using asynchronous socket IO, file IO is prefered, let the worker threads to the job. I happen to like the principle of accepting the client request, and let it process by the worker thread and unblock the client app -- then notify the client when task completes.

Is SOA all that?

Service Oriented Architecture seems to be the new buzz word everywhere!!! SOA will solve all the problems, everyone wants SOA -- or so I hear. Anyway, I remember similar claims when CORBA was coming to town. It was supposed to change the way IT does business. To me SOA is XML/SOAP based CORBA with evolving standards around it... I might be wrong.

Yet here we are and very little has changed. Well, maybe not, one thing is for sure -- IT complexity is growing at an alarming rate and I am wondering -- how much complexity can we a take until we loose control. IT shops take this concern seriously and spend big bucks trying to manage this complexity. I have a question --- Will introduction of SOA help IT better manage complexity and reduce cost or will it fail just like its predecessor? So far I see lots of traction in the field, but also a lot of skepticism.

What I do know for sure, is that organizations have to find new ways of leveraging technology and turn it into a competitive advantage. SOA is an experiment. I like experiments it keeps us busy and in business.