Virtualizing Monitoring Infrastructure: Virtual CEP

Virtualization offers clear advantages when it comes to storage, server and desktop virtualization. Today we can run Mac OS, Windows, Linux and others OS on a single hardware all at the same time with seamless integration and switching from one to another. Servers and storage can be consolidated easily and reduce energy, costs associated with new hardware, storage and management overhead. Benefits are clear especially for those managing complex data centers.

Virtualization is an interesting concept when applied in the area of application performance monitoring, Business Activity Monitoring and a like. When we apply concepts of CEP (Complex Event Processor), it would be nice to achieve the following:

• Linear scalability with increased loads -- meaning it takes the same effort to go from 1 million rules/sec to 2 as 10 million to 20 million
  
• Installation, deployment and reconfiguration within minutes
• Unlimited processing capacity (only limited to the physical boundaries of the servers) -- meaning the number and rate of events that can be processed per unit of time.
  

Virtualizing CEP capabilities delivers these benefits - Virtual CEP Environment (VCE). VCE is a virtual collection of individual CEP engines pooled together as a single virtual processing environment. In this model the processing capacity of each instance can be added to the overall VCE processing capacity.
VCE can be implemented on top of virtual machines such as VMWare, XEN, Parallels -- meaning virtual machines on a single and separate hardware boxes can be pooled together to deliver processing capability.

The diagram above depicts VCE concept where 3 physical servers are aggregated into a single virtual CEP capable of processing 5.5 million rules/sec. It is easy to add more capacity by simply instantiating CEP instances either on existing box/VM or additional hardware. Instances can also be taken offline with little or no disruption.