Using Apache Camel to Manage Amazon EC2 Startup/Shutdown

A lot of you are likely familiar with Amazon's EC2 cloud service. It's a "platform-as-a-service" solution that allows you to rent, on-demand, virtual machines (VMs). Those VM instances can run a variety of OSs, but the most common are Linux-based distros. Amazon offers a whole variety of VM types, depending upon your need. They range from high-performance VMs, to those sporting a huge amount of RAM.

While the lower-end Amazon instances are very cost-effective, they can still get rather pricey over time. In many cases, you likely don't need them running all-the-time. Using Amazon's EC2 API, you can programmatically manage just about any aspect of the VM, from startup to shutdown.

My son Owen recently has been using an EC2 instance to host some popular games such as Mindcraft. By hosting it himself, he can make modifications to the game logic, and apply other mods (apparently). However, since he's in school during the day, the instance really doesn't need to running all the time.

Given that, I had started to just write some Java routines that leverage the Amazon API to startup or shutdown his EC2 instance. I was then going to use Quartz to provide the scheduling capabilities. This would obviously work fine. However, I figured it might be a whole lot easier to use Apache Camel to manage the lifecycle.   In this post, I'll describe how I did that -- here's a link to the Eclipse project (you don't have to use Eclipse to run the project -- you can just unzip it as well).

Prerequisites

In order to follow this example, I'm pretty much assuming you're pretty familiar with Java and related tools such as Maven and Eclipse. To run the project yourself, you'll need to have Java and Maven installed. While recommended, you really don't need Eclipse if you just want to run everything as-is from the command line.

Also, since we're using this project to start/stop Amazon EC2 instances, I also presume you have an EC2 account setup with an EC2 instance available to use.

To use the example, all you have to do is unzip the project into a folder of your choice. If you have Eclipse configured with the Maven plugin, you can simply import it -- within Eclipse choose File->Import->Existing Projects into Workspace. Then, choose the option Select Archive File, and select the zip file.

Quick Intro to Camel

While this article isn't intended as a introduction to Camel (I will actually have a future post on this), a brief intro is necessary. Camel can be described as a messaging orchestration and routing product. It shares many similar characteristics to a conventional ESB, insofar as it contains a wealth of different protocol adapters. However, it's a lot lighter-weight than an ESB, and doesn't have it's own container in which it runs. As with most ESBs, you can also add your own components to extend the out-of-the-box functionality.

There are two mains ways in which you can configure Camel. Once is using the Java DSL, and the other is the XML-based Spring DSL. We're going to use the Spring approach for this project, since that's the one I generally prefer (is also a lot easier to learn if you don't know Java).

Here's a simple example of a configuration that produces the customary "Hello World" output to the console:

<beans xmlns="http://www.springframework.org/schema/beans"

    // namespaces not shown >

    <camel:camelContext xmlns="http://camel.apache.org/schema/spring">

        <!-- this route will run a remote request against an ec2 instance -->

        <camel:route>

            <camel:from 
              uri="timer://myTimer?fixedRate=true&amp;period=10&amp;repeatCount=1"/>

            <camel:setBody>

                <camel:constant>Hello World!</camel:constant>

            </camel:setBody>

            <camel:to uri="stream:out"/>

        </camel:route>

    </camel:camelContext>

</beans>

For those of you who know Spring, the above should seem pretty familiar. The top-level element is the typical beans element, which has associated with it a bunch of namespace definitions (not shown, but this simple example can be found in the camel-spring-hello-world.xml file in the project).  The most relavent part for our discussion starts with the camel:route element. That's where we define the route that the message will traverse. The message is initiated using the camel:from element's @uri definition. In this case, we're using the Camel Timer component (there are something like 80 components in Camel). The part which follows the timer: defines the frequency by which a message will be initiated (kicked-off). In this case, it will start generating messages every 10 milliseconds following startup based upon the setting in the @period element. However, because @repeatCount is set to 1, it will only generate a single message.

Most of the time, the camel:from element is used poll or periodically call a remote service such as an FTP site (for periodically scanning for files) or some HTTP service. In this case, only an empty message is being generated. So, the next statement in the route defines a message body. In this case, it's the proverbial "Hello World!". That is done using the camel:setBody element. Since we're assigning it a static value, we use child node camel:constant to set the static value (values can also be set more dynamically using expressions, which we'll see later).

Lastly, we're using the stream component to output the results (will print "Hello World!")  to the command line. If you want to run this very simple demo, edit the pom.xml file located at the root directory so that the configuration element within the project/build/plugins/plugin element for the org.apache.camel resembles:

   

         META-INF/spring/camel-spring-hello-world.xml

Then, run at the command line, enter  mvn camel:run. This will launch camel in an embedded type fashion and run the route.

Moving on to the EC2 Example

With that basic overview in mind, let's dive into the real problem we are trying to solve -- using Camel to schedule the starting and stopping of an Amazon EC2 instance. 

Camel comes out-of-the-box with some existing integrations to Amazon's web services. However, there isn't one specifically for EC2. However, Amazon does have a Java API that makes it pretty straightforward to use. We'll just need to write a little Camel component using that to satisfy our needs. 

In Camel, there are a few different ways you can extend it's functionality. We'll demonstrate one of the easiest -- just using a plain old Java object (POJOs). I'm calling the Java bean EC2Controller. We will define 3 methods -- one method for starting the EC2 instance (startInstance), one for stopping the EC2 instance (stopInstance), and one general purpose method for just setting up the necessary communication configuration (setup). 

Let's start with the setup method:

private void setup() throws Exception {
 
   try {
     credentials = 
           new PropertiesCredentials(EC2Controller.class.getResourceAsStream(
           "../../../AwsCredentials.properties"));
   } catch (IOException e1) {
      System.out.println("Credentials were not properly entered into" +
            " AwsCredentials.properties.");
      System.out.println(e1.getMessage());
      throw new Exception (e1.getMessage());
   }
}

Not really much happening here - - just trying to load in the AWS credentials which must be present in the file referenced above (NOTE: you will need to update this file with your own AWS credential settings).

The startInstance method isn't terrible complicated either. Time doesn't permit me to go through what each of the Amazon API calls is doing, but I think you'll see it's pretty easy to follow-along (I've added some annotations to the code).

public void startInstance(@Body String body, Exchange exchange) throws Exception {
 
   Log.debug("Entering EC2Controller.startInstance(). " + 

             "Processing Ec2 Instance: " + body);
 

   // covered this above

   setup();
 
   // Create the AmazonEC2Client object so we can call various APIs. 

   // The credentials class variable gets populated in the setup method.
   AmazonEC2 ec2 = new AmazonEC2Client(credentials);

   // We'll use this EC2 object to determine whether the EC2 instance is already 

   // running
   DescribeInstanceStatusRequest statusRequest 

                 = new DescribeInstanceStatusRequest();
 
   ArrayList instances = new ArrayList();
   // The message body, passed by camel, will contain the EC2 instance we 

   // are working with
   instances.add(body);
 
   statusRequest.setInstanceIds(instances);
 
   Log.debug("Checking server status");
 

   // This will actually run the request now wrapping Amazon's web services
   DescribeInstanceStatusResult result = 

                (ec2.describeInstanceStatus(statusRequest));
 
   // If the result contains a status, that means it's already running. 

   // Otherwise, start it
   if (result.getInstanceStatuses().size() > 0) {
      exchange.getOut().setBody("Camel Start Instance Status: " + 

                             "Instance already running...not started");
   } else { 
      Log.debug("Starting ec2 instance: " + body);
  
      // This EC2 object is used to start an EC2 instance.
      StartInstancesRequest startRequest = new StartInstancesRequest(instances);
  
      // Sends the web services command to Amazon AWS
      ec2.startInstances(startRequest);

      // this new message in the body will be sent via email
      exchange.getOut().setBody("Camel Start Instance Status: Ec2 Instance " + 

                                  body + " started");
    }
}

I think for the most part, the code is pretty self-explanatory, if you are already familiar with Amazon's EC2 concepts. As you will see in a moment, we pass the Amazon EC2 instance Id in the Camel message body. The method parameter annotation, @Body, is a Camel annotation, and based upon that, Camel will automatically try and pass the appropriate object based upon the annotation used (Exchange is also passed automatically by Camel -- since that's a native Camel object, it doesn't require the annotation).

With the EC2 instance id at hand, we then populate that into an ArrayList. This is required because the Amazon EC2 API calls often require a list since the calls can support performing activities on multiple instances at once. We use the Amazon API's to determine whether the instance in question is already running, and if not, we use the StartInstanceRequest to fire it up.

The stopInstance method is actually very similar, but we instead use the StopInstanceRequest Amazon API call to stop the running instance. Pretty easy, huh?

Now that we've got our custom Java bean written to interface with Amazon, let's wire it all up in Camel.

Camel Setup/Configuration

Before we delve into the route configuration, which is where all the fun resides, let's first look at some of the initial setup within the camelContext element (we're working with the camel-context.xml file here, located under src/main/resource/META-INF/spring subdirectory.). This is where you can specify property files that need to be referenced, exception handling logic, and, as a convenience, your endpoint URIs. Here's what we have:

<camel:camelContext xmlns="http://camel.apache.org/schema/spring">

   <camel:propertyPlaceholder id="props" location="zazarie.camel.properties" />
  
   <!-- define quartz endpoints -->
   <camel:endpoint id="Quartz.Start" uri="quartz://startEc2?cron=0+0+08+*+*+?"/>
   <camel:endpoint id="Quartz.Stop" uri="quartz://stopEc2?cron=0+0+20+*+*+?"/>

   <camel:endpoint id="RunOnce" uri="timer://myTimer?fixedRate=true&amp;period=10&amp;repeatCount=1"/> <!--  for testing -->

   <camel:onException>

 <camel:exception>java.lang.Exception</camel:exception>

 <camel:handled>

  <camel:constant>true</camel:constant>

 </camel:handled>

 <camel:setBody>

  <camel:simple>${properties:SMTP.FAILED}</camel:simple>

 </camel:setBody>

        <camel:to uri="seda:SendStatusEmail"/>

   </camel:onException>

The first thing we do is define a location for a property file. As we'll see below, this is simply used to avoid some hard-coding of values in the Camel configuration (some is unfortunately required).  This is done using the camel:PropertyPlaceholder element, which specifies a location of a properties file in the classpath. The next thing we do is define some endpoints. The first two represent Quartz component endpoints that define schedules for starting and stopping the instances. The Camel Quartz configuration has the details on how to interpret the configuration strings, but suffice it to say that the Quartz.Start will kick off a message at 8am, and Quartz.Stop will do the same at 8pm.  While we could include these directly within the @uri attribute of the camel:from element, centralizing the endpoints makes maintenance a bit more straightforward with it done in this fashion (although I'm a bit loose when following that pattern myself, as you likely have noticed).

Camel actually comes with two scheduling type services/components, and we've actually seen both of them here. The Timer component, which we use for defining the endpoint called RunOnce, is pretty bare-bones in functionality (this endpoint is used for testing only -- handy for kicking off a message right after Camel starts without having to twiddle with the cron settings used in the Quartz endpoints). If you need more advanced scheduling capabilities, the Quartz component is the way to go (gives you functionality like Unix/Linux cron).

As for the exception handling, all we're basically doing here is trapping for any exceptions that occur when the process is run, with an email ultimately being sent (very basic error handling, to be sure) in the event of an exception. We'll talk a little about the route we use for sending the email (the  seda:SendStatusEmail uri).

Now that we have the basic housekeeping stuff in place, let's look at the route configuration.

Camel Route Configuration for Starting an EC2 Instance

We'll actually define two main routes -- one for starting up the EC2 instance, and the other for shutting it down.  They are both very simple -- a Quartz component will fire off a message at the preordained time, which will then kickoff the Java bean we created for interfacing with Amazon. Finally, we'll send an email out with the status update.

Let's look at the startup route first:

<camel:route>

   <camel:from uri="ref:Quartz.Start" />

   <camel:setBody>

      <camel:simple>${properties:EC2.INSTANCE1}</camel:simple>

   </camel:setBody>

   <camel:bean ref="Controller" method="startInstance"/>

   <camel:to uri="seda:SendStatusEmail"/>

   <camel:to uri="stream:out" />

</camel:route>

The camel:from element, which defines how the message will be trigged in the route, references the Quartz.Start endpoint we defined a bit above.  When that endpoint fires the message, we then set the Camel message body to equal the value associated with the property EC2.INSTANCE (as you recall, the property file was loaded using the propertyPlaceholder statement).  In other words, we're setting the Camel message body to be the Amazon EC2 instance Id we want to start.  Then, the bean identified as Controller is called. This invokes the EC2Controller Java bean we created earlier that interfaces with Amazon's AWS. This bean is configured in the beans element of this same XML file as:

<bean id="Controller" name="Controller" class="zazarie.camel.beans.EC2Controller" />

Lastly, we make that curious called to seda:SendStatusEmail. This is actually another defined endpoint, and it's used for sending out the emails. Why do this? For one, it allows us to easily re-use that endpoint, since we that functionality in a few places. Why the seda protocol? That is an asynchronous protocol used by Camel, which is perfect for this, because it's really just a fire-and-forget piece of functionality (i,.e,. sending the email).

Here's how we define that seda endpoint:

<camel:route>

   <camel:from uri="seda:SendStatusEmail"/>

   <camel:setHeader headerName="to">
       <camel:simple>${properties:SMTP.SENDTO}</camel:simple>
   </camel:setHeader> 

   <camel:setHeader headerName="subject">
       <camel:simple>${properties:SMTP.SUBJECT}</camel:simple>
   </camel:setHeader> 

   <camel:to uri="smtps://smtp.gmail.com?&username=someuser@gmail.com&password=password-here&contentType=text/plain"/>

</camel:route>

So, when you saw the statement earlier:

<camel:to uri="seda:SendStatusEmail"/>

This was invoking this route. This route first sets the to and subject headers using properties from the property file, and then calls the smtps (Mail component) protocol to send the outbound message. For some reason, Camel doesn't seem to allow certain of the properties, such as username and password, to be set using the camel:setHeader mechanism (which is a minor bummer).

We've now covered how the route works that starts the Amazon EC2 instance, let's look at the one use for stopping it -- it's pretty much identical.

Camel Route Configuration for Starting an EC2 Instance


The route used to stop the EC2 instance is actually identical, with one minor change -- we invoke the stopInstance method of the Controller bean:

<camel:bean ref="Controller" method="stopInstance"/>

Setup and Run


After you've extracted the project into some directory location, you will need to do a few things to get it ready for your environment.

1. Edit the zazarie.camel.properties file. You will want to enter in your own email address where you want the status messages sent, and change the EC2 instance Id to the one you want to manage.
2. In the camel.context file, you will want to edit the smtps URI so that it contains the SMTP server you are using to send the outbound status message. 
3. Edit the AwsCredentials.properties file to use your own Amazon EC2 security credentials.
4. To setup it up for testing, use the RunOnce endpoint in lieu of the Quartz.Start or Quartz.Stop @ref's in the routes used for starting and stopping EC2. 

Finally, we are ready to run it using mvn camel:run. 

Additional Functionality


In the camel-context.xml file, I also included an additional route that demonstrates how you can run some remote commands on an an EC2 instance. This is the commented out route that uses the exec Camel component. One of the common requirements before starting or stoping an EC2 instance is to run some system commands. This might be to cleanly shutdown a database, or issue some command to start a service upon startup. I had attempted to use Camel's built-in SSH component, but never could get it working correctly. However, it's pretty easy to just invoke a .bat (Windows) or .sh (Linux) script that uses something like command line SSH or Putty to invoke a remote command. The Mina component could also be used for this purpose (it's very slick), but I didn't have time to fully explore that.

Other areas that this example could be enhanced would be to demonstrate how Camel could be used to periodically poll an EC2 instance for load/usage. Then, if the load exceeds a certain high-water mark, Camel could kick off instantiating a new EC2 instance.

Once thing that I haven't had a chance to tackle yet is how to run this routine under a Tomcat container. Having to run it through through the command line is good for testing, but not for a production environment, you likely want to configure it as a war file. That's not hard to do -- maybe I'll add a note on that in the near future if folks are interested.

Let me know your feedback!

jeff davis, Monument Co

References -- Project Source Code.

Back on Blogger

Sorry for the extended absence. I had previously been using a wordpress blog hosted through one of my Amazon EC2 instances, but decided that was getting a little costly. So, I'm back on blogger. Looks for some new blog entries soon. I'll be covering such topics as Apache Camel, Google GWT and GAE (Google App Engine), and a variety of other open source topics.

New interesting book on WS-BPEL

I recently read a new Packt book titled "SOA and WS-BPEL: Composing Service-Oriented Solutions with PHP and ActiveBPEL" by Yuli Vasiliev. I've used ActiveBPEL in the past, and there certainly is a learning curve involved in learning BPEL. However, Yuli does an excellent job of explaining both the fundamentals of BPEL, and more advanced BPEL topics such as parallel vs. sequential processing. The use of a ongoing case study walks the user logically through the learning process.

The first section of the book, which I found particularly useful given my lack of PHP experience, primarily focuses on how to consume and build SOAP web services with PHP. Using PHP with SOAP doesn't seem as straightforward as it should be, but Yuli does a good job at describing the process and best practices. The remaining sections cover BPEL, and ActiveBPEL's implementation of them.

The book is fairly brief in size (280 pages), but covers a lot of material in an effective fashion. I would recommend it to anyone interested in using PHP with SOAP and/or ActiveBPEL (the BPEL portions are beneficial even if you are not a PHP developer).

jeff

SOA Essentials, Part 1

A Service Oriented Architecture, or SOA, represents an approach towards software development that emphasizes the creation of reusable software services that are based upon discrete units of business functionality. An example of such functionality might be a software component that can be used for creating a new contact within a customer relationship management (CRM) system. It qualifies as discrete if it can be invoked in a stand-alone fashion by a variety of external applications without undue dependencies on the CRM system. To be considered a truly reusable “service”, it must exhibit certain characteristics, such as a well-defined interface or contract along with a standards-based means of invocation. Once a collection of such services becomes available they, in turn, can then wired together to create new business processes and applications.

For instance, let’s assume Acme Company, as a disciple of SOA, and has created a multitude of such services, including CRM functionality such as creating contacts; inquires/cases; email correspondence; action items and the like. Now, in an effort to improve and streamline lead generation they want to develop a new web-based form so that anyone interested in their products or services can easily place an inquiry. The previous form would only send the inquiry details to an email distribution list that was poorly monitored and tracked. Further, they prospects information would have to be re-keyed into the CRM system manually, which was very tedious (what I call “swivel-chair integration”). With the new inquiry form, they want to use their SOA services to automatically perform the following business process:

• Create a new Contact in their CRM system.

• Create a new CRM “Case” using the data provided within the form and assign that Case to an appropriate sales team member (perhaps based on geographic business rules).

• Create a new “Activity” for the sales team member, which is analogous to a to-do item to follow-up on the inquiry.

• Automatically generate a response email to the prospect with their Case # and sales team member.

• Create an entry in their internal Training Management System (TMS) so that the prospect is automatically notified of any upcoming webinars or seminars.

• Create an event into their Business Activity Monitoring (BAM) system, which is carefully monitored by company executives as part of their Key Performance Indicators (KPI).

In this fictitious example, there are three different enterprise systems that are involved: CRM, TMS and BAM. Fortunately, Acme has already developed service for each of the functions identified, and so they can easily create a new application that incorporates the desired business process. Figure 1 depicts the how the reusable business services, which draw on the enterprise applications, can be used to construct a new, integrated business process. The result of Acne’s new process is improved lead generation, productivity, and response time.

Figure 1

Since Acme has developed a repository of reusable services, they can now respond quickly to market forces by creating new application and processes in a fraction of the time otherwise required in absence of an SOA.

SOA, being the latest in a string of IT “silver bullets, has been subject to a lot of misuse and bastardization. In part, this is due to the eagerness of vendors to associate their products with this new architecture style. Many of the common definitions of SOA do share important characteristic:

“Contemporary SOA represents an open, agile extensible, federated, composable architecture comprised of autonomous, QoS-capable, vendor diverse, interoperable, discoverable, and potentially reusable services, implemented as Web services” (Erl, Service Oriented Architecture, 2005, pg 54).

“Service-Oriented Architecture is an IT strategy that organizes the discrete functions contained in enterprise applications into interoperable, standards-based services that can be combined and reused quickly to meet business needs.” (Domain Model for SOA: Realizing the Business Benefit of Service-Oriented Architecture. BEA White Paper. 2005)

As you can see, the common theme is obviously the notion of discrete, reusable business services that can be used to construct new and novel business processes or applications. The history of IT is replete, however, with component-based frameworks that attempted similar objectives. They include Common Object Request Broker Architecture, or CORBA, along with Microsoft’s Distribute Component Object Model (DCOM) and Java’s Enterprise Java Beans (EJB). What distinguishes these approaches from the newer SOA? There are several:

• CORBA, EJB and DCOM are all based on remote-procedure call (RPC) technologies. This created tight dependencies between the client application and the underlying RPC components. The RPC components were, by nature, very narrow in scope and usually tailored specifically for use by a single application, thus limiting reusability. Also, RPC solutions tended to generate significant network traffic, and suffered from complications in transaction management (i.e., two-phase commits etc).

• In the case of EJB and DCOM, they are both tied to specific platforms and were thus not interoperable. Unless a homogenous environment existed (rare in today’s enterprises, which are often grown through acquisition), the benefits from them could not be easily achieved. SOA-based web services were designed with interoperability in mind.

• Complexity.CORBA, EJB,and to a lesser degree DCOM, were complicated technologies that often required commercial products to implement. SOA can be introduced using a multitude of off-the-shelf, open-source technologies.

• SOA relies upon XML as the underlying data representation, unlike the others which used proprietary, binary-based objects. XML’s popularity is undeniable, in part because it is easy to understand and generate.

Another distinction between an SOA and earlier component-based technologies is that an SOA is more than technology per se, but also constitutes a set of principles or guidelines. This includes notions such as governance, service-level agreements; meta-data definitions, and registries. These topics will be addressed in greater detail in the sections that follow.

So what does an SOA resemble conceptually? Figure 1.1 depicts the interplay between the backend systems, exposed services, and orchestrated business processes.

Figure 1.1

As you can see, service components represent the layer atop the enterprise business systems/applications. These components allow the layers above to interact with these systems. The composite services layer represents more course-grained services that are comprised from two or more individual components. For example, a “createPO” composite service may include integrating finer-grained services such as “createCustomer”, “createPOHeader”, and “createPOLineItems”. The composite services,, in turn, can then be called by higher-level orchestrations, such as one for processing orders placed through a web site.

1.1 BENEFITS OF SOA

Although we’ve already touched on a few of the inherit benefits of an SOA, we can summarize the major benefits as:

• Business Agility. Business functions, exposed as services, can be stitched together to create new applications using orchestration and workflow products. This allows for rapid creation of new business processes and the ability to quickly alter existing ones. Today’s fast-moving business environment demands such agility in order for companies to remain competitive.

• Code Reuse. The ability to effectively reuse existing functionality constitutes a cost savings. Perhaps equally significant is the impact on quality. Reusing proven and well-tested assets inevitably improves quality.

• Platform Agnostic. Services exposed in a standards-based fashion can bridge between differing platforms and languages. For example, a “getCustomer” service written in Java can now be accessed by a .NET client. The popularity of new interpreted scripting languages such as Ruby and Groovy further intensifies the relevance of having agnostic-based services. This also reduces vendor lock-in and switching costs, and encourages true “best-of-breed” adoption.

• Legacy Enablement. The investment in legacy applications can be preserved by exposing their business rules and functions through service wrappers. This not only saves money, but opens these systems up for more creative purposes. Thus, integration costs are reduced.

• Standardized XML Vocabularies. The XML-centric nature of SOA encourages the development of a common set of well-defined business object documents. By creating a common vocabulary within your enterprise, integration becomes vastly simplified.

Fundamentally, an SOA is about exposing services that facilitate the sharing of information across application and business boundaries. In our example earlier describing Acme Corporation’s new web inquiry form (figure 1), we demonstrated a business process that spanned across several enterprise systems. This is a common requirement and enables the rapid creation or modification of business processes. This translates into faster turnaround time for the introduction of new products; cost savings derived from greater productivity and efficiency; and faster response to changes in market conditions.

The benefits of moving to an SOA are real, but how can they be realized? The next article will cover what elements are necessary to succeed with an SOA.

Add to Technorati Favorites

Using Embedded ServiceMix Client

One of the options for integrating ServiceMix with BPM solutions such as jBPM is to simply instantiate an 'on-the-fly' Servicemix client, then pass an XML message directly to a ServiceMix component that has been configured to receive it. An example of where this approach maybe warranted is if you had several ESB-type tasks that you wanted to perform in sequence, such as logging jBPM results to a database, then sending them via email to a predefined audience. Obviously, these are tasks that ServiceMix handles with aplomb, since it has prebuilt components for such things.

In the example presented in this article, I'm going to provide an example of calling ServiceMix File and XSTL components. The SM file component example is InOnly (i.e., fire-and-forget), whereas the XSLT example is asynchronous (block-and-wait for response).

We'll first look at the setup.xml file, which sets up the component definitions and ServiceMix JBI container.

setup.xml

<beans xmlns:sm="http://servicemix.apache.org/config/1.0"
 xmlns:foo="http://servicemix.org/cheese"
 xmlns:eip="http://servicemix.apache.org/eip/1.0">

 <!-- the JBI container -->
 <sm:container id="jbi" embedded="true">
  <sm:activationSpecs>

   <!-- returns dummy xml for testing -->
   <sm:activationSpec componentName="test" service="foo:test" >
    <sm:component>
     <bean class="org.apache.servicemix.components.xslt.XsltComponent">
      <property name="xsltResource" value="classpath:resources/test.xslt"/>
     </bean>
    </sm:component>
   </sm:activationSpec>
   
   <sm:activationSpec componentName="receiver"
    service="foo:receiver">
    <sm:component>
     <bean
      class="org.apache.servicemix.tck.ReceiverComponent" />

    </sm:component>
   </sm:activationSpec>

   <sm:activationSpec componentName="fileSender"
    service="foo:fileSender">
    <sm:component>
     <bean
      class="org.apache.servicemix.components.file.FileWriter">

      <property name="directory" value="/tmp/test" />

      <property name="marshaler">
       <bean
        class="org.apache.servicemix.components.util.DefaultFileMarshaler">
        <property name="fileName">
         <bean
          class="org.apache.servicemix.expression.JaxenStringXPathExpression">
          <constructor-arg
           value="'test.xml'" />
         </bean>
        </property>

       </bean>
      </property>
     </bean>
    </sm:component>
   </sm:activationSpec>

  </sm:activationSpecs>
 </sm:container>


 <bean id="client"
  class="org.apache.servicemix.client.DefaultServiceMixClient">
  <constructor-arg ref="jbi" />
 </bean>
</beans>

In the above, you can see that I've setup two components -- the XSLT service named "test" and the file writer component named "fileSender". In the case of the XSLT component, the test.xslt file associated with it simply returns the following xml:

<hello>
<text>Hi!</text>
</hello>

In the case of the fileSender, it is configured to write a file to /tmp/test called "test.xml" with the contents of what was passed when it was invoked.

Next, we'll look at a jUnit test that illustrates how a ServiceMix instance can be instantiated and run through a Java client.

ServiceMixClientTest.java

package company.cop.check.test;
import javax.xml.namespace.QName;
import junit.framework.TestCase;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.servicemix.client.ServiceMixClient;
import org.apache.servicemix.jbi.container.SpringJBIContainer;
import org.apache.servicemix.jbi.jaxp.SourceTransformer;
import org.apache.servicemix.jbi.resolver.EndpointResolver;
import org.apache.servicemix.tck.Receiver;
import org.apache.xbean.spring.context.ClassPathXmlApplicationContext;
import org.springframework.context.support.AbstractXmlApplicationContext;
import org.w3c.dom.Document;

/**
* @version $Revision: 1.2 $
*/
public class ServiceMixClientTest extends TestCase {

private static final transient Log LOG = LogFactory.getLog(ServiceMixClientTest.class);

protected AbstractXmlApplicationContext context;
protected ServiceMixClient client;
protected Receiver receiver;

protected SourceTransformer transformer = new SourceTransformer();

public void testCallingComponent() throws Exception {
  QName service = new QName("http://servicemix.org/cheese", "fileSender");
  EndpointResolver resolver = client.createResolverForService(service);
  client.send(resolver, null, null, "world");

}

public void testXslt() throws Exception {
    QName service = new QName("http://servicemix.org/cheese", "test");
    EndpointResolver resolver = client.createResolverForService(service);
    Document response = (Document) client.request(resolver, null, null, "world");
    assertEquals(response.getChildNodes().item(0).getTextContent(), "Hi!");
    System.out.println("Response: " + response.getChildNodes().item(0).getTextContent());
}

protected void setUp() throws Exception {
  context = createBeanFactory();
  client = getClient();
  SpringJBIContainer jbi = (SpringJBIContainer) getBean("jbi");
}

protected ServiceMixClient getClient() throws Exception {
  return (ServiceMixClient) getBean("client");
}

protected void tearDown() throws Exception {
  super.tearDown();

  if (context != null) {
      context.close();
  }
}

protected Object getBean(String name) {
  Object answer = context.getBean(name);
  assertNotNull("Could not find object in Spring for key: " + name, answer);
  return answer;
}

protected AbstractXmlApplicationContext createBeanFactory() {
  return new ClassPathXmlApplicationContext("resources/setup.xml");
}
}

The setup() method, called automatically by jUnit prior to each test, establishes the JBI ServiceMix container/server. Then, each of the test methods are called. The testCallingComponent() method sends a message to the "fileSender" ServiceMix component/service. It identifies the target service by QName, which is the namespace and name provided the service in the setup.xml file. The client.send() method sends a message using the provided parameters, notice the last of which is the XML packet. When run, this should result in a file called "test.xml" being written. The contents should be what was passed in the client.send operation, namely "world".

The testXslt() method is used to invoke the test XSLT service defined in setup.xml. Unlike the fileWriter, this is a synchronous call, as we anticipate the transformed XML being sent back to us. To do so, the client.request() method is used, and it will return back the transformed XML in the form of a DOM Document. That Document object is then traversed to printout the text contents of the "text" node. This should be "Hi!", as defined in the XSLT document.

Final Thoughts

As you can see, it's pretty straightforward to invoke ServiceMix within a standalone Java client, without requiring ServiceMix to be running as a stand-alone instance. Doing this, however, does impose some performance penalties, as instantiating the JBI container does consume some overhead.

We'll explore calling remote services from a running ServiceMix instance in another article. Another intriguing option would be to use a stateful EJB to maintain a ServiceMix context, but there could be some tricky concurrency issues with that approach.

Add to Technorati Favorites

SOA Design Principles

I've just completed an excellent book titled "SOA: Principles of Service Design" by Thomas Erl. Unfortunately, SOA has become such a hackneyed expression that its virtually lost all meaning. However, Erl does a nice job a describing what it really means to create a service-based architecture. These are some of the design principles he identified:

Service Loose Coupling

Coupling refers to a connection or relationship between two things. A measure of coupling is thus comparable to a level of dependency. In SOA, the objective is to reduce ("loosening") dependencies between the service contract, it's implementation, and its service consumers. This makes a service more reusable and flexible. A well-defined meta-data contract, such as provided through a web service WSDL, defines the interfaces with service.

Service Abstraction

Sometimes known as "data hiding", this principle emphasizes the need to hide as much as possible the underlying details of a service. This works in conjunction with Service Loose Coupling to help make services more reusable. This also permits changes to be made to the service without unduly impacting clients. In essence, the service should be a "black-box" that performs its operations without any understanding of the mechanics behind it.

Service Reusability

A service should not be specific to a single functional context. Instead, it should be "agnostic", and support numerous usage scenarios, without having functional dependencies. One challenge with service reusability is that it often promotes more granular-type services, because a service with narrower focus is often more easily reused. This can have performance implications and must be weighed carefully.

Service Autonomy

A service should exercise a high level of control over its underlying runtime environment. In other words, it should managed within its own environment, and be able to support a high degree of reliability and performance. The client should no visibility or concern over the environment in which the service is run.

Service Statelessness

Services should minimize resource consumption by deferring the management of state information, when necessary. A service that maintains state also likely reduces reusability, as it then is likely imposing a certain process or functional dependency on the service.

Service Discoverability

Services should be supplemented with communication meta-data by which they can be effectively discovered and interpreted. In other words, services should be defined within a common registry that enables users to readily understand its purpose and requirements. Obviously, this need becomes more pronounced as an organization grows and spawns multiple development groups or organizations.

Service Composability

Services should be effective composition participants, regardless of the size and complexity of the composition. For instance, this assumes that it will include comprehensive exception handling features and support multiple run-time scenarios. It also requires that the service be capable of handling multiple, concurrent transactions and be highly performing. Using a consistent invocation pattern is also instrumental in this.

Add to Technorati Favorites

What's wrong with BPEL?

Recently, I've spent a lot of time examining various BPEL-based products. They include Intalio, Cape Clear and Active Endpoints. After evaluating each of them, I've come to the conclusion that WS-BPEL is often not the right fit. If you're a pure-play Java shop, a solution such as JBoss jBPM maybe a much better fit (a future blog will go into some detail on how to use jBPM). The issues I see with BPEL are:

* BPEL, in an effort to support all vendors/platforms, only works with SOAP-based services. This means that every time an invocation is required, say to call an FTP or email adapter, the adapter/service must be exposed as a web service. That imposes a lot of extra overhead on the developer, and for internal services, seems onerous. Most of us don't live in a world where everything is automatically exposed as web services (I'm a big proponent of SOAP/Web services, but also realistic).

* Passing of data between activities/steps within a BPEL orchestration must be done using XML and WSDL constructs. Complex data transformations and assignments must frequently be performed using XSL, and developers who aren't well versed in the nuances of XSD schemas (in particular, namespaces) can become frustrated. Java and .NET developers must often wonder why they can't just pass a data class objects between the nodes.

* BPEL, by it's definition, is a closed standard. If a product is 100% BPEL compliant, extending it by supporting direct API java calls, for instance, automatically makes your orchestrations non-compliant, thus losing the benefits of the vendor neutrality and compatibility BPEL promises. Then what's the point of using BPEL?

* BPEL-based products are often app-server centric, meaning they have to be deployed an hosted. This often poses challenges from a testing standpoint, as developers must deploy their orchestration prior to being able to test it. Again, this impacts developer productivity in a negative way.

* BPEL introduces often very confusing nomenclature that must be learned. Concepts like "Partner Links", "Partner Link Types", "Partner Roles", "Port Types" and "Correlation Sets" are often not intuitive, and require a deep understanding of all things WSDL.

* Some products, such as Intalio, recognize the difficulties in modeling things natively in BPEL, so they use alternative, presumably more intuitive, solutions such as BPML. While that address some of the developer productivity concerns mentioned, they then rely on translation/code generation to output BPEL code. That "black-box" translation steps seems like a source of ongoing problems (the developer must, for example, try and determine why a diagram in BPML got translated into such-and-such BPEL code). Seems like a headache waiting to happen.



Add to Technorati Favorites