Saturday, January 1, 2011

POJO Remoting over Camel-XMPP within Eclipse App

I spent a lot of hours trying to make Remote Observer Pattern with Camel over XMPP work inside my Eclipse RCP Application, so here's my guide to make it run successfully.

Getting Camel


Getting Apache Camel itself to run is the easiest part, because Camel is already packaged as OSGi bundles, also available from the Maven Central repository.

Here are the artifacts that I needed:

  1. org.apache.camel:camel-core:2.5.0 (dependency of the ones below)
  2. org.apache.camel:camel-blueprint:2.5.0
  3. org.apache.camel:camel-eclipse:2.5.0
  4. org.apache.camel:camel-xmpp:2.5.0
  5. org.apache.camel:camel-xstream:2.5.0
These will pull additional dependencies, here are the JARs:

camel-blueprint-2.5.0.jar
camel-core-2.5.0.jar
camel-eclipse-2.5.0.jar
camel-xmpp-2.5.0.jar
camel-xstream-2.5.0.jar
commons-management-1.0.jar
javax.xml_1.3.4.v201005080400.jar
javax.xml.stream_1.0.1.v201004272200.jar
jettison-1.2.jar
org.apache.aries.blueprint-0.2-incubating.jar
org.apache.servicemix.bundles.xmlpull-1.1.3.1_1.jar
org.apache.servicemix.bundles.xpp3-1.1.4c_4.jar
org.apache.servicemix.bundles.xstream-1.3_4.jar

You'll notice that it includes the Apache Aries Blueprint container.
Camel can also use Blueprint service to run in OSGi. (It seems Camel can work in Eclipse without Blueprint, but I haven't tried it. Besides I think Blueprint is a nice touch, hehe..)
It's nice to know that Apache Aries Blueprint works well within Eclipse/Equinox.
(I was tempted to use Eclipse Gemini Blueprint but it seems "not ready yet", and I suspect it brings on Spring Framework dependencies, which I'm not interested at the moment.)

I need to replace some of the dependencies with OSGi bundles from:


Note that I've left out Smack JARs there, for reasons later below.

Hint: To know which OSGi-ready dependencies are available, check Camel for Karaf features list here :
http://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/2.5.0/

Starting Apache Camel Bundles


Apache Camel and the Apache Aries Blueprint needs to be explicitly started, so after copying these files to the target platform, I need to go the Eclipse Launch configuration and mark these bundles as "start". I'm not sure which exact bundles need to be started, but I just start all camel-* bundles and all its dependencies and Apache Aries Blueprint.

If there is a trouble, it might be useful to tweak the start order, but I hope it's not necessary.

Using Camel with Blueprint

There are several ways to use Camel inside Eclipse, the traditional way without Blueprint is like this:

PackageScanClassResolver eclipseResolver = new EclipsePackageScanClassResolver();
CamelContext context = new DefaultCamelContext();
context.setPackageScanClassResolver(eclipseResolver);

To use Camel from a Blueprint beans XML file, create OSGI-INF/blueprint/config.xml file (directly below your plugin project folder, not inside the src folder) like this:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">

<camelContext xmlns="http://camel.apache.org/schema/blueprint" id="camelContext" autoStartup="true">
<route>
<from uri="direct:invoice" />
<to uri="seda:invoice.async" />
</route>
</camelContext>

<bean id="invoicing" class="id.co.bippo.usexmpp.Invoicing"
init-method="init" destroy-method="destroy">
<property name="camelContext" ref="camelContext" />
</bean>
</blueprint>

Note that autoStartup is true by default, you can change it to false.

The CamelContext instance will be injected into the class by Blueprint, I created a class like this:

import org.apache.camel.CamelContext;
import org.apache.camel.builder.ProxyBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.eclipse.EclipsePackageScanClassResolver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Invoicing {

private CamelContext camelContext;
private Logger logger = LoggerFactory.getLogger(getClass());
protected InvoiceListener loggerInvoiceListener = new LoggerInvoiceListener();

public void init() throws Exception {
camelContext.setPackageScanClassResolver(new EclipsePackageScanClassResolver());
camelContext.addRoutes(new RouteBuilder() {

@Override
public void configure() throws Exception {
// .. configure additional routes ...
}
});
camelContext.start();

InvoiceListener invoiceListener = new ProxyBuilder(camelContext)
.endpoint("direct:invoice").build(InvoiceListener.class);
invoiceListener.invoiceCreated(123, "Bippo Indonesia");
}

public void destroy() throws Exception {
camelContext.stop();
}

public void setCamelContext(CamelContext camelContext) {
this.camelContext = camelContext;
}

}

The simple application seems to run fine without setting EclipsePackageScanClassResolver, but I guess it's safer to use it.

Another way create a CamelContext using Blueprint XML is by instantiating it:

<bean id="camelContext" class="org.apache.camel.blueprint.BlueprintCamelContext">
<property name="bundleContext" ref="blueprintBundleContext" />
<property name="blueprintContainer" ref="blueprintContainer" />
<property name="packageScanClassResolver">
<bean class="org.apache.camel.component.eclipse.EclipsePackageScanClassResolver" />
</property>
</bean>

Using the factory bean directly:

<bean id="camelFactory" class="org.apache.camel.blueprint.CamelContextFactoryBean">
<property name="bundleContext" ref="blueprintBundleContext" />
<property name="blueprintContainer" ref="blueprintContainer" />
<property name="autoStartup" value="false" />
</bean>
<bean id="camelContext" factory-ref="camelFactory" factory-method="getContext" />

Xstream Dependencies

Xstream is needed to marshal/serialize objects to/from XML representation and/or JSON.

Xstream requires several dependencies:

  • javax.xml
  • javax.stream (aka StaX API)
  • xmlpull
  • xpp3 (aka xml.pull.mxp1)
  • Jettison

These are available from Apache ServiceMix repositories (above).

Making Xstream work with OSGi

Camel has a shortcut way of marshalling/unmarshalling objects with Xstream :

from("direct:invoice-json").marshal().json().("log:invoice");
from("direct:invoice-xstream").marshal().xstream().("log:invoice");
from("seda:json").unmarshal().json().("log:unmarshal");
from("seda:xml").unmarshal().xstream().("log:unmarshal");

However with OSGi we need to set the classloader which is used to resolve marshalled classes. The configuration becomes:

import org.apache.camel.dataformat.xstream.XStreamDataFormat;
import com.thoughtworks.xstream.XStream;
...
camelContext.addRoutes(new RouteBuilder() {
@Override
public void configure() throws Exception {
XStream xstream = new XStream();
xstream.setClassLoader(getClass().getClassLoader());
XStreamDataFormat xstreamDataFormat = new XStreamDataFormat(xstream);

from("direct:invoice-xstream").marshal(xstreamDataFormat).("log:invoice");
from("seda:xml").unmarshal(xstreamDataFormat).("log:unmarshal");
}
});

To use the JSON format with Xstream, we need to configure the Xstream driver:

import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver;
...
xstreamDataFormat.setXstreamDriver(new JettisonMappedXmlDriver());

OSGi-fying Smack


In the plain Java world, we'll be done. But not so with Eclipse and OSGi.

I had the most trouble with the Smack library. The repackaged Smack library from Apache ServiceMix also didn't just work due to missing META-INF/smack.providers file and XMPPConnection not registering ServiceDiscoveryManager, which seem to happen because of static { ... } statements.

One problem is this code from Smack:

    public static Collection<String> getServiceNames(XMPPConnection connection) throws XMPPException {
        final List<String> answer = new ArrayList<String>();
        ServiceDiscoveryManager discoManager = ServiceDiscoveryManager.getInstanceFor(connection);
        DiscoverItems items = discoManager.discoverItems(connection.getServiceName());
        for (Iterator<DiscoverItems.Item> it = items.getItems(); it.hasNext();) {

discoManager returns null, because the map of instances is empty.

The static code block here is not being called within OSGi runtime:

    // Create a new ServiceDiscoveryManager on every established connection
    static {
        XMPPConnection.addConnectionCreationListener(new ConnectionCreationListener() {
            public void connectionCreated(XMPPConnection connection) {
                new ServiceDiscoveryManager(connection);
            }
        });
    }

So I had to tweak Smack and published the changes to my Smack repository on GitHub.

I've also published the tweaked Smack 3.1.0 library as Eclipse plugin to Bippo OSS p2 Repository.

You can also download the bundle directly from: http://oss.bippo.co.id/p2/features/id.co.bippo.oss.smack_3.1.0.201101020232.jar

Note that my Smack bundle now requires SLF4j API for logging.

Tip: A nice way to debug Smack is by adding -Dsmack.debugEnabled=true to VM launch options.

I also learned several (re-)packaging tips with Eclipse plugins:

  • make sure to add "." in Runtime classpath when adding other classpaths (Apache ServiceMix repackaged Smack library includes other dependencies in folder containing .class files)
  • Workarounds for missing files can potentially be done using OSGi Fragment bundles, for example to augment
META-INF/smack.providers if so desired.

The App

Here's how the Camel XMPP test Eclipse app looks like:

import org.apache.camel.CamelContext;
import org.apache.camel.builder.ProxyBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.eclipse.EclipsePackageScanClassResolver;
import org.apache.camel.dataformat.xstream.XStreamDataFormat;
import org.apache.camel.model.dataformat.JsonDataFormat;
import org.apache.camel.model.dataformat.JsonLibrary;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver;

public class Invoicing {

private CamelContext camelContext;
private Logger logger = LoggerFactory.getLogger(getClass());
protected InvoiceListener loggerInvoiceListener = new LoggerInvoiceListener();

public void init() throws Exception {
logger.info("Invoicing started");
camelContext.setPackageScanClassResolver(new EclipsePackageScanClassResolver());
camelContext.addRoutes(new RouteBuilder() {

@Override
public void configure() throws Exception {
JsonDataFormat dataFormat = new JsonDataFormat(JsonLibrary.XStream);
XStream xstream = new XStream();
xstream.setClassLoader(getClass().getClassLoader());
XStreamDataFormat xstreamDataFormat = new XStreamDataFormat(xstream);
xstreamDataFormat.setXstreamDriver(new JettisonMappedXmlDriver());

final String xmppUri = "xmpp://abispulsabot@geulis.local/?room=invoice&password=test";
from("direct:invoice").marshal(xstreamDataFormat).threads().multicast()
.to(xmppUri)
.to("log:xmpp-output");
from(xmppUri).threads().multicast()
.to("log:xmpp-input")
.to("seda:loggerInvoiceBean");
from("seda:loggerInvoiceBean").unmarshal(xstreamDataFormat).bean(loggerInvoiceListener);
}
});
camelContext.start();

InvoiceListener invoiceListener = new ProxyBuilder(camelContext)
.endpoint("direct:invoice").build(InvoiceListener.class);
invoiceListener.invoiceCreated(123, "Bippo Indonesia");
}

public void destroy() throws Exception {
camelContext.stop();
logger.info("Invoicing stopped");
}

public void setCamelContext(CamelContext camelContext) {
this.camelContext = camelContext;
}

}

Log output:

04:41:04.302 [Blueprint Extender: 2] WARN  o.a.a.b.c.BlueprintContainerImpl - Bundle id.co.bippo.usexmpp is waiting for namespace handlers [(&(objectClass=org.apache.aries.blueprint.NamespaceHandler)(osgi.service.blueprint.namespace=http://camel.apache.org/schema/blueprint))]
04:41:04.301 [Start Level Event Dispatcher] INFO  org.apache.camel.impl.osgi.Activator - Camel activator starting
04:41:04.318 [Start Level Event Dispatcher] INFO  org.apache.camel.impl.osgi.Activator - Camel activator started
04:41:05.683 [Blueprint Extender: 2] INFO  org.apache.camel.impl.osgi.Activator - Found 13 @Converter classes to load
04:41:05.707 [Blueprint Extender: 2] INFO  o.a.c.b.BlueprintCamelContext - Starting Apache Camel as property ShouldStartContext is true
04:41:05.708 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 2.5.0 (CamelContext: camelContext) is starting
04:41:05.708 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - JMX enabled. Using ManagedManagementStrategy.
04:41:05.711 [Blueprint Extender: 2] WARN  o.a.camel.impl.DefaultCamelContext - Could not find needed classes for JMX lifecycle strategy. Needed class is in spring-context.jar using Spring 2.5 or newer (spring-jmx.jar using Spring 2.0.x). NoClassDefFoundError: org/springframework/jmx/export/metadata/JmxAttributeSource
04:41:05.711 [Blueprint Extender: 2] WARN  o.a.camel.impl.DefaultCamelContext - Cannot use JMX. Fallback to using DefaultManagementStrategy (non JMX).
04:41:05.717 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Total 0 routes, of which 0 is started.
04:41:05.718 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 2.5.0 (CamelContext: camelContext) started in 0.010 seconds
04:41:05.718 [Blueprint Extender: 2] INFO  id.co.bippo.usexmpp.Invoicing - Invoicing started
04:41:06.613 [Smack Packet Reader (0)] ERROR org.jivesoftware.smack.PacketReader - Empty IQ packet! </iq>
04:41:06.715 [Blueprint Extender: 2] INFO  o.a.c.c.xmpp.XmppGroupChatProducer - Joined room: invoice@conference.geulis as: abispulsabot
04:41:06.729 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: Endpoint[direct://invoice]
04:41:06.779 [Blueprint Extender: 2] INFO  o.a.c.component.xmpp.XmppConsumer - Joined room: invoice@conference.geulis as: abispulsabot
04:41:06.780 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Route: route2 started and consuming from: Endpoint[xmpp://abispulsabot@geulis.local/?password=******&room=invoice]
04:41:06.805 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Route: route3 started and consuming from: Endpoint[seda://loggerInvoiceBean]
04:41:06.806 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 2.5.0 (CamelContext: camelContext) is starting
04:41:06.806 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Total 3 routes, of which 3 is started.
04:41:06.806 [Blueprint Extender: 2] INFO  o.a.camel.impl.DefaultCamelContext - Apache Camel 2.5.0 (CamelContext: camelContext) started in 0.000 seconds
04:41:06.846 [Camel Thread 1 - Threads] INFO  xmpp-output - Exchange[ExchangePattern:InOut, BodyType:byte[], Body:{"org.apache.camel.component.bean.BeanInvocation":{"org.apache.camel.component.bean.MethodBean":{"name":"invoiceCreated","type":"id.co.bippo.usexmpp.InvoiceListener","parameterTypes":{"java-class":["int","java.lang.String"]}},"object-array":{"int":123,"string":"Bippo Indonesia"}}}]
04:41:06.858 [Camel Thread 2 - Threads] INFO  xmpp-input - Exchange[ExchangePattern:InOnly, BodyType:String, Body:{"org.apache.camel.component.bean.BeanInvocation":{"org.apache.camel.component.bean.MethodBean":{"name":"invoiceCreated","type":"id.co.bippo.usexmpp.InvoiceListener","parameterTypes":{"java-class":["int","java.lang.String"]}},"object-array":{"int":123,"string":"Bippo Indonesia"}}}]
04:41:06.880 [Camel Thread 0 - seda://loggerInvoiceBean] INFO  i.c.b.usexmpp.LoggerInvoiceListener - Invoice #123 name: Bippo Indonesia created
04:41:08.380 [Camel Thread 0 - seda://loggerInvoiceBean] INFO  i.c.b.usexmpp.LoggerInvoiceListener - 123/Bippo Indonesia done!

Conclusion


Apache Camel is unobtrusive way to add flexibility of routing, messaging, and integration patterns with XMPP or other connectors supported by Camel.
(you can create your own connectors if you want...)

I highly suggest the Camel in Action Book for the best in-depth guide and examples for using Camel to develop your applications more productively.

9 comments:

  1. As an addendum to the post, a bunch of exceptions that can occur when not properly set up:

    org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: xmpp://abispulsabot:test@localhost/?room=abispulsa.refill due to: No component found with scheme: xmpp
    at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:444)
    at com.abispulsa.integration.Setup.setupBot(Setup.java:23)
    at com.abispulsa.integration.Setup.startCamel(Setup.java:58)
    at com.abispulsa.integration.Setup.start(Setup.java:78)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)

    ...

    Caused by: java.lang.NoClassDefFoundError: org/jivesoftware/smack/filter/PacketFilter
    at java.lang.Class.forName0(Native Method) ~[na:1.6.0_22]
    at java.lang.Class.forName(Class.java:169) ~[na:1.6.0_22]
    at org.jivesoftware.smack.SmackConfiguration.parseClassToLoad(SmackConfiguration.java:237) ~[na:na]
    at org.jivesoftware.smack.SmackConfiguration.(SmackConfiguration.java:82) ~[na:na]
    at org.jivesoftware.smack.XMPPConnection.(XMPPConnection.java:120) ~[na:na]
    at org.apache.camel.component.xmpp.XmppEndpoint.createConnection(XmppEndpoint.java:137) ~[na:na]
    at org.apache.camel.component.xmpp.XmppGroupChatProducer.doStart(XmppGroupChatProducer.java:76) ~[na:na]
    at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:65) ~[na:na]

    ...

    Caused by: java.lang.NullPointerException: null
    at org.jivesoftware.smackx.muc.MultiUserChat$1.connectionCreated(MultiUserChat.java:84) ~[na:na]
    at org.jivesoftware.smack.XMPPConnection.initConnection(XMPPConnection.java:957) ~[na:na]
    at org.jivesoftware.smack.XMPPConnection.connectUsingConfiguration(XMPPConnection.java:904) ~[na:na]
    at org.jivesoftware.smack.XMPPConnection.connect(XMPPConnection.java:1415) ~[na:na]
    at org.apache.camel.component.xmpp.XmppEndpoint.createConnection(XmppEndpoint.java:140) ~[na:na]
    at org.apache.camel.component.xmpp.XmppGroupChatProducer.doStart(XmppGroupChatProducer.java:76) ~[na:na]
    at org.apache.camel.impl.ServiceSupport.start(ServiceSupport.java:65) ~[na:na]

    ...

    22:34:26.176 [Smack Listener Processor (0)] ERROR o.a.c.processor.DefaultErrorHandler - Failed delivery for exchangeId: ID-annafi-51080-1293896062455-0-4. Exhausted after delivery attempt: 1 caught: com.thoughtworks.xstream.mapper.CannotResolveClassException: org.apache.camel.component.bean.BeanInvocation : org.apache.camel.component.bean.BeanInvocation
    com.thoughtworks.xstream.mapper.CannotResolveClassException: org.apache.camel.component.bean.BeanInvocation : org.apache.camel.component.bean.BeanInvocation
    at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(DefaultMapper.java:62) ~[na:na]
    at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:38) ~[na:na]
    at com.thoughtworks.xstream.mapper.DynamicProxyMapper.realClass(DynamicProxyMapper.java:71) ~[na:na]

    ReplyDelete
  2. Nice.

    Wondering if the Smack team will try to make it OSGi-compatible.

    ReplyDelete
  3. everything you do is just so mystical and beautiful. loved this.

    ReplyDelete
  4. We've created a good EMF Cheap Runescape Goldobject product based on XML schema. I will modify this and also increase brand new features and include fresh little one aspects for the item design. However i can not create textual content items towards the physical objects which should be achievable Sell Rs Goldbased on the schema.

    ReplyDelete
  5. These details are created available by a little bit machine crafted in the vehicle, referred to as the Engine Manage Unit or ECU. The ECU is much like the car's "brain" which measures and collects data about your vehicle's wellbeing and then relays this information to the data storage machine that's attached to the OBD 2 interfaces for straightforward entry and convenience.

    ReplyDelete
  6. AWESOME operate. As well as wonderful choice of resources to be effective... Excellent!!
    Cheap D3 Items
    GW2 Cd key kaufen

    ReplyDelete
  7. Microsoft has pulled a Home windows windows 7 product key< seven protection update launched as element of this windows 7 home premium product key month's Patch Tuesday soon after finding it triggered some devices to become unbootable.

    ReplyDelete
  8. the areas will only hold rs gold one or two people maximum as far as spawn rates go, and you have too many people and mobs to support runescape money.

    ReplyDelete
  9. Fieldrunners 2Sony and Subatomic Studios team have confirmed that the sequel to the popular tower defense sell runescape 2007 gold game Fieldrunners, PSP title released in the distant 2009, PS Vita will be released along verano.Basado months where each increasingly popular Tower Defense genre, Fieldrunnes 2 debut a new engine that has allowed the development team to make runescape gold sale each and every one of the enemies describe their own unique path through the battlefield. What does this mean? What if we plant an obstacle in front of them, react naturally and credible adapting to the new circumstances of juego.El program also will feature more than 25 towers to deal with enemies, with news as interesting best site to buy runescape gold as precision strikes and other special skills.

    ReplyDelete