Monday, December 27, 2010

Querying EMF Models using Model Query

Querying i.e. retrieving the list of EMF objects that meet a specific criteria by only navigating the object graph, finding the right objects, plus loops and if-elses... is inconvenient! Not to mention the performance of the lookup would be very slow on large models.

Fortunately there are several solutions (actually, alternatives, since there are still some restrictions) for this:

  1. EMF Model Query
  2. EMF Model Query with OCL constraint language
  3. HQL (Hibernate Query Language) with CDO / Teneo and Hibernate Data Store

Using EMF Model Query

For now I'll only show you how it's done with EMF Model Query because it's the only one I've tried at this point. ;-)

I'm currently porting one of my PHP apps to Eclipse Platform and EMF.

Here's how it looks:

/**
* Finds a refill that is in Refill::STATUS_POSTED status matching this code.
* @param string $voucher_code
* @param string $mobile
* @model
*/
public Refill lookupPosted(String voucherCode, String destNumber) {
EObjectAttributeValueCondition statusCondition = new EObjectAttributeValueCondition(
AbispulsaPackage.Literals.REFILL__STATUS,
new ObjectInstanceCondition(RefillStatus.POSTED));
EObjectReferenceValueCondition voucherCondition = new EObjectReferenceValueCondition(
AbispulsaPackage.Literals.REFILL__VOUCHER,
new EObjectAttributeValueCondition(
AbispulsaPackage.Literals.VOUCHER__CODE, new StringValue(
voucherCode)));
EObjectAttributeValueCondition destNumberCond = new EObjectAttributeValueCondition(
AbispulsaPackage.Literals.REFILL__DEST_NUMBER, new StringValue(
destNumber));
SELECT statement = new SELECT(1, new FROM(this), new WHERE(
statusCondition.AND(voucherCondition).AND(destNumberCond)));
IQueryResult result = statement.execute();
if (!result.isEmpty())
return (Refill)result.iterator().next();
else
return null;
}

If you think the above code is verbose... Yeah, it certainly is! :-(

Here's the original code in PHP :

Refill refill = Doctrine_Query::create()->from('Refill')
->where('status = ?', self::STATUS_POSTED)
->andWhere('voucher_code = ?', $voucher_code)
->andWhere('dest_number = ?', $mobile)
->limit(1)->fetchOne();

Much more compact. Not to mention it's definitely much faster than EMF Query since the PHP code directly accesses the backend storage (MySQL) with all the indexing goodies of Relational DBMS.

However, it's still a good way than coding your own loops and ifs. For small models and if you don't mind verbosity, plain EMF Model Query perfect and very flexible.

EMF Model Query with OCL

EMF Model Query with OCL works just like the plain EMF Model Query, but with a much nicer and widely used standard constraint syntax.

A simple OCL is :

self.name = 'Bob'

Expressing the same criteria in plain EMF Model Query would be much longer.

Typical code looks like this (from EMF Model Query guide):

Resource myResource = ... // get the resource

OCL ocl = org.eclipse.ocl.ecore.OCL.newInstance(); Condition condition = new BooleanOCLCondition<EClassifier, EClass, EObject>( ocl.getEnvironment(), "self.books->collect(b : Book | b.category)->asSet()->size() > 2", EXTLibraryPackage.Literals.WRITER); SELECT statement = new SELECT(SELECT.UNBOUNDED, false, new FROM(myResource.getContents()), new WHERE(condition), new NullProgressMonitor()); IQueryResult results = statement.execute(); // do something with the results selectInEditor(results);
See more EMF Model Query with OCL Examples.

HQL on CDO/Teneo + Hibernate

This one is sort of cheating, because it doesn't really use EMF capabilities at all, but under the hood delegates to the underlying persistence storage which is Hibernate which in turn is backed by a relational database (such as MySQL, HSQLDB, PostgreSQL, you name it).

The objects returned by Hibernate are then "converted" into EMF Objects. Actually they're already EMF Objects (of course!) but I suspect CDO/Teneo does additional processing like attaching the objects to the proper EMF Resource.

The code is much more compact and developer-friendly: (especially if you're a Hibernate developer)

CDOSession session = openSession();
CDOTransaction transaction = session.openTransaction();  
CDOQuery cdoQuery = transaction.createQuery("hql", "from Product where vat=:vat"); cdoQuery.setParameter("vat", VAT.VAT15); List<Product> products = cdoQuery.getResult(Product.class);
transaction.commit();
Cool, huh? Compact, easy, and strongly typed! And it supports parameterized queries! And results are fast!

More about HQL Support on CDO Eclipsepedia Wiki.

Conclusion

I think using HSQL is the best solution from both the performance standpoint and ease of use.

However it requires to use a data store that supports a query language, such as the Hibernate Data Store (I think the CDO Objectivity data store should also support querying).

In theory, CDO supports pluggable query handler so that CDO Data Stores are free to implement their own optimized query language (with parameter support!) However whether this applies in practice (for other data stores), I still have to find out. :-)

8 comments:

  1. Hi Hendy,

    CDO supports a number of different storage backends like plain SQL databases, Hibernate, Objectivity/DB, DB4O and MongoDB (coming soon). Most of these come with their own, native query language and CDO is able to execute queries in these languages on the server by delegating directly to the storage backend.

    In addition we've just added the first "common" query language handler for OCL queries that works independently of the configured storage backend. With the same bugzilla 256931 we've introduced the ability to have local dirty state be considered by the query execution engine on the server.

    Cheers
    /Eike

    ReplyDelete
  2. You can have a look at EMF Path, a new project to query larges EMF models with a focus on performance: http://code.google.com/a/eclipselabs.org/p/emfpath/

    There will be a talk at EcliseCon on this project : https://www.eclipsecon.org/submissions/2011/view_talk.php?id=2164&search=emf+path

    ReplyDelete
  3. @Eike thanks for the information.

    The OCL query handler seems cool if it's practical to work with, i.e. querying a medium sized database. How it compares to HQL? With anything bigger than tiny models query performance is important, not just convenience..

    @Etienne thanks for the info on EMFPath. It looks very convenient, I still doubt its performance though, as I believe it still requires the EMF models be loaded into memory.

    When models can occupy several hundred MBs, I believe the only performant querying is one that is index-based (not just iterative, no matter how many threads you use, since the loading of models from disk is one bottleneck).

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Additional info from Ashwani Kr Sharma from SAP:

    EMF Query2 is another option. It is quite powerful and destined to replace EMF Query.

    Check EMF Query2 Dev Guide for more details: http://wiki.eclipse.org/EMF/Query2/DevGuide

    ReplyDelete
  6. While versions can takeBuy Cheap RS Gold up hundreds of MBs, In my opinion the one performant querying is certainly one that's index-based (not just repetitive, no matter how a lot of posts you utilize, Sell Rs Goldsince the launching associated with versions through computer is a bottleneck).

    ReplyDelete
  7. Blizzard's account support teams experienced high requirement during this period, declaring that many users had been infected www.arm2teeth.com

    ReplyDelete
  8. Ubisoft has confirmed that his remarkable musical title Rocksmith, mainly for players who want to learn rs gold 07 to play guitar or bass, has sold 1.4 million copies worldwide since its launch on the market a year ago and medio.Adem s, the company gala has been referred to a report which highlights that enjoy Rocksmith is "the fastest way to learn to play the guitar." So much cheap 07 runescape gold so, that about 95 percent of users say they have improved their ability to guitarra.Por addition, Ubisoft has released more curious as other data that have been downloaded more than 3 million additional songs, or have performed more than 120,000 million runescape gold shop musicales.

    ReplyDelete