Skipper - The ORM Designer Blog
  • Skipper - The ORM Designer
  • VsBuilds - Parallel building
  • Pulpo - Free Skipper CLI

Tag: #annotations

Magic behind ORM Designer

mac bright

Hi All,

Quite a lot of people asked us about how ORM Designer2 insides work. Some wanted to customize it, some were asking because they were curious about some future features. So I decided to give you exhausting inside look:

Historic background

We have first thought of ORM Designer as a tool to help us with our work at that time, web sites and e-commerce. I spent a lot of time thinking about how to deal with definitions in different formats. In those early days only Doctrine and Propel were in widespread use. So we only needed to work with YAML and XML, and the file structure was also quite simple: one module = one file.

First version was really rudimentary, compared with the latest ORM Designer2 version. Whole import/export logic was hard-coded in the application itself. But as ORM Designer grew in complexity, it became necessary to customize and expand import/export process. To deal with it, we have created our internal XML scripting language (in version 1.2). While still pretty basic, this internal language helped a lot in further expanding ORM Designer.

With the Doctrine2 we have met series of big challenges. Most notably need to import/export each entity as a separate file, and need for native support of annotations format. To implement these changes, we decided to rework the application completely.

ORM Designer 2 comes in January 2013. Application is completely redone using Qt framework. Now, ORM Designer not only becomes multi-platform, but the focus was also on the maximum modularity and possibility to expand and customize the whole application. With the new version comes also the support for the annotations format.

Dealing with different formats

As you know, ORM Designer 2 supports three different formats for the definition files: YAML, XML and PHP annotations. To process such different formats, all the files are at first transformed into XML. Its easy for XML files :). And its pretty straightforward for YAML. The PHP annotations require an extra parser which transforms the PHP file to the AST (abstract-syntax-tree), and this tree is subsequently transformed to the XML format.

XSLT transformations</h4>

Now, we have the input in the XML format, but the content of the file is still dependent on the original file format. These XML files are transformed using specific XSLT template and the result is the so called abstract-XML. This file contains all the information of the input file, but its structure is unified and format independent.

Last step is to transform the abstract-XML to ORM Designer project file. The project file is a regular XML file with additional information for the model layout. This file is loaded by ORM Designer, but can be as easily edited manually.

Export uses the same principles, only in the reversed order: from ORM Designer project file to abstract-XML, then to XML files that have the structure representing the final output format, and in the end the actual definition files are generated.

The XSLT processor used is the LibXslt library, but we have added several transformation functions of our own.

ORM Designer import process

ORM Designer import process

Scripting

Another crucial concept in ORM Designer is the scripting support. Because every ORM is different, different way of processing and different number of transformations is required. To accommodate this, we have implemented the JavaScript support together with a few script extensions.

This scripting allowed us to easily describe the transformation process in required detail for each ORM Framework individually. Its even possible for users to tweak and extend the import/export script as much as they want.

Scripting is used for import, export and directory lookup prior to the import. I have plans to expand ORM Designer functionality in the future. Users then should be able to write and launch their own scripts directly from the ORM Designer GUI.

Annotations

Annotation support is very different from the way XML/YAML files are processed. Annotation source files needs to be parsed into individual language elements. These elements are then organized in the AST (abstract syntax tree), which represents the information and logic stored in the code.

The AST is then further parsed and only the Classes, Attributes and their annotations are kept, the simplified syntax tree is created. Simplified syntax tree contains only ORM specific information. This simplified tree is finally transformed to the XML and further processed using XSLT transformations.

Because the annotations contain further information apart the ORM specific code, the export is even more complicated. The import process is reversed: from ORM Designer project file, through abstract XML to AST. But now comes one extra step.

Second AST is created from the target file and it is compared to the AST created by the export process. These two trees are then merged, so that only changed or newly added ORM annotations are modified. This ensures that no other part of the original file is modified or removed.

Few words at the end

I hope this article helped you to take a peek on the insides of your favorite software. If you are more interested in the inner workings of ORM Designer, tell me in the comments below.

Ludek Vodicka Chief developer

24 Jan 2014