// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

[[ugr.tug.xmi_emf]]
= XMI and EMF Interoperability
// <titleabbrev>XMI &amp; EMF</titleabbrev>


[[ugr.tug.xmi_emf.overview]]
== Overview

In traditional object-oriented terms, a UIMA Type System is a class model and a UIMA CAS is an object graph.
There are established standards in this area –specifically, #UML(TM) is an #
      OMG(TM) standard for class models and XMI (XML Metadata Interchange) is an OMG standard for the XML representation of object graphs.

Furthermore, the Eclipse Modeling Framework (EMF) is an open-source framework for model-based application development, and it is based on UML and XMI.
In EMF, you define class models using a metamodel called Ecore, which is similar to UML.
EMF provides tools for converting a UML model to Ecore.
EMF can then generate Java classes from your model, and supports persistence of those classes in the XMI format.

The UIMA SDK provides tools for interoperability with XMI and EMF.
These tools allow conversions of UIMA Type Systems to and from Ecore models, as well as conversions of UIMA CASes to and from XMI format.
This provides a number of advantages, including:

____
You can define a model using a UML Editor, such as Rational Rose or EclipseUML, and then automatically convert it to a UIMA Type System.

You can take an existing UIMA application, convert its type system to Ecore, and save the CASes it produces to XMI.
This data is now in a form where it can easily be ingested by an EMF-based application.
____

More generally, we are adopting the well-documented, open standard XMI as the standard way to represent UIMA-compliant analysis results (replacing the UIMA-specific XCAS format). This use of an open standard enables other applications to more easily produce or consume these UIMA analysis results.

For more information on XMI, see Grose et al. _Mastering XMI. Java Programming with XMI, XML, and
      UML._ John Wiley & Sons, Inc.
2002.

For more information on EMF, see Budinsky et al. _Eclipse Modeling Framework 2.0._ Addison-Wesley.
2006.

For details of how the UIMA CAS is represented in xref:ref.adoc#ugr.ref.xmi[XMI format].

[[ugr.tug.xmi_emf.converting_ecore_to_from_uima_type_system]]
== Converting an Ecore Model to or from a UIMA Type System

The UIMA SDK provides the following two classes:

*``**Ecore2UimaTypeSystem:**``* converts from an .ecore model developed using EMF to a UIMA-compliant TypeSystem descriptor.
This is a Java class that can be run as a standalone program or invoked from another Java application.
To run as a standalone program, execute:

`java org.apache.uima.ecore.Ecore2UimaTypeSystem <ecore
      file> <output file>`

The input .ecore file will be converted to a UIMA TypeSystem descriptor and written to the specified output file.
You can then use the resulting TypeSystem descriptor in your UIMA application.

*``**UimaTypeSystem2Ecore:**``* converts from a UIMA TypeSystem descriptor to an .ecore model.
This is a Java class that can be run as a standalone program or invoked from another Java application.
To run as a standalone program, execute:

`java org.apache.uima.ecore.UimaTypeSystem2Ecore
      <TypeSystem descriptor> <output file>`

The input UIMA TypeSystem descriptor will be converted to an Ecore model file and written to the specified output file.
You can then use the resulting Ecore model in EMF applications.
The converted type system will include any ``<import...>``ed TypeSystems; the fact that they were imported is currently not preserved.

To run either of these converters, your classpath will need to include the UIMA jar files as well as the following jar files from the EMF distribution: common.jar, ecore.jar, and ecore.xmi.jar.

Also, note that the uima-core.jar file contains the Ecore model file uima.ecore, which defines the built-in UIMA types.
You may need to use this file from your EMF applications.

[[ugr.tug.xmi_emf.using_xmi_cas_serialization]]
== Using XMI CAS Serialization

The UIMA SDK provides XMI support through the following two classes:

*``**XmiCasSerializer:**``* can be run from within a UIMA application to write out a CAS to the standard XMI format.
The XMI that is generated will be compliant with the Ecore model generated by ``UimaTypeSystem2Ecore``.
An EMF application could use this Ecore model to ingest and process the XMI produced by the XmiCasSerializer.

*``**XmiCasDeserializer:**``* can be run from within a UIMA application to read in an XMI document and populate a CAS.
The XMI must conform to the Ecore model generated by ``UimaTypeSystem2Ecore``.

Also, the uimaj-examples Eclipse project contains some example code that shows how to use the serializer and deserializer: 

____
`org.apache.uima.examples.xmi.XmiWriterCasConsumer:` This is a CAS Consumer that writes each CAS to an output file in XMI format.
It is analogous to the XCasWriter CAS Consumer that has existed in prior UIMA versions, except that it uses the XMI serialization format.

`org.apache.uima.examples.xmi.XmiCollectionReader:` This is a Collection Reader that reads a directory of XMI files and deserializes each of them into a CAS.
For example, this would allow you to build a Collection Processing Engine that reads XMI files, which could contain some previous analysis results, and then do further analysis.
____

Finally, in under the folder `uimaj-examples/ecore_src` is the class ``org.apache.uima.examples.xmi.XmiEcoreCasConsumer``, which writes each CAS to XMI format and also saves the Type System as an Ecore file.
Since this uses the `UimaTypeSystem2Ecore` converter, to compile it you must add to your classpath the EMF jars common.jar, ecore.jar, and ecore.xmi.jar – see ecore_src/readme.txt for instructions.

[[ugr.tug.xmi_emf.xml_character_issues]]
=== Character Encoding Issues with XML Serialization

Note that not all valid Unicode characters are valid XML characters, at least not in XML 1.0.
Moreover, it is possible to create characters in Java that are not even valid Unicode characters, let alone XML characters.
As UIMA character data is translated directly into XML character data on serialization, this may lead to issues.
UIMA will therefore check that the character data that is being serialized is valid for the version of XML being used.
If  non-serializable character data is encountered during serialization, an exception is thrown and serialization fails (to avoid creating invalid XML data).  UIMA does not simply replace the offending characters with some valid replacement character; the assumption being that most applications would not like to have their data modified automatically. 

If you know you are going to use XML serialization, and you would like to avoid such issues on serialization, you should check any character data you create in UIMA ahead of time.
Issues most often arise with the document text, as documents may originate at various sources, and may be of varying quality.
So it's a particularly good idea to check the document text for characters that will cause issues for serialization. 

UIMA provides a handful of functions to assist you in checking Java character data.
Those methods are located in ``org.apache.uima.internal.util.XMLUtils.checkForNonXmlCharacters()``, with several overloads.
Please check the javadocs for further information. 

Please note that these issues are not specific to XMI serialization, they apply to the older XCAS format in the same way. 