// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

[[ugr.project_overview]]
= UIMA Overview
// <titleabbrev>Overview</titleabbrev>

The Unstructured Information Management Architecture (UIMA) is an architecture and software framework for creating, discovering, composing and deploying a broad range of multi-modal analysis capabilities and integrating them with search technologies.
The architecture is undergoing a standardization effort,  referred to as the _UIMA specification_ by a technical committee within http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=uima[OASIS]. 

The _Apache UIMA_ framework is an Apache licensed, open source implementation of the UIMA Architecture, and provides a run-time environment in which developers can plug in and run their UIMA component implementations and with which they can build and deploy UIM applications.
The framework itself is not specific to any IDE or platform.

It includes an all-Java implementation of the UIMA framework for the development, description, composition and deployment of UIMA components and applications.
It also provides the developer with an Eclipse-based (http://www.eclipse.org/ ) development environment that includes a set of tools and utilities for using UIMA.
It also includes  a C++ version of the framework, and enablements for Annotators built in Perl, Python, and TCL.

This chapter is the intended starting point for readers that are new to the Apache UIMA Project.
It includes this introduction and the following sections:

* <<ugr.project_overview_doc_overview>> provides a list of the books and topics included in the Apache UIMA documentation with a brief summary of each. 
* <<ugr.project_overview_doc_use>> describes a recommended path through the documentation to help get the reader up and running with UIMA 

The main website for Apache UIMA is http://uima.apache.org.
Here you  can find out many things, including: 

* how to download (both the binary and source distributions
* how to participate in the development
* mailing lists - including the user list used like a forum for questions and answers
* a Wiki where you can find and contribute all kinds of information, including tips and best practices
* a sandbox - a subproject for potential new additions to Apache UIMA or to subprojects of it. Things here are works in progress, and may (or may not) be included in releases.
* links to conferences


[[ugr.project_overview_doc_overview]]
== Apache UIMA Project Documentation Overview

The user documentation for UIMA is organized into several parts. 

* Overviews - this documentation 
* Eclipse Tooling Installation and Setup - also in this document 
* Tutorials and Developer's Guides 
* Tools Users' Guides 
* References 
* Version 3 users-guide

The first 2 parts make up this book; the last 4 have individual  books.
The books are provided both as (somewhat large) html files, viewable in browsers, and also as PDF files.
The documentation is fully hyperlinked, with tables of contents.
The PDF versions are set up to  print nicely - they have page numbers included on the cross references within a book. 

If you view the PDF files inside a browser that supports imbedded viewing of PDF, the hyperlinks between different PDF books may work (not  all browsers have been tested...).

The following set of tables gives a more detailed overview of the various parts of the documentation. 

[[ugr.project_overview_overview]]
=== Overviews

[cols="1,1", frame="all"]
|===

|__xref:#ugr.project_overview_doc_overview[Overview of the Documentation]__
| What you are currently reading.
Lists the documents provided in the Apache  UIMA documentation set and provides a recommended path through the documentation for getting started using UIMA.
It includes release notes and provides a brief high-level description of  the different software modules included in the Apache UIMA Project.

|__xref:#ugr.ovv.conceptual[Conceptual Overview]__
|Provides a broad conceptual overview of the UIMA component architecture; includes references to the other documents in the documentation set that provide more detail.

|__xref:#ugr.faqs[UIMA FAQs]__
|Frequently Asked Questions about general UIMA concepts. (Not a programming resource.)

|__xref:#ugr.issues[Known Issues]__
|Known issues and problems with the UIMA SDK.

|__xref:#ugr.glossary[Glossary]__
|UIMA terms and concepts and their basic definitions.
|===

[[ugr.project_overview_setup]]
=== Eclipse Tooling Installation and Setup

Provides step-by-step instructions for installing Apache UIMA in the Eclipse Interactive Development Environment.
See <<ugr.ovv.eclipse_setup>>.

[[ugr.project_overview_tutorials_dev_guides]]
=== Tutorials and Developer's Guides

[cols="1,1"]
|===

|__xref:tug.adoc#ugr.tug.aae[Annotators and Analysis Engines]__
|Tutorial-style guide for building UIMA annotators and analysis engines. This chapter
                introduces the developer to creating type systems and using UIMA's common data structure,
                the CAS or Common Analysis Structure. It demonstrates how to use built in tools to specify and create
                basic UIMA analysis components.

|__xref:tug.adoc#ugr.tug.cpe[Building UIMA Collection Processing Engines]__
|Tutorial-style guide for building UIMA collection processing engines. These
               manage the analysis of collections of documents from source to sink.

|__xref:tug.adoc#ugr.tug.application[Developing Complete Applications]__
|Tutorial-style guide on using the UIMA APIs to create, run and manage UIMA components from
                your application. Also describes APIs for saving and restoring the contents of a CAS using an XML
                format called XMI(TM).

|__xref:tug.adoc#ugr.tug.fc[Flow Controller]__
|When multiple components are combined in an Aggregate, each CAS flow among the various
                components. UIMA provides two built-in flows, and also allows custom flows to be
                implemented.

|__xref:tug.adoc#ugr.tug.aas[Developing Applications using Multiple Subjects of Analysis]__
|A single CAS maybe associated with multiple subjects of analysis (Sofas). These are useful
                for representing and analyzing different formats or translations of the same document. For
                multi-modal analysis, Sofas are good for different modal representations of the same stream
                (e.g., audio and close-captions).This chapter provides the developer details on how to use
                multiple Sofas in an application.

|__xref:tug.adoc#ugr.tug.mvs[Multiple CAS Views of an Artifact]__
|UIMA provides an extension to the basic model of the CAS which supports 
              analysis of multiple views of the same artifact, all contained with the CAS. This 
              chapter describes the concepts, terminology, and the API and XML extensions that 
              enable this

|__xref:tug.adoc#ugr.tug.cm[CAS Multiplier]__
|A component may add additional CASes into the workflow. This may be useful to break up a large
                artifact into smaller units, or to create a new CAS that collects information from multiple other
                CASes.

|__xref:tug.adoc#ugr.tug.xmi_emf[XMI and EMF Interoperability]__
|The UIMA Type system and the contents of the CAS itself can be externalized using the XMI
                standard for XML MetaData. Eclipse Modeling Framework (EMF) tooling can be used to develop
                applications that use this information.
|===

[[ugr.project_overview_tool_guides]]
=== Tools Users' Guides

[cols="1,1"]
|===

|__xref:tools.adoc#ugr.tools.cde[Component Descriptor Editor]__
|Describes the features of the Component Descriptor Editor Tool. This tool provides a GUI for
                specifying the details of UIMA component descriptors, including those for Analysis Engines
                (primitive and aggregate), Collection Readers, CAS Consumers and Type Systems.

|__xref:tools.adoc#ugr.tools.cpe[Collection Processing Engine Configurator]__
|Describes the User Interfaces and features of the CPE Configurator tool. This tool allows the
                user to select and configure the components of a Collection Processing Engine and then to run the
                engine.

|__xref:tools.adoc#ugr.tools.pear.packager[PEAR Packager]__
|Describes how to use the PEAR Packager utility. This utility enables developers to produce an
                archive file for an analysis engine that includes all required resources for installing that
                analysis engine in another UIMA environment.

|__xref:tools.adoc#ugr.tools.pear.installer[PEAR Installer]__
|Describes how to use the PEAR Installer utility. This utility installs and verifies an
                analysis engine from an archive file (PEAR) with all its resources in the right place so it is ready to
                run.

|__xref:tools.adoc#ugr.tools.pear.merger[PEAR Merger]__
|Describes how to use the PEAR Merger utility, which does a simple merge of multiple PEAR
                packages into one.

|__xref:tools.adoc#ugr.tools.doc_analyzer[Document Analyzer]__
|Describes the features of a tool for applying a UIMA analysis engine to a set of documents and
                viewing the results.

|__xref:tools.adoc#ugr.tools.cvd[CAS Visual Debugger]__
|Describes the features of a tool for viewing the detailed structure and contents of a CAS. Good
                for debugging.

|__xref:tools.adoc#ugr.tools.jcasgen[JCasGen]__
|Describes how to run the JCasGen utility, which automatically builds Java classes that
                correspond to a particular CAS Type System.

|__xref:tools.adoc#ugr.tools.annotation_viewer[XML CAS Viewer]__
|Describes how to run the supplied viewer to view externalized XML forms of CASes. This viewer
                is used in the examples.
|===

[[ugr.project_overview_reference]]
=== References

[cols="1,1"]
|===

|__xref:ref.adoc#ugr.ref.javadocs[Introduction to the UIMA API Javadocs]__
|Javadocs detailing the UIMA programming interfaces.

|__xref:ref.adoc#ugr.ref.xml.component_descriptor[XML: Component Descriptor]__
|Provides detailed XML format for all the UIMA component descriptors, except the CPE (see next).

|__xref:ref.adoc#ugr.ref.xml.cpe_descriptor[XML: Collection Processing Engine Descriptor]__
|Provides detailed XML format for the Collection Processing Engine descriptor.

|__xref:ref.adoc#ugr.ref.cas[CAS]__
|Provides detailed description of the principal CAS interface.

|__xref:ref.adoc#ugr.ref.jcas[JCas]__
|Provides details on the JCas, a native Java interface to the CAS.

|__xref:ref.adoc#ugr.ref.pear[PEAR Reference]__
|Provides detailed description of the deployable archive format for UIMA components.

|__xref:ref.adoc#ugr.ref.xmi[XMI CAS Serialization Reference]__
|Provides detailed description of the deployable archive format for UIMA components.

|===

[[ugr.project_overview_v3]]
=== Version 3 User's guide

This book describes Version 3's features, capabilities, and differences with version 2. 

[[ugr.project_overview_doc_use]]
== How to use the Documentation

. Explore this chapter to get an overview of the different documents that are included with Apache UIMA.
. Read xref:#ugr.ovv.conceptual[xrefstyle=full] to get a broad view of the basic UIMA concepts and philosophy with reference to the other documents included in the documentation set which provide greater detail. 
. For more general information on the UIMA architecture and how it has been used, refer to the IBM Systems Journal special issue on Unstructured Information Management, on-line at http://www.research.ibm.com/journal/sj43-3.html or to the section of the UIMA project website on Apache website where other publications are listed. 
. Set up Apache UIMA in your Eclipse environment. To do this, follow the instructions in xref:#ugr.ovv.eclipse_setup[xrefstyle=full]. 
. Develop sample UIMA annotators, run them and explore the results. Read the xref:tug.adoc#ugr.tug.aae[Annotator and Analysis Engine Developer's Guide] and follow it like a tutorial to learn how to develop your first UIMA annotator and set up and run your first UIMA analysis engines. 
** As part of this you will use a few tools including 
*** The UIMA Component Descriptor Editor, described in more detail in the xref:tools.adoc#ugr.tools.cde[Component Descriptor Editor User's Guide] and 
*** The Document Analyzer, described in more detail in xref:tools.adoc#ugr.tools.doc_analyzer[Document Analyzer User's Guide].
** While following along in xref:tug.adoc#ugr.tug.aae[Tutorials and User's Guides], reference documents that may help are: 
*** xref:ref.adoc#ugr.ref.xml.component_descriptor[Component Descriptor Reference] for understanding the analysis engine descriptors 
*** xref:ref.adoc#ugr.ref.jcas[JCas Reference] for understanding the JCas. 
. Learn how to create, run and manage a UIMA analysis engine as part of an application. Connect your analysis engine to the provided semantic search engine to learn how a complete analysis and search application may be built with Apache UIMA. The xref:tug.adoc#ugr.tug.application[Application Developer's Guide] will guide you through this process. 
** As part of this you will use the document analyzer (described in more detail in xref:tools.adoc#ugr.tools.doc_analyzer[Document Analyzer User's Guide] and semantic search GUI tools.
. Pat yourself on the back. Congratulations! If you reached this step successfully, then you have an appreciation for the UIMA analysis engine architecture. You would have built a few sample annotators, deployed UIMA analysis engines to analyze a few documents, searched over the results using the built-in semantic search engine and viewed the results through a built-in viewer -- all as part of a simple but complete application. 
. Develop and run a Collection Processing Engine (CPE) to analyze and gather the results of an entire collection of documents. xref:tug.adoc#ugr.tug.cpe[Collection Processing Engine Developer's Guide] will guide you through this process. 
** As part of this you will use the CPE Configurator tool. For details see xref:tools.adoc#ugr.tools.cpe[Collection Processing Engine Configurator User's Guide]
** You will also learn about CPE Descriptors. The detailed format for these may be found in the xref:ref.adoc#ugr.ref.xml.cpe_descriptor[Collection Processing Engine Descriptor Reference].
. Learn how to package up an analysis engine for easy installation into another UIMA environment. xref:tools.adoc#ugr.tools.pear.packager[PEAR Packager User's Guide] and xref:tools.adoc#ugr.tools.pear.installer[PEAR Installer User's Guide] will teach you how to create UIMA analysis engine archives so that you can easily share your components with a broader community. 

[[ugr.project_overview_changes_from_previous]]
== Changes from UIMA Version 2

See the separate document Version 3 User's Guide.s

[[ugr.project_overview_migrating_from_v2_to_v3]]
== Migrating existing UIMA pipelines from Version 2 to Version 3

The format of JCas classes changed when going from version 2 to version 3.
If you had JCas classes for user types, these need to be regenerated using the  version 3 JCasGen tooling or Maven plugin.
Alternatively, these can be  migrated without regenerating; the migration preserves any customization  users may have added to the JCas classes.

The Version 3 User's Guide has a chapter detailing the migration, including a description of the migration tool to aid in this process.

[[ugr.project_overview_summary]]
== Apache UIMA Summary

[[ugr.ovv.summary.general]]
=== General

UIMA supports the development, discovery, composition and deployment of multi-modal analytics for the analysis of unstructured information and its integration with search technologies.

Apache UIMA includes APIs and tools for creating analysis components.
Examples of analysis components include tokenizers, summarizers, categorizers, parsers, named-entity detectors etc.
Tutorial examples are provided with Apache UIMA; additional components are available from the community. 

[[ugr.ovv.summary.programming_language_support]]
=== Programming Language Support

UIMA supports the development and integration of analysis algorithms developed in different programming languages. 

The Apache UIMA project is both a Java framework and a matching C++ enablement layer, which allows annotators to be written in C++ and have access to a C++ version of the CAS.
The C++ enablement layer also enables annotators to be written in Perl, Python, and TCL, and to interoperate with those written in other languages. 

[[ugr.ovv.general.summary.multi_modal_support]]
=== Multi-Modal Support

The UIMA architecture supports the development, discovery, composition and deployment of multi-modal analytics, including text, audio and video. xref:tug.adoc#ugr.tug.aas[Annotations, Artifacts, and Sofas] discuss this is more detail.

[[ugr.project_overview_summary_sdk_capabilities]]
== Summary of Apache UIMA Capabilities

[cols="1,1", frame="all"]
|===

|Module
|Description

|UIMA Framework Core
|

A framework integrating core functions for creating, deploying, running and managing UIMA components, including analysis engines and Collection Processing Engines in collocated and/or distributed configurations. 

The framework includes an implementation of core components for transport layer adaptation, CAS management, workflow management based on declarative specifications, resource management, configuration management, logging, and other functions.

|C++ and other programming language Interoperability
|

Includes C++ CAS and supports the creation of UIMA compliant C++ components that can be deployed in the UIMA run-time through a built-in JNI adapter.
This includes high-speed binary serialization.

Includes support for creating service-based UIMA engines.
This is ideal for wrapping existing code written in different languages.

|Framework Services and APIs
|Note that interfaces of these components are available to the developer
              but different implementations are possible in different implementations of the UIMA
              framework.

|CAS
|These classes provide the developer with typed access to the Common Analysis Structure (CAS),
              including type system schema, elements, subjects of analysis and indices. Multiple subjects of
              analysis (Sofas) mechanism supports the independent or simultaneous analysis of multiple views of
              the same artifacts (e.g. documents), supporting multi-lingual and multi-modal analysis.

|JCas
|An alternative interface to the CAS, providing Java-based UIMA Analysis components with
              native Java object access to CAS types and their attributes or features, using the
              JavaBeans conventions of getters and setters.

|Collection Processing Management (CPM)
|Core functions for running UIMA collection processing engines in collocated and/or
              distributed configurations. The CPM provides scalability across parallel processing pipelines,
              check-pointing, performance monitoring and recoverability.

|Resource Manager
|Provides UIMA components with run-time access to external resources handling capabilities
              such as resource naming, sharing, and caching. 

|Configuration Manager
|Provides UIMA components with run-time access to their configuration parameter settings. 

|Logger
|Provides access to a common logging facility.

| Tools and Utilities 

|JCasGen
|Utility for generating a Java object model for CAS types from a UIMA XML type system
              definition.

|Saving and Restoring CAS contents
|APIs in the core framework support saving and restoring the contents of a CAS to streams 
              in multiple formats, including XMI, binary, and compressed forms.  
              These apis are collected into the CasIOUtils class.

|PEAR Packager for Eclipse
|Tool for building a UIMA component archive to facilitate porting, registering, installing and
              testing components.

|PEAR Installer
|Tool for installing and verifying a UIMA component archive in a UIMA installation.

|PEAR Merger
|Utility that combines multiple PEARs into one.

|Component Descriptor Editor
|Eclipse Plug-in for specifying and configuring component descriptors for UIMA analysis
              engines as well as other UIMA component types including Collection Readers and CAS
              Consumers.

|CPE Configurator
|Graphical tool for configuring Collection Processing Engines and applying them to
              collections of documents.

|Java Annotation Viewer
|Viewer for exploring annotations and related CAS data.

|CAS Visual Debugger
|GUI Java application that provides developers with detailed visual view of the contents of a
              CAS.

|Document Analyzer
|GUI Java application that applies analysis engines to sets of documents and shows results in a
              viewer.

|CAS Editor
|Eclipse plug-in that lets you edit the contents of a CAS

|UIMA Pipeline Eclipse Launcher
|Eclipse plug-in that lets you configure Eclipse launchers for UIMA pipelines

| Example Analysis Components 

|Database Writer
|CAS Consumer that writes the content of selected CAS types into a relational database, using
              JDBC. This code is in cpe/PersonTitleDBWriterCasConsumer. 

|Annotators
| Set of simple annotators meant for pedagogical purposes. Includes: Date/time, Room-number,
              Regular expression, Tokenizer, and Meeting-finder annotator. There are sample CAS Multipliers
              as well. 

|Flow Controllers
| There is a sample flow-controller based on the whiteboard concept of sending the CAS to whatever
              annotator hasn't yet processed it, when that annotator's inputs are available in the CAS. 

|XMI Collection Reader, CAS Consumer
|Reads and writes the CAS in the XMI format

|File System Collection Reader
| Simple Collection Reader for pulling documents from the file system and initializing CASes. 
|===