About this Article
In my previous article, Introduction to NIEM and IEPDs, I created an IEPD browser
to navigate a NIEM conformant schema. There are a variety of online tools available for navigating the NIEM model, building schema subsets and IEPDs,
and mapping the schema to a logical model created in UML. This article reviews the online tools available on the www.niem.gov site.
Granted, this is not your usual CodeProject article. My purpose in publishing this article is to introduce CodeProject to some of the free tools
currently available for working with the NIEM, should anyone ever encounter a requirement to work with this model. There is also a lot of opportunity
to further develop tools that work with this model, which is something that I'm interested in pursuing, and perhaps a reader will be interested
in this as well. It is always important to understand the existing tools to better determine what the needs are for the NIEM community,
and that's the purpose of this article.
Introduction
Although it might be possible to build something with just bare hands and ingenuity, it always helps to have a few tools in your toolbox to make
the job easier and more efficient. The same goes for working with the National Information Exchange Model ((NIEM). There are a number of tools available
to help with NIEM, many of which are included on the www.niem.gov website. In this article, we will review the following tools,
and discuss what they do, why they are needed, and how you can use them:
- Search and navigation of the NIEM (part of the SSGT)
- Build Schema Subset using the Subset Schema Generation Tool (SSGT)
- Map Information Exchange
- Work With IEPDs
- Generate Code List Schemas
- Migration Assistance
Note: Bear with me in the listing of the tools, as they don't have catchy names or succinct titles. Many are named simply for what they do;
this can assist in immediate recognition of their value, but also make for awkward article subheadings.
Search and Navigation of the NIEM
Clicking on this link takes you to the NIEM Schema Subset Generation Tool (SSGT). I'll discuss the schema subset generation tool later; for now,
we'll look just at the search and navigation features. The default web page provides the basic search capability, allowing you to search by:
- Property
- Type
- Namespace
- Facet
- External
- Association
The most common searches are for "Property" and "Type". An advanced search option brings up a user interface in which you can specify
advanced options, such as searching for the exact phrase, what you want to match on, and which domains you want to search.
During the IEPD creation process, you will most likely, at some point, create a UML diagram of your logical model. Once the logical model
is created, you will want to search the NIEM for the semantic equivalent types (classes) and elements (properties) of your UML model, so they
can be mapped to NIEM schema types and elements. Furthermore, you will discover NIEM types that are not semantic equivalent, but are suitable
to extend or substitute for a custom type (which goes into your extension schema) required in your exchange.
For example, to find if the NIEM contains a "Citation" type, we can change the "Search for a" to "Type", enter the word "Citation"
in the search box, and click on the Search button. This yields a single result, "j:CitationType".
At this point, we can drill into the elements of the CitationType by clicking on the plus-sign to the left of the "Add" button.
You can drill further into the sub-elements, and so forth, by navigating into the hierarchy. Similarly, to see what type j:CitationType inherits,
you can click on the text "show inheritance", which reveals that j:CitationType inherits from nc:ActivityType.
Lastly, next to each element is the text "details"; when you click on this, it shows the definition for that element.
This is very valuable in checking whether the NIEM type has semantic equivalence to the desired type in the exchange.
Building a Schema Subset Using the Schema Subset Generation Tool (SSGT)
The SSGT allows you to select a subset of NIEM components to include in your exchange. Your exchange will never require all of the types
defined in the NIEM—instead, your exchange will include a subset of components that either have direct semantic equivalence or that you
will extend in the extension schema(s) that are included in your exchange. By selecting a subset of the NIEM components, you reduce the size
of the exchange and improve the parsing/validation performance. The SSGT is used to build this subset, which can then
be downloaded using the "Generate Documents" link. Each component that you add in your subset schema will be placed in the schema
file that corresponds to the namespace from which the component was selected. For example, if we add j:CitationType to our subset schema,
the j:CitationType will be found in the jxdm domain folder and the nc:ActivityType, from which
j:CitationType derives (or inherits), will be found in the niem-core folder.
After we have searched and found the desired component in the NIEM, there are several options for adding the component to our subset schema.
For example, searching for the type "Citation" finds "j:CitationType". We can now add just that type without any elements by clicking
on the "Add" button to the left of "j:CitationType". This adds just the type without any elements.
It is unlikely that you will not want to include any elements of that type; however, you can always go back later and all elements,
and the SSGT, will include them correctly under the CitationType. Alternatively, you can click on the "Add" button to the left
of the text "Add All", and all the elements of that type will be added.
Note that this does not add child elements. For example, the elements under j:CitationIssuedLocation are not added, so if you wanted
specific elements related to the citation issued location (such as nc:LocationAddress), you will need to expand j:CitationIssuedLocation
and add the desired sub-elements. This is a "top-down" approach to generating the subset schema.
Conversely, if you are more comfortable with a "bottom-up" approach, you can drill into the types and start adding types from
the bottom. For example, you could drill into j:CitationType/j:CitationIssuedLocation and first add nc:LocationAddress,
then work your way up by adding j:CitationIssuedLocation.
Because j:CitationIssuedLocation is an element of j:CitationType, j:CitationType is automatically added to the subset schema:
Whereas, j:CitationIssuedLocation was not automatically added when adding just nc:LocationAddress.
This is because the type containing the element nc:LocationAddress is nc:AddressType, whereas the type containing the element j:CitationIssuedLocation
is j:CitationType. When generating a subset schema, the containing type is always included.
When adding elements, you can specify the cardinality right away by clicking on the down arrow next to the "Add" button.
This provides a list of cardinality options and will generate the appropriate "minOccurs" and "maxOccurs" attributes for that element.
The desired cardinality is usually determined by your model and should be documented in the UML model which you are using as a reference for mapping
to the NIEM. You can, however, defer selecting the cardinality until later. Once your subset schema is created, you can click
on the "Edit Cardinality" link to define the cardinality for all elements.
You can also delete types and elements by clicking on the checkbox to the left of the type/element and then clicking on the "Delete" button.
Creating a large subset schema probably won't be accomplished in a single session, and may be the work of several people as well.
The SSGT lets you save the schema subset as a "wantlist", allowing you (or others) to return to your work later on.
The "Save current wantlist to a file" is available on the "Generate Documents" page. You can later load your wantlist from
the "Options" link on the main page of the SSGT, where you will find the section "Load Wantlist".
Map Information Exchange
The mapping tool helps you create the exchange and extension schemas, and to map your UML model with the NIEM. It will also generate
the subset schema for your exchange. In order to use this tool, you will need to generate a UML 1.4 or UML 2.1 metamodel in XMI representation using a tool such as ArgoUML or Eclipse.
Once you have uploaded the XMI model, you can use this tool to map the classes and properties in your UML to the NIEM.
While this tool provides NIEM search capabilities, it is less sophisticated than the SSGT navigation and search, so you may want
to have a separate browser tab or window open to the SSGT. When you locate a type or element using the mapping tool, you have the option
to use this tool, create an exchange on the main web page, and then upload one or more XMI models. You can then begin mapping the exchange by clicking on the XMI model link.
Classes and their derivatives are presented as root level artifacts in a navigation tree. For example, if you load the XMI for the Citation UML,
the navigation tree displays "DocumentType" and "CitationBatch", and both can be expanded to show their properties. Classes that are part
of the UML model by association are not directly represented; instead, the properties of these associative classes can be mapped by navigating
the association described in the UML model. Therefore, it is important that you label your associations (some UML modelers allow for unnamed associations)
because the association names will appear in the navigation tree of the mapping tool.
If the association name is missing, then the mapping tool displays the associated class name. So, when working with the mapping tool,
remember that it is association-centric. Whereas the SSGT will explicitly show you the containing types (for example, adding j:CitationType
automatically brings in nc:ActivityType), the mapping tool will not explicitly show this to you. However, it behaves the same as the SSGT,
so when you map Citation to j:Citation, the niem-core subset schema will also include nc:ActivityType. This brings up another difference
between the mapping tool and the SSGT—with the mapping tool, you are mapping properties to elements.
Once you have mapped your UML classes and properties, you can generate the subset schema, extension schema, and exchange schemas,
by clicking on the "here" in the text: "To generate and download a mapping report, click here." This will take you to a page where,
near the bottom, you can click on the text "generate an exchange schema." From this page, you may also create a mapping report, mapping set, wantlist, subset schema, and constraint schema.
Work with IEPDs
With this tool, you can upload or enter the schemas and documentation required for an IEPD. Along with the metadata, the tool assembles
everything in a package that conforms to the NIEM IEPD specification. With this tool, you can also validate that the minimum artifacts and
metadata are present for a NIEM-conformant IEPD. You can work with several IEPDs at once in various stages of completion. Once the IEPD has
been created, you can make the IEPD, or parts of the IEPD, publically accessible. You may also download IEPDs that other people have shared.
Begin with signing in, as IEPDs are specific to your account. Once you have signed in, you can create a new IEPD by uploading an IEPD
zip file or by creating one using the tool. When creating a new IEPD, you will be asked to specify the root directory name and will be able
to upload various artifacts for that IEPD.
The artifact list is quite extensive, and includes artifact types such as business requirements and rules, constraint, subset, extension and exchange schemas,
various documentation types, and so forth.
After specifying the artifacts (this does not have to be a complete list), you will be asked to enter some basic information about the IEPD.
The last step offers you the option to validate the IEPD and/or upload the IEPD artifacts. The validation does not validate the artifacts—instead,
it validates the information in the metadata, such as version, URI, domain, authoritative source point of contact, etc. You can always add or edit
your artifacts by clicking on the "Edit" link, then selecting one of the options: Edit Metadata and Artifacts, Edit Visibility/Sharing, or Deleting the IEPD.
You may also download the IEPD. This includes all of your artifacts; in addition, the download includes the file "metadata.xml", which describes
your metadata entered on the tool's web pages, as well as a catalog.html file. The catalog file can be viewed in your web browser and provides links
to various artifacts. Most browsers support viewing the XML/XSD files directly in the browser. (Note that Google's Chrome shows only the comments—it
appears to strip off the XML tags in XSD and XML files.)
If you wish, you can make your IEPD publicly available by registering it with the IEPD Clearinghouse. You can also search public IEPDs by clicking
on the "Search" link in the "work with IEPDs" section of the tool's webpage.
Generate Code List Schemas
This tool generates a code list (eye color, hair style, etc.) from an Excel spreadsheet. Code lists are commonly used as substitution
groups along with generic text entry, and represent a pick list of valid options for a particular element. To create your own code list,
download the template zip file. When you extract it and open the XLS document, note that there are two tabs with sample code lists.
You can replace/remove/add new tabs as appropriate. (The nice thing about the tool being tab-aware is that you can manage all of your code lists in a single spreadsheet.)
When you're ready to generate your code list, fill in the information presented on the tool's page and click the "Build Schema"
button (on my browser, the word "Schema" is truncated). After the code list schema is generated, your browser will open a window
to save the file locally. This will include your code list schema, the XLS file you uploaded, and the necessary NIEM artifacts to support the code list, such as the structures schema.
Migration Assistance
The migration assistance tool can convert GJXDM 3.0.3 wantlist to a NIEM 2.0 wantlist, or a NIEM 1.0 wantlist to a NIEM 2.0 wantlist.
The resulting output will be a NIEM 2.0 wantlist, NIEM 2.0 subset schema, and a migration report. The migration report contains actions
taken and choices made in migrating the wantlist, issues that cannot be resolved automatically, and statistics indicating the degree of migration
resolution. This tool cannot migrate extension or constraint schemas.
To use this tool, specify the GJXDM 3.0.3 or NIEM 1.0 wantlist file and the wantlist version, then click on the Migrate button. The generated
documents can then be saved locally for your review.
Conclusion
When creating an IEPD, the search and navigation tools are indispensible when mapping your logical model to the NIEM. Most IEPDs will usually
have code lists that are specific to the domain—creating these code lists with the code list generator tool is a simple process of editing
an Excel spreadsheet to generate the code list artifacts. The mapping tool itself is very useful in creating the subset, extension, and exchange
artifacts required by the IEPD. These artifacts, supplemented with other artifacts (such as documentation, examples, and metadata) using
the "Work with IEPDs" tool provides a comprehensive tool for managing and publishing the IEPD.
Links
