IMS Metadata Extensibility Proposal

CO 10/14/98

Overview

Basic requirements:

Base IMS metadata schemas must be extensible easily

It must be possible to validate extensions

Must be extremely simple

Extended meta-data must be "interleaved" with base meta-data, not in a separate semantic space

Must conform to XML 1.0 and be upward compatible with future XML data standards

Must not be tied to a current proprietary XML extension standard

Data created must be retrievable for use when better meta-data standard storage and encoding formats emerge

Meta-data created with Spec 1.0 must be convertible with simple tools to future industry standard meta-data formats, such as, for example, the promising XMI proposed standard.

Interpreting and creating the meta-data exchange format should not depend on proprietary or patented tools or algorithms.

Basic assumptions

XML will be used to exchange Meta-Data

The current state of the XML standard is unstable. However, there is a stable XML 1.0 specification.

The abstraction (meaning) and semantic structure of the meta-data is more important than religious adherence to the limitations of a particular technological standard. The primary goal of meta-data is to convey semantic information. Encoding and technological restrictions or rules are secondary to the primary goal.

Synopsis of proposal

Well-formed XML files are acceptable as a means of exchanging meta-data..

The use of a DTD is optional.

The base schema will not change. The complete name of an element (including its "path" in the meta-data structure) is critical in identifying the element and where it fits in the semantic meta-data schema.

As specified in XML 1.0, semantics and particularly semantics of subschemas are a matter of documentation, and are not encoded anywhere in IMS MD1.0

Validation is optional. There are 3 kinds of DTDs: Base DTD, registered DTD (e.g. in NIST registry), unregistered DTD.

Validation is done by combining more than one DTD

There are levels of compatibility and validity warning when validating each element in a well-formed meta-data file:

0 = (No error) Entirely valid, or don’t care
1 = Warning: unknown element
2 = Custom validation error (attempt to redefine an element specified in an unregistered but accessible extension DTD, or missing element specified as required by the DTD),
3 = Base validation error (attempt to redefine a base or registered extension element, or missing element specified as required by the DTD).

The default error handling of a system that reads metadata is to ignore warnings (i.e., to ignore unknown data elements).
The base schema is based on the IMS/ARIADNE/IEEE LTSC LOM schema adopted at the Sept. 1998 IEEE LTSC meeting in Leuven. As of this writing, the updated schema has not been posted.

How it woks

This proposal assumes that there are 3 kinds of implementations using the IMS metadata.

Non-validating systems
Strict validating systems (non extensible)
Extensible validating systems

Non-validating systems:

A system that interprets IMS Metadata encoded as XML, including possible extensions, but does not attempt to validate, does the following:

Accept any well-formed XML file
Ignore any DTD reference in the XML file
Accept well-formed XML files that do not contain a DTD
Use documented semantics to interpret the data by name.

Such a system should be allowed where the cost of implementing DTD valication in either an XML Meta-data authoring tool or in a simple data reading tool may exceed the usefulness of validation.

Validating system with strict base compliance

Such a system does not allow any extensions. It uses only the current approved W3C approved XML specification and the generic tools available to parse data formatted according to that specification. Such a system must:

Use XML 1.0 DTD specification with reference to the currently published IMSMD10.DTD file.
Use a validating parser
Flag as error any unknown or missing data element that does not conform to the DTD.

Such a system is of limited usefulness, since it can be extended only by specifying an "ANY" branch that cannot be validated. However, it may be useful in testing systems for strict compliance.

Such a system may be extended also by using a reference to a "super-DTD" that is built for a specific domain. The construction of such a super-DTD for each domain or domain combination is currently an exercise for the imagination.

Validating system with extension support

A system that validates and support extensions does the following:

Read any specified DTD (there may be more than one). The DTD specified according to the XML 1.0 specification has the highest priority. Normally, this should be IMSMD10.DTD
From the DTDs, generate a data schema. This is a tree in which each node is a data element descriptor, including type information, identifier(s) of the DTD(s) in which the element is defined, obligation, etc.
From the data schema, generate a data tree.
Read the XML data into the data tree. Each node in this tree contains a reference to the data schema tree (for type and other information.
While reading XML data into the data tree, two things can happen, depending on system implementation and/or settings that specify whether data is filtered according to the schema, or not.
If filtering data according to the schema, any data element that is not specified in the schema tree is ignored.
If not filtering data, any data element that is not specified in the schema tree causes the addition of a node in the schema tree the first time such a data element is encountered. The path name of the element is used to determine where the element fits in the schema tree. Because the element is "unknown", the DTD reference in the schema tree will be null. An element may have attributes. For example:
<mything aType="Numeric" Obligation="true">

Note that what a system does with unknown data elements may vary. For example, a meta-data editor tool may choose to preserve unknown data elements in case it needs to re-export the data, while others would just discard them. A meta-data editor tool may also filter which data it shows, depending on which DTD specifies the data. For example, a Math Exercise metadata editor might choose to ignore any meta-data not defined in either the Base DTD or a "Math_Education" extension DTD.

Example

This example uses a snippet of a sample xml file:

Base schema	"Woven in" extension data element
… <Title> <Locale> en_US </Locale> <String> Extending Metadata <String> </Title> …	… <Title> <Locale> en_US </Locale> <String> Extending Metadata <String> <Abbreviation> Ext-MD </Abbreviation> </Title> …

Advantages of this proposal:

Allows easy, well-integrated extension of the base IMS metadata

Allows validation against one or more DTDs.

Very simple to read and write. For example, a search engine that only searches known meta-data elements can use very simple parsers and ignore the DTD altogether.

Allows powerful implementations of rigorous validity checking and filtering.

Does not preclude migration of the data into more capable future standards based on XML or any other standard

Not tied to a current proprietary XML extension standard

Data created will be easily retrievable for use when better meta-data standard storage and encoding formats emerge

Does not depend on any proprietary or patented tools or algorithms.

Disadvantages of this proposal:

May require deleting the DTD identification from an XML file before passing the XML data to a parser that cannot parse "invalid" data.

Extensions if they define new data types or abstract types may not be interpretable without documentation.

Validation requires code that can parse a DTD, because the mechanism to reference more than one DTD is not defined by XML1.0 or implemented by current freeware parsers.