[ OVERVIEW ]          [ STRUCTURE/PARTICIPANTS ]          [ MDDL DOCUMENTS ]          [ LINKS ]
[ DISCUSSION ]             [ JOIN MAILING LIST ]             [ MEETING SCHEDULE ]          [ NEWS ]
 

MDDL User Application Apr. 2 Meeting

April 11, 2001


Organizations Present

Associated Press, Bank of New York, Bear Stearns, Fidelity Investments, FISD, Goldman Sachs, JP Morgan Chase, Lehman Brothers, Merrill Lynch, Morgan Stanley Dean Witter

Meeting Objective

The primary purpose of the meeting was to create a functional user application requirement for MDDL. The function of the user requirement will be to ensure that MDDL is meeting the core business objectives of the user firms.

Action Points

1) Prioritize the vocabulary fields. The user firms will review the working domain structure(s) for prioritization and rationale (i.e. organization, groupings) as it relates to usage. Strong support for a standard definition of and a relationship diagram among fields. Rich Robinson (BONY) is the champion.

2) Definition of functionality with examples. This is the key objective. Alex Kogon (Bear Stearns) and Mark Rayman (Merrill) are the champions. There are two "buckets"

  • a) data warehousing -- raw feed to populate databases (define what's being provided by vendors) and
  • b) Client server applications -- how applications will request the data (define how to get the data in/out of applications).

3) Definition of a standard request/response. Will address communication message format issues as well as the two types of requests (user pull and vendor push). Myank Gupta (Morgan Stanley) is the champion.

Functionality Requirement

There is broad consensus among FISD members that user application requirements are the key business drivers of the MDDL initiative. User firms distribute a significant amount of pre-defined market data to a wide variety of database applications. As such they spend time and money translating market data formats and modifying applications for internal communication.

The goal is this activity is to enable users to integrate data from multiple sources by standardizing both the input feeds used for data warehousing (e.g. define what's being provided by vendors) and the output methods by which client applications request the data (e.g. ensure compatibility on how to get data in and out of applications). The basis of the user requirement is for MDDL to:

1. Support multiple vendors

2. Common vendor interface (identical query and delivery protocol to all vendors)

3. Common request format for different vendors with standard request types and field names

4. Standard snap request by vendor specific instrument names or by vendor independent ISIN, exchange code, instrument type (or any combination)

5. Dynamic (user pull) snapshot data elements to be included in request/response

6. Allow requests for vendor specific fields

7. Allow requests for vendor specific services

8. Each request must contain user, application and host information. Each request to a service must be made with user information to allow entitlement checking or entitlement checking must be done by our application

9. Have receiver respond to server with ID# to confirm receipt, redeliver if never confirmed

10. Tag records with sequential IDs to facilitate partitioning of data stream and delivery guarantees

11. For all standardized requests, response must use standard field names

12. Allow receiver to report erroneous records to server for validation and data quality

13. Provide standards for how fields such as IDs should be used

14. Provide standard for common request/response interface to ensure that with proper implementation of MDDL, applications are ensured plug-and-play compatibility between back-ends

Request/Response Format

An ongoing discussion on request/response formats and the importance of including a query protocol as part of MDDL 1.0 emerged as a result of this meeting. The chain of e-mail messages is being posted to group.yahoo.com. In the meantime, here's a copy of the e-mail discussions

The initial attempt at documenting request/response capabilities included: session/request identifiers, user/application identifiers, multiple identifiers and types per request, pattern matching, aggregate identifiers (i.e. exchange or options chains, index chains, etc.), standard field sets versus or variable field lists, multi-part requests/responses, and the ability to match the request input with the response.

---------------------------

(FROM MIKE BENVENISTE, FIDELITY) I've had some time to think over Monday's discussions, and I'd like to give my reasons why we should not make a query protocol part of the MDDL 1.0 specifications

1. We can provide a useful and viable standard without it.
My criterion for this is pretty simple; can my team create and deploy better applications more efficiently if we receive data in an MDDL format even if the query format is application specific? My first four potential applications for MDDL involve a scheduled feed, a wireless application, a direct user request for a particular security, and a portfolio-based request. The first two applications are both "push" applications with decoupled queries and responses. The third app will require a servlet in any case, so there is a clear integration point for query formation. In the fourth case, a standardized request format could eliminate a conversion tier, which is a significant gain. Even in this fourth case, the gains from standardization of data are still quite large. Note that my requirement is not for a perfect standard, but a viable one.

2. "Time to Market" is critical for management buy-in.
Quite simply, we have enough open issues on how to represent data. Since the format of "downstream" data is independent of the query, deferring this issue will not create any backwards compatibility issues within the spec. By deferring the issue, we reach our 1.0 milestone faster. I feel the gains from such an "early victory" outweigh the technical advantages at this time. For proof of concept, we will create and hopefully deploy MDDL-based solutions based on current data stores and vendor feeds. I'm prepared to write and support inward looking filters to do so only if they are fairly simple. If the filters have to comply with a generalized query format on Day 1, it will significantly add to the complexity of such initial projects and the time of completion.

3. Integrity of the Specification
Are we prepared to say that in order to call a service or application MDDL-compliant, it must use the MDDL query format? If not, how do we answer that traditional management buy-in question: "what is MDDL, anyway?"

4. MDDL's need for a standardized query format is not industry specific.
W3 started work on a query language in 1998, and the issues raised in that workshop are the same ones we're examining. See
http://www.w3.org/TandS/QL/QL98/pp.html for some examples. The result of that effort is Xquery, http://www.w3.org/XML/Query. While Xquery is still in draft form, it would fulfill the needs that Mayank laid out with the exception of transactional or session identification. For anyone attending XMLDevCon next week, there is a session on Xquery: <http://www.xmldevcon2001.com/NY/html/session.php?code=S5> Using a non-industry specific standard increases the likelihood of being able to leverage commercial tools. For example, Tamino, an XML native database from Software AG, already implements a subset of Xquery. While dusting off my 'lex' and 'yacc' skills might be fun, one of the major goals of XML is to eliminate the need to write custom parsers. Let's not reintroduce that need.

5. We have a bit of a chicken and egg problem.
MDDL is likely to have higher network bandwidth requirements and CPU processing requirements than traditional vendor protocols. The engineering and architectual tradeoffs in determining how and where to deploy MDDL in the enterprise are unsolved problems. Until we get some pilot programs in the field, we probably won't know enough to solve them. Many of these problems fall into query formation. For example, Mayank's proposal raises the issue of transactional information in the query, and, perforce, the response. Whether this should be part of the query or managed through an independent session manager depends on the types of systems and communication protocols involved. A related example involves the concept of field sets. These could be predefined or transactional. If predefined, is it necessary to standardize? If so, do we do so as part of the spec, or via URL/URN's? If transactional, we end up requiring all MDDL service providers to keep state information per user. This latter requirement has significant implications for high-availablity and large
load-sharing applications.

In summary, I don't want to sweep these issues under the rug forever, just long enough to "declare victory" on 1.0 and to understand what we're going to do with this thing called MDDL. All comments, revisions, and suggestions welcome.

-------------------------------

(FROM TONY ZHANG, FINPORTFOLIO) Mike raised some very good points regarding the scope of MDDL 1.0 spec. I have also took some time to think over the issues related to a messaging protocol in MDDL. I think it is an important decision for MDDL and we probably want to sort it out as quickly as possible. I agree with Mike's points and my inclination is not to include it in MDDL 1.0, but to keep the option open for future MDDL revisions. In addition to Mike's comments, I would like to throw in a few more points to stir up the discussion:

1. MDDL should be a layering standard where its messaging layer can be relatively independent of its format layer. We understand the importance of having a messaging standard for MDDL, but we should take a progressive approach to build up the full MDDL spec (this seems to be the spirit of our approach so far) - we may declare MDDL 1.0 as the format standard and some future version as the format and messaging standard.

2. The complexity of mixing a messaging protocol with a format protocol may be a multiple of the aggregate. We can take a look at some of the existing messaging protocols, such as FIX (Financial Information Exchange Protocol, www.fixprotocol.org), OFX (Open Financial Exchange, www.ofx.net) and IFX (Interactive Financial Exchange, www.ifxforum.org). Many of the issues raised in MDDL, such as request/response, session and transaction, have been discussed and dealt with extensively. Unfortunately, these efforts produce some complicated specs, which are both time-consuming and difficult to implement. Given the timeframe we current have to wrap up MDDL 1.0, I would suspect we would like to go down that route.

3. As to coming up with a messaging protocol, I would recommend everyone to check out SOAP, opposed to a mixed-in messaging protocol like the one proposed in Mayank's email and some other protocols. SOAP (Simple Object Access Protocol, see <http://www.w3.org/TR/2000/NOTE-SOAP-20000508/>), developed by Microsoft and IBM, has been submitted to W3C as the base for XMLP (XML Protocol, see http://www.w3.org/TR/2001/WD-xmlp-reqs-20010319/, and has been adopted by OASIS as a messaging layer for ebXML (see http://www.ebxml.org/news/pr_20010222.htm) There have been many tools developed for SOAP, such as Apache/IBM SOAP 2.1, Sun Java XML-RPC, and Microsoft SOAP Toolkit 2.0, just to name a few.

4. As a general note, I think leveraging current standards and standard-based tools will greatly enhance MDDL's market position. As Mike pointed out, we probably don't want to reinvent the wheel for non-industry specific aspects of the MDDL spec. This will help us develop MDDL spec faster and reach broader market acceptance.

-----------------------------------

(FROM MYANK GUPTA (MORGAN STANLEY) I started Monday's meeting feeling there was no need for a request/response format...but changed my mind during the discussions. If snapping is in scope for version 1.0 request/response criterion have to be documented for reference. They will influence MDDL 1.0 content format decisions both in the vocabulary and technical committees.

Both proposed structures are somewhat arbitrary. This seemed to be a good way of documenting application requirements. The query structure is just two lists (idenfiers, fields). The response has a table like view(ie Identifer-field(s)). Many other alternatives could be easily substituted. I'll try to summarize the main points that Mike and Tony have raised. Please add/remove from this as necessary.

1. Messaging Protocol vs. Format Protocol
The proposed structure borrows heavily from the SOAP specification. It separates transactional/control elements from the "query" portion. Both of these elements are optional. The only condition placed is that if the query defines them...the server has to return them back(as they were in the query) with the response (needed for asynchronous processing). Parsers not interested in these sections (elements) would ignore them and/or would not error out if they were missing altogether. The request/response elements(with or without the tran/control elements) can be wrapped in any other document.

2. SOAP (see above).
The SnapRequestHeader corresponds to a SOAPHeader, The SnapRequest/SnapResponse sections correspond to a SOAPBody and support is included for Faults when and where needed. SOAP does not endorse any specific elements for dispatching, routing, user info and leaves this to individual DTD's/Schemas. I'm proposing that we at least be aware of, if not define some of these elements (eg. userId, application, data quality etc.)

3. Need for Query functionality (as per Mike's use case examples)
I agree with Mike and Tony that if MDDL 1.0 only defines the format layer we've made tremendous progress. However keeping the future in mind we need to design a robust Identifier element capable of supporting multiple identifiers, identifier types and pattern matching. A restriction I'm proposing is that the response from a snap include identifiers as they were specified in the request. Additionally in the response results (fields) be wrapped by the identifier they belong to. The proposal also creates a status hierarchy (document level status, identifier level status, field level status).

4. Compliance with MDDL
The proposal is an attempt to document application requirements. Since the documents are stand alone they can be parsed independently. Without further work on the format portion we cannot say what is a required for MDDL compliance.

5. Xquery
A cursory review of the XQuery spec leads me to believe this is more for querying/transforming doucments. We can view all of the vendor's content as one big document that can be queried against to produce the "filtered" document we're interested in. The proposed format and XQuery are not exclusive. All vendors would have to implement - field sets are/should be optional. Some vendors have them and it would be necessary to support them not necessary to require them from all vendors.

 

 

 

 

 

 

 



Copyright ©2001, The Software & Information Industry Association.
SIIA's Privacy Policy and Use Agreement.
All rights reserved.