Blogger: Joseph M. Bugajski
Six years ago, I sat in a meeting of business managers who had gathered in a large conference room at Visa's HQ. The participants were experts at handling electronic payment problems experienced by banks and merchants in their part of the world. These managers were an odd lot of business managers. They actually understood arcane things about payment message data. Indeed, they defined the meaning of payment message data in Visa's payment training manuals and technical specifications. I was their guest at that this meeting. They invited me to talk about preserving the semantic interoperability of message elements across Visa's global network. Before considering the reason for my visit to this particular meeting, it is important to understand something about electronic payments.
Visa's network (Visa Inc. plus Visa Europe) is vast and maintaining data interoperability is challenging. Visa's retail payment network processes over USD $5 trillion annually. It connects 1.2+ billion cards with 20+ million merchants, 22 thousand banks, and a large amount of payments made by other card schemes. (See Visa Inc. publications and Visa Europe facts for more information.) MasterCard and American Express also have large networks, albeit not as large as Visa's. To understand payment message data, one must first understand payment message processing. A retail payment comes to life as an authorization message inside a point-of-sale terminal when the payment card (e.g., a credit card or a debit card) meets the electronic point-of-sale terminal via the "mag-swipe" or "chip-dip" method, or when a cardholder types card information into a web-form on an e-commerce site. The retail payment message contains information about the card, cardholder, payment, environment in which the payment occurred, merchant, banks, and payment network. The message then travels from the terminal to the retailer's transaction system, to the acquirer's processor and acquiring bank, the card payment network, the card issuer's processor, the issuing bank, then back and forth a few times as different parts of the transaction complete, and eventually information reaches the cardholder. (See http://www.corporate.visa.com/md/in/in_transactions.jsp) for more information about payment processing.) While a message traverses the many processing points of the payment network, that message is transformed, repackaged, added to, amended by, logged, warehoused, encrypted, decrypted, copied, recalculated, and forwarded. One challenge for network participants is to ensure payment data is correct so the payment completes quickly. If it completes quickly and for the right reasons (e.g., it is legitimate and not fraudulent) then network participants happily make some money and cardholders get their goods or services. If the payment fails, participants are unhappy because no one gets paid and cardholders tend to feel embarrassed by declines. But, keeping data accurate and payments flowing is not easy. Not only are there tens of thousands of people involved, there are thousands of message construction rules, hundreds of bilateral agreements between financial institutions, a hundred national jurisdictions, and as many unique laws about payments, all of which go into the design of the payment message.
Like the rest of the global financial industry, Visa's (and MasterCard's, American Express', JCB's, and others') payment message formats and data coding technology dates to the 1970's and 1980's (a "bit mapped" message format derived from ISO 8583, in particular the version published in 1987). This means that new card electronic payment products (e.g., commercial cards and up-market cards like platinum, signature, and black), and new payment services (e.g., mobile, e-commerce, and recurring payments) rely on an ancient and tired message infrastructure. The information required to operate these new products and services is simply overlaid on existing data fields, or more often, combinations of several data fields. Reuse (aka overloading) of data fields in existing message data structures (illustrated in figure 1) has the obvious drawback (at least, this is obvious to data architects) of making payment messages more convoluted and confusing for developers and business persons. The less obvious benefit (i.e., less obvious to data architects but completely obvious to business executives), and the reason that data field reuse is commonly used by the financial industry, is that reuse permits new products and services to be put on the market faster at lower initial cost. Naturally, the total cost of ownership (TCO) associated with reuse grows exponentially more expensive with time and the addition of new produces and new services. Unfortunately, complications inherent in legacy financial messages pervade information technology (IT) infrastructure thereby engendering higher maintenance costs, error propagation, and lost revenues. These costs include added investment in transaction processing systems, payment gateways, data architecture, SOA (even if deceased), message and data integration middleware, risk management systems, data warehousing, business intelligence components, e-commerce sites, customer relationship management (CRM) systems, enterprise resource planning (ERP) and accounting systems, call centers support systems, front office systems, and other financial networks. Data errors add costs by making fraud more difficult to detect (see Marcus Collins discussion of fraud management in his blog about variations on the BI model, "Realizing the value of the BI initiative - analysis when the business needs it"), inappropriately truncated transactions more common, and market analyses nearly impossible. Faulty financial messages and good financial messages both become integrated into infrastructure. All of which helps to explain why the Visa business managers at the meeting I attended several years ago did what they did. I sat in that meeting six years ago for several hours before it was my turn to present. I was happy that I sat there because it gave me the opening line of my presentation. During those several hours, and for many hours afterward, on this and other occasions, business managers, experts in payment data, argued about the meaning of just one or two codes in just one data field, in just one type of payment message. The opening line of my presentation was, "If Visa's business experts cannot agree about the meaning of one code in a payment message, how can we imagined that a developer thousands of miles away, working for a merchant or a bank, with no little knowledge of the payment business, can know how to enter the correct data into a message?" To be fair to my former employer and my esteemed former colleagues at Visa, I heard the same discussions repeated by similarly knowledgeable managers from all parts of the financial industry at standards meetings and at private meetings with other experts. The same problem with unclear semantics exists for securities trading, wholesale payments, property and casualty insurance, and health insurance claims. The problem all these managers had with message data semantics stemmed not from their lack of knowledge about the business but from bad data architecture.
Faults in data architecture cost the global economy at least one trillion dollars annual growth because of friction added into financial networks by semantically incoherent financial messages. In short, the problem is defined as the failure of message data architecture to convey the correct semantic information from one place to another within and among financial institutions and trading partners. The friction induced thereby is endemic. Business decisions that fail to consider total costs and risks associated with data field reuse far exceed localized savings. Fortunately, there are solutions being developed to correct these deficiencies.
XML for financial messages, instead of legacy formats like bit maps, is the preferred solution of business managers who develop financial industry standards. (This is the reason that SWIFT MX, FIXML, IFX, and others adopted XML formats.) The good thing about XML is its flexibility. Systems do not fail whenever new data fields are added to messages at only one point of a transaction. Contrast this with legacy messages that require lock-step updates by all network participants whenever new data fields are added. XML also helps with message semantics by reducing the depth and extent of message semantics convolution. XML does not however substitute for achieving a common understanding of the meaning of a data element within a financial message - the semantics. For this reason, the global financial community banded together under the umbrella of the International Organization for Standardization (ISO) to create a common data dictionary, ISO 20022. SWIFT, who maintain the ISO 20022 dictionary, learned the hard way that XML is not a good way to create conceptual data models. Of course, those of us who read Joe Maguire's (Burton Group, Data Management Strategies) papers know this is a fact (see the Burton Group's, Data Management Strategies overview, "Data Modeling: A Necessary and Rewarding Aspect of Data Management".) They are rebuilding the ISO 20022 central dictionary to assure that data fields are continuously added or accidentally redefined.
ISO 20022, and UN/CEFACT's related work underway, UN/CEFACT Modeling Methodology (UMM) and Core Components, requires the creation of a conceptual data model defined using the Object Management Group's (OMG) Unified Modeling Language (UML) artifacts, and definitions of business terms clearly written using clear business English. ISO 20022 (and UMM) represents a grand experiment in striving to achieve semantic interoperability across the global financial network; i.e., the work is limited to particular business activities within particular segments of a particular industry - finance. Unfortunately, this goal is poorly understood even within the financial industry where business managers erroneously associate ISO 20022 with XML messages in the ISO 20022 repository. Indeed, even ISO representatives make this mistake. Nonetheless, ISO 20022 is about building conceptual models of financial information that must be coded into financial messages. Those models are expressed as UML. The UML is transformed into XML schema (logical models) via a proprietary workstation at SWIFT (experiments using OMG's XML Metadata Interchange [XMI] standard for this purpose were unsuccessful because XMI does not provide interoperability for M1 models; it is limited to exchange of metamodels [M2 models]). The logical models are then coded into specific wire-line formats.
To clarify this point, and to open the ISO 20022 to legacy specifications for retail payments and securities trades, the ISO group that governs 20022, the Repository Management Group (RMG), a subcommittee of ISO Technical Committee 68 (TC68 - financial standards) met in Helsinki Finland last year and agreed that XML messages were not the only messages that should be coded for ISO 20022. Legacy payment messages like ISO 8583 also should be added to the standard.
Whether or not standards like ISO 20022 are successful will depend in large measure on the participation of technology suppliers to the financial industry who until now have remained conspicuous by their absence from the financial message standards world. Perhaps this is because they want to sell the industry more hardware and software to support the present cacophony of discordant messages that run through all global financial networks, or perhaps because they want to sell their proprietary industry data models. Whatever the reason, their absence from this standards work is shortsighted and unconscionable.
Another important experiment in establishing semantically rigorously framework for message interoperability is the OMG's Model Driven Message Interoperability (MDMI) beta specification. OMG has organized a consortium of financial industry firms, including a few important suppliers and other standards organizations, who are working together to test the MDMI specification and to improve the design of the central dictionary concepts expressed by ISO 11179 and ISO 20022. MDMI is designed to provide
Contrast these very much narrower projects with those being undertaken by the Worldwide Web Consortium's (W3C) Semantic Web project. This scheme desires to make the entire web semantically interlinked and thereby, understandable to machines. After spending 10 years working to resolve semantic interoperability issues for just one small part of the financial industry - retail electronic payments - and finding much of the problem intractable, this author questions the wisdom - nay, the audacity - of investing finite engineering and business resources to achieve such a grand vision; i.e., a semantically searchable web. Developers of RDF, SPARQL, and OWL are making excellent contributions to the design of semantically consistent messages and documents. It should be clear that those contributions should be applied in much narrower problem domains than "everything" on the web.
Regardless of whether one standard or another succeeds in the marketplace, semantic interoperability remains a central challenge for message interoperability in networks used by industries and governments. Its study remains an important academic research area. While networks have made the world smaller, semantically vague messages have sent us flying apart. In particular, economic growth cannot long be sustained on a foundation of legacy message technology, and XML is not the panacea for all things message-like. New technologies like ISO 20022, MDMI, Core Components, and yes, RDF and OWL may someday help us communicate globally and well during the 21st century and beyond.
Comments