Blogger: Lyn Robison
My nephew Dave is a major in the U.S. Army. Dave is risking life and limb serving in Iraq. I am intensely and justifiably proud of him. Everything that my nephew does in Iraq is difficult. The treatise “On War” rightly observes, “Everything in war is very simple, but the simplest thing is difficult. The difficulties accumulate and end by producing a kind of friction that is inconceivable unless one has experienced war.” The soldiers and marines in Iraq experience this “friction” daily. They address this friction with the phrase, “embrace the suck”. This phrase is indicative of their grim determination to succeed in a hostile environment where even the simplest tasks are difficult. “Embrace the suck” is both an implied order and wise advice. It means, “Face it, soldier. This ain't easy. Now let's deal with it.” With bravado and grit like that in the face of a daily and dangerous grind, you can see why I am proud of Dave.
Enterprise IT has its own kind of friction. In enterprise IT, a simple task, such as adding a data attribute to an information system, is difficult. The irony is that this friction is of our own making. Enterprise information systems do not have to suck. Instead of embracing the suck, we ought to make a deliberate effort to reject the suck of today’s enterprise information systems.
In two previous blog entries ("A Less-Stodgy Enterprise IT" and "Flexible Metadata for a Less-Stodgy Enterprise IT"), I talked about the stodginess of enterprise IT. Making even simple changes and improvements to enterprise IT systems is difficult. But it doesn’t have to be this way. To reject the suck and stodginess of enterprise information systems, you need just two things:
Front-end software that can understand new data and metadata without modification
Back-end software that that can help you create and recreate your metadata
The first thing you need is front-end software that can understand new data and metadata without modification. This is amazingly straightforward: web browsers and productivity applications such as Microsoft Office and Open Office perform this job very nicely.
When you add new data attributes to an enterprise information system, new data and metadata can be rendered simply as new sections in a Word document, new columns and rows in an Excel spreadsheet, new fields in an InfoPath form, and new hyperlinks in a browser. When you use web browsers and productivity applications as the UI for enterprise information systems, changing backend data and metadata requires few if any changes to the front-end software.
You might not have realized this, but Microsoft Word 2007 is a terrific XML editor. Try it. Click here to use Word 2007 to edit an XML file. (Word’s native file format is XML.) Actually, Word is more than an XML editor; it is a powerful UI into XML data. So is Excel. So is InfoPath. And web browsers are quite good at displaying XML data. When you have XML-savvy UI tools on the front-end of enterprise information systems, you are free to be flexible with XML data and metadata on the back-end.
The second thing you need to reject IT system stodginess is back-end software that that can help you create and recreate your metadata. This is also amazingly straightforward: XML databases perform this magic.
The data inside enterprise information systems is usually structured in a way that makes its metadata obvious. Metadata is usually defined via the native structure of the data itself (columns, rows, paragraphs, sections, etc.) However, it is vital to realize that metadata can also be defined through tags and annotations. If the data storage format supports it, existing data can be annotated with new tags. By using annotations, you can change the metadata of your data. In other words, you can use tags to add new metadata to your existing data.
Relational database technology is over 30 years old, and it shows. Relational databases do not readily enable annotations to existing data. The metadata in relational databases is defined exclusively through the native structure of the data – the columns and rows.
Unlike relational databases, XML is built around tags and annotations. XML data offers flexible metadata. XML is designed from the ground up to enable you to change the metadata. You are free to add and/or transform the annotations to XML data anytime you need to. A good XML database, such as MarkLogic Server, can automate the task of creating and recreating your metadata. In fact, a good XML database will do entity extraction, whereby you can tell the database to search through your existing data and insert new annotations around new entity types you define and want to find and process.
Relational databases can’t really do this. It is difficult to change the metadata of relational data. About the only thing you can do is change the structure and thereby the metadata of relational data (by using the DBMS’s DDL or by using ETL processes). And then you still have to change all of the software that sits in front of that relational data, because there are no nifty UI tools that render relational data the way that Office can render XML data. So information systems that have relational data on the backend tend to have lots of stodginess.
With XML, you can change the metadata on the backend without changing the software that implements the UI. Yes, you will likely be obligated to change the software that implements the business logic, but that is precisely where XQuery and keeping the data in XML all of the time makes things easier.
In a typical enterprise N-tier application, you have data in a relational database, with OO code (and probably some SQL stored procedures too) to implement the business logic, and more code of some kind to implement the UI. If you have adopted SOA, the app also has internal and external service interfaces. So a typical enterprise information system has a data model, an object model, a message model, and a UI model (which all must align with the business model). And then it has a bunch of code to deal with the impedance mismatches between all of those models. So, when you have to change the data model and its associated business logic, you are obligated to change lots of extraneous code to boot.
Now imagine an enterprise information system that has all of its data in XML and its business logic in XQuery. Adding a new data attribute is straightforward. Sure, you might have to change some XQuery code for the business logic, but the XQuery code is pure business logic, and is not encumbered by any of that extraneous code that translates the data between the various models in a traditional app. XML is used as the data model, the message model, and the UI model, and the data readily flows between the back-end and the front-end. I have defined a set of XML-based development tools that I call the "XQuery Development Stack". (The XQuery Dev Stack is ideally suited for MODS -- the Methodology for Overcoming Data Silos.) Information systems built with MODS and the XQuery Dev Stack are not stodgy. They are not stodgy because the metadata is easy to change and the data is not buried beneath a mountain of software.
You will notice that I did not mention implementing an object model in an XML-based information system. There should be no object model in an XML-based information system. Object-oriented code is largely responsible for the suck of enterprise information systems. Don’t embrace the suck of enterprise information systems, reject it. With OO development, the dose is the poison, and enterprise IT has overdosed on OO programming. My colleague Joe Maquire wrote about this in an earlier blog entry here. I will talk more about the poison that is OO development in a future blog entry.