Data Management Strategies

Future of the DBMS, considering Hadoop, NoSQL, and XQuery

Blogger: Lyn Robison

In Burton Group’s recent “2010 Planning Guide: Data Management Strategies” paper, I said, “the data management foundation that [relational] DBMSs provide is not adequate for all of the needs of modern enterprises.” Clearly, I believe the era in which enterprises use relational database servers to store all of their data is nearing an end.

Today I listened to a briefing from a company that is betting its future on “internet-scale data processing” which Hadoop offers. I imagine that this company will do fine, but it appears to me that Hadoop and NoSQL databases will not supplant relational database servers in enterprises until there are clear standards for client data access and processing. The widespread adoption of Structured Query Language is largely what made relational databases so popular. SQL made it possible for clients (both humans and software applications) to easily process and retrieve data in relational data stores. Until we see the widespread adoption of some programming metaphor or language for client data access and data processing in the NoSQL world, NoSQL databases are likely to remain niche products.

Does this mean that enterprises are stuck forever in the relational-only world of DBMSs? Not necessarily, because in the NoSQL world there is no equivalent to SQL … except, of course, XQuery.

XQuery is poised to do for XML databases what SQL did for relational databases. Using XQuery, clients (both humans and software applications) can easily process and retrieve data in XML data stores. This means that developers can use XQuery with their favorite object-oriented programming language to build XML-based applications. Application developers now have a choice for their data storage needs that enables them to select the meta-model that best fits their data. The relational meta-model is best for tabular, structured data, while the XML meta-model is best for document-centric, semi-structured data.

Like I said, developers can use object-oriented programming languages with XQuery to build XML-based applications –- that is, if they want to build those XML-based applications the wrong way and write a bunch of unnecessary software. In one vital sense, XML data is fundamentally more powerful than relational data: XML data does not need to be instantiated as objects in OO code in order to be processed and presented to users. Instead, XML data can be decorated and enriched with tags, and these tags can be processed and understood by software that enterprise IT groups do not have to write. If an enterprise’s data is in XML, lots of off-the-shelf, well-tested, well-proven, enterprise-ready software packages can process, display, and edit that data.

The software applications that can understand XML tags include Microsoft Word, Excel, InfoPath, as well as the Open Office productivity applications. IOW, enterprises can now use Office apps, including Word, Excel, and InfoPath for the user interface for presenting and editing enterprise data, provided that data is in XML -- more specifically, provided that data is in the native XML file format of one of the Office applications. But how do you get enterprise data into the native XML file format of Word, Excel, or InfoPath? You use an XML database such as MarkLogic Server that understands and that can transform enterprise data into and out of those Office XML file formats.

The vendors of relational database servers have not yet made their DBMSs capable of transforming enterprise data into and out of those Office XML file formats, which shows that they are somewhat out of touch with power of XML-based enterprise data. At least the relational database vendors have been smart enough to make their DBMSs expose relational data as XML, which means that application developers can choose the right meta-model (relational for tabular data and XML for semi-structured data) and then can treat all of that data as if it is XML. By treating all data as if it is XML, developers can finally say goodbye to the ever-troublesome object-relational impedance mismatch.

These are radical ideas that can take solution delivery in enterprise IT to new heights of effectiveness. These ideas will no doubt be viewed as heresy in certain circles. I have published my heretical ideas in a document entitled “The Methodology for Overcoming Data Silos (MODS): Using the New XQuery Development Stack”. In this document, I specify an application development stack that is built around XML- and XQuery-based development tools. Give it a read and let me know what you think.

Fix What Ails Your IT Group: Focus on Information

Blogger: Lyn Robison

Something is not right in enterprise IT. Corporations spend piles of money on enterprise IT, but business leaders do not seem satisfied with the work of their IT groups. There is a simple antidote. A focus on information will fix what ails enterprise IT. Businesses pay for IT systems not because they want technology, but because they want information. If enterprise IT groups would focus on delivering information (instead of just delivering systems and technology), the business would at last be pleased with IT’s work.

Sound too simple? Let’s examine the alternatives.

Without a focus on information, IT’s work on services and architecture turns into activities that benefit IT instead of the business.

Without a focus on information, IT’s business process automation efforts turn into rigid silos that often don’t fit the business processes.

Without a focus on information, the BI systems that IT builds turn into Business Stupidity (BS) that cannot answer businesspeople’s questions.

Without a focus on information, IT agility turns into yet more useless IT systems delivered faster and cheaper.

Without a focus on information, IT’s focus on business outcomes heightens the risk of unemployment for IT people.

The simple fact is that if an IT group does not focus on delivering useful information to businesspeople, nothing else that the IT group does will work.

The place to start is IT metrics. The first step in focusing on information is to establish information quality metrics as the primary metrics by which an IT department measures its work. For guidance on information quality metrics, click here.

One Way for IT to Satisfy the Business

Blogger: Lyn Robison

Imagine a businessperson saying, “I want to see all of the information that exists (and that I am authorized to see) in all of our company’s systems about this particular customer. That includes transactions, contracts, correspondence, payments, shipments, complaints, history, everything.”

As an enterprise IT person, what would be your reaction?

1. “That’s impossible. I mean, good luck. You are naïve to think that diverse information can be meaningfully integrated from that many disparate information systems. It would take forever to hard code the logic to translate all of that information from its original context into the context that you are requesting. ”

2. “That’s not needed. No one ever asks for that. There are no business scenarios where finding all of the existent information on a particular topic would ever be needed. And besides, the purpose of IT systems is to automate and optimize business processes. IT systems do not ever need to provide impromptu access to information.”

3. “That’s what data warehouses and BI apps are for. We’ll just create a data warehouse and BI application for you. Of course, that will only give you access to the structured data. And, if you want to ask a follow-up question on related topic, we will probably have to create yet another data warehouse and BI application for you. BTW, how much money can you put into this and how long can you wait?”

4. “Can do. You can see what every system in the enterprise has on each customer by visiting our MODS registry with your browser or favorite Office application. The source system for each piece of data will be clearly shown so that you can see the context that the data is coming from. The contact info for the business data stewards is included with each piece of information, so you can get answers to any questions you have about the meaning of the data from each system.”

Now imagine that the businessperson says, “How reliable is this information?” What would be your reaction?

1. “I am sure it’s good enough. It’s probably not 100% reliable, but it’s better than nothing, right?”

2. “Couldn’t tell ya. The data might be wrong, but that’s not really my problem. I only deliver the technology and manage the systems.”

3. “It’s not black and white. The data is not reliable or unreliable. It’s somewhere between completely erroneous and totally true.”

4. “Included with the information from the MODS registry is a link to the information quality metrics for the data from each system. This will give you measures of the reliability of the data from the businesspeople who know it best, and will also give you the lineage of the data from each system.”

What answers would the businessperson prefer to hear from you? Based on my own experience, the answer 4s are going to be the preferred answers.

Many may think that the answer 4s would be impossible, or impossibly expensive to deliver. I respectfully disagree. The technology to do this has existed for several years. The data is already lying around the enterprise. MODS projects are inexpensive. Information quality metrics are easy to institute. And data stewardship and governance are not onerous to implement, and with the demands of regulation and compliance, are going to be essential anyway. Do those things and you will be able to give answer 4 to both questions, which are answers that the businesspeople will find quite satisfactory.

It is interesting to note that these “find all the info on customer X” questions get asked and answered every day. This is why CEOs and EVPs have executive assistants (otherwise known as poor go-fers). The CEO says, “I have a big meeting with customer X in 3 days and I need to be armed with all our interactions with them.” With that, the go-fer whips open the BI dashboard, does a query for the systems he or she can get at, calls Operations and has them do a scan of archived e-mails from the past 6 months, reads and summarizes those, logs on to every search system the company has, and does a query for customer X. After three 12-hour days, the go-fer hands the CEO a printed out Word document with everything neatly formatted. So it isn’t so much that the question can’t be answered, it’s that the question could be answered in a much faster and higher quality way if the IT systems were changed.

In short, assuming a company has the labor to hand-stitch the information together, I’m not sure high level management recognizes this as a problem. Their questions get asked and answered. The issue is that only CEOs and EVPs can ask these questions; everyone else is out of luck. A MODS registry makes the information readily avialable to everyone who is authorized to access it.

Ultimately, the MODS registry could be made transparent to businesspeople. The businessperson could use a simple query or search tool to submit a request for complete customer information. That request could automatically route (e.g., via a data services platform) to the MODS registry server. The MODS registry system could resolve the query or search by accessing registry data and business logic. The logic would resolve duplicative, inconsistent, and conflicting customer information from multiple sources. The resultant solution to the query (i.e., a complete view of a customer) could appear in a window on the businessperson’s access device regardless of its location (i.e., assuming appropriate access and security controls). You can learn more about MODS by clicking here.

Building the 2010 Plan

Burton Group's teams are currently formulating research plans for 2010.  Let us know how Burton Group should focus our efforts to provide you with effective decision making tools, perspectives, and insights. 

What initiative or project will be a critical part of your success next year?  How can Burton Group improve our coverage and our research deliverable formats?

Suggestions and comments can be sent to chaddad@burtongroup.com.   Thanks for your feedback.

Enterprise Trends driving Data Management

Blogger: Lyn Robison

Three data management trends seem to be driving enterprise data management to higher levels of attention and importance. Those trends are:

1. Regulatory and compliance requirements
2. Enterprises’ desire to do more with the systems they already have
3. Information management as a differentiator.

1. Regulatory and compliance requirements are forcing enterprises to exercise a modicum of competence in data management. The traditional view is that applications own their data. This view results in fragmented, duplicated, balkanized data. When IT people concentrate on managing applications and letting those applications manage their data, they are not performing competent data management. Enterprises face regulatory requirements to manage their data as a whole, and thus the discipline of data management is becoming important.

2. During this economic recession, enterprises are being forced to make do with what they have. There is not a lot of money for new information systems and IT infrastructure. Companies are generally trying to spend less on IT, not more. Smart enterprises are realizing that the best way to get more out of information systems is to do a better job of managing the data inside those systems so that those systems produce better information. This realization is elevating the status of data management.

3. Enterprises are finding that information management is a differentiator. Every large enterprise has an IT department and a bunch of systems. Having an IT department and IT systems does not differentiate an enterprise, but managing the information inside those systems does. Any enterprise that does a better job of managing its information than its competitors do will differentiate itself in the marketplace. This fact will make data management all the more important during this economic recession.

2010 Planning Guide, Data Management Strategies

Blogger: Lyn Robison

I recently recorded a presentation about our 2010 Planning Guide for Data Management Strategies. The Planning Guide and the presentation about it are intended to assist IT leadership in setting priorities in enterprise IT planning for the coming year. The entire Planning Guide is published (here) on the Burton Group web site for Burton Group clients. The presentation outlines the enterprise IT planning considerations for data management in 2010. You can view the presentation below:

To see how trends are driving enterprise IT groups to adopt a new model for their work, see slide 3 of the presentation.

To learn about the importance of data management in a cloud computing environment, you can go to slides 7, 11, and 12.

To get a fresh perspective on data silos, jump to slide 9.

To gain insight into the pattern of diminishing returns that enterprise IT is delivering to the business, go to slide 10.

To hear a Burton Group analyst say that Information Lifecycle Management (ILM) tools are “frankly, garbage”, even as the need for ILM grows to near crisis proportions, skip to slide 14.

To learn why Moore’s Law won’t save us, jump to slide 17.

To encounter the unvarnished truth about Business Intelligence (“Gee, it’s not working for some reason…”), skip to slide 18.

To listen to me hammer the big three DBMS vendors for neglecting to provide a platform on which enterprises can actually manage all of their information , skip to slides 19 - 23.

And to hear three things that an enterprise IT department can do in 2010 that are basically free and that will generate hard-dollar benefits for the business, jump to slides 24 and 25.

Here are the links to the resources that I point to in the presentation:

• A Data Management Freebie
• Foundations for Data Governance
• IT Metrics: Measuring IT’s Business Value
• Information Lifecycle Management and Data Management: The Chicken or the Egg?
• The Data Management Organization: Key to Effective Data Management
• Master Data Management (MDM): A Pivotal Process
• The Methodology for Overcoming Data Silos (MODS): Using the New XQuery Development Stack
• Delivering Integrated Information from Data Silos Using MODS