Data Management Strategies

DMS 2008 theme #1: DBMS redux, part 2: one size still fits plenty

(See this post for part 1)

Okay, time for a DBMS reality check.

First, IBM, Microsoft, and Oracle collectively control, by most estimates, around 85% of the commercial DBMS market (which was generally assumed to be ~$21B in 2007). Sun's $1B acquisition of MySQL AB earlier this year also suggests that perhaps there's still some mileage left in the DBMS market as we've known it for the last decade or two.

Second, DBMS and RDBMS (relational DBMS) are not 1:1; the latest DBMS products from IBM, Microsoft, and Oracle, in particular, manage multiple database models (e.g., relational, XML, files, streaming, and OLAP). And they can do so with much lower total administration/license/etc. costs, compared with the alternative of using "best-of-breed"/specialized products for each database model domain.

Some architectural diagrams of what this looks like, when the bits hit the disk:

(Wow, these vendors are so fiercely competitive, they even use the same general color schemes:)...)

XQuery, the subject of my next post in this series, is a game-changer that brings significant potential for the DBMS leaders to leapfrog specialized XML content/data management systems (and SQL certainly isn't going away, nor are proprietary DBMS programming languages such as Transact-SQL or PL/SQL).

With XQuery and full-fidelity XML database model management in the leading DBMSs, it will also be possible for enterprises to shift some of their document management/content management infrastructure to DBMSs (rather than continuing to rely on traditional, proprietary specialized alternatives). This means enterprises will be able to have fewer moving parts in their data management infrastructure, vastly simplified data/document (a.k.a. structured/unstructured information) integration/synchronization opportunities, and fewer product licenses to renew each year.

The specialized vendors will also continue to have considerable market opportunities for the immediate future, of course, as enterprises don't undertake DBMS upgrades lightly, and some edge cases will remain out of the scope of the leading DBMSs for a while (e.g., petabyte XML document management scenarios), but for the most part it's likely the next generation of market leaders in the enterprise DBMS market will look a lot like the current generation, perhaps with relative market share changes along the way.

To close by addressing the questions from part 1 of this post:

• It is not the end of the road for RDBMSs as we've known them for the last ~25 years; they're morphing to become multi-database model DBMSs, and they're likely to gain rather than lose momentum over the next few years.
• Hang on to those SQL books; you're going to need them. The relational database model is still the most robust and powerful general-purpose database model, and it has significant synergy with XML (as the leading document model).
• It would indeed be prudent to also invest some time and attention in learning more about XML (data and documents), along with XQuery (which will be relatively easy to learn, if you've already mastered SQL). Burton Group has a bunch of research content in these domains, incidentally...
• Specialized XML data management systems are gaining momentum, but don't bother exploring IMS, IDMS, or other pre-relational systems for career enhancement potential, and it is still not the case that "everything is an object", so don't expect OODBMSs to rise from their graves anytime soon.  Some object database systems are being re-treaded and repositioned as XML database systems, but you probably want to look very closely before betting your next app on XML systems that aren't fundamentally designed to support XQuery (as opposed to bolting XQuery support on top of an existing object database system).
• Don't worry about Larry Ellison's financial stability; he's probably going to be on that Forbes list for the foreseeable future...

The next DMS 2008 theme: XQuery

Blogger: Peter O'Kelly

DMS 2008 theme #1: DBMS redux, part 1: the end of the "one-size-fits-all" era?

Context-setting:

• If you haven't seen the post titled "A few things you should know about Burton Group and this blog", please skim it for context-setting, before reading further in this post
• This post is the first in a series I referenced in the initial post for this blog, when I mentioned data management-related topics the DMS team thinks will be most influential over the next few years.  The list is neither exhaustive nor rank-ordered, and we (the DMS team) will get around to the rest of the initial post series set -- on XQuery, data modeling, open source DBMS, and the simultaneous commoditization, democratization, and specialization of business intelligence -- over the next week or two.

So -- about database management systems (DBMSs).  Many people assume the evolution of DBMSs peaked about a dozen years ago, coinciding with the rush to web apps. Before then, DBMSs were often serving as the full application server stack, handling database, transaction, identity/authentication/access control, and even application logic needs. It was a pretty fun and lucrative time to be a database designer on, say Oracle Database or Sybase SQL Server, circa 1993.

Web app servers often displaced DBMSs at the center of application platform stack priorities, however, as organizations rushed to integrate disparate back-end systems (often running on multiple types of DBMSs) into web apps. DBMSs were often relegated to basic storage tier services, and application logic, integrity constraints, and even transactions sometimes migrated to the mid-tier.

In another important dimension, DBMS vendors have not always been the most lovable suppliers over the last couple decades, e.g., with unpopular (oligopoly-centric) pricing/licensing programs that helped to create mainstream market opportunities for open source DBMS vendors and projects.

The rapid growth of XML content/data management over the last decade has presented another challenge to business-as-usual in the DBMS market, as the XML database model cannot be completely accommodated in the relational database model ("shredding" XML documents into relational database tables, for instance, is an apt image, as it can entail metadata loss).

Even DBMS market pioneers such as Michael Stonebraker have been asserting, over the last couple years, that the "one-size-fits-all" DBMS era is history, and that specialized data management systems (for streaming data, columnar data domains such as analytics, and XML, for example) will increasingly displace traditional relational DBMSs.  A couple of Stonebraker's most recent companies, StreamBase Systems and Vertica, are leading examples of the latest wave of specialized data management vendors, as is XML content platform Mark Logic (there are compelling open source projects in these domains as well, e.g., the eXist-db open source XML database system).

Is it the end of the road for RDBMSs as we've known them for the last ~25 years?  Time to sell your SQL books on eBay, before they become worthless?  To sign up for a distance-learning XML management course, in order to invest in your future career potential?  Perhaps some of the "legacy" database models, such as hierarchical and object-oriented DBMSs, will finally start to gain (regain, for hierarchical) meaningful market momentum, with the market shift to XML for content and data?  Is Oracle CEO Larry Ellison at risk of being bumped off the short list of the richest people on the planet (he's already down to #14 on this year's Forbes "The World's Billionaires" list...)? 

(In the interest of keeping these posts to a reasonable length, I'll continue on this topic in a second post.)

Blogger: Peter O'Kelly