Data Management Strategies

A Less-Stodgy Enterprise IT

Blogger: Lyn Robison

Enterprise IT isn’t as cool as the Web, and here are some reasons why. The Web allows information to flow, and the Web also allows people to collaborate and build relationships with one another. Enterprise IT generally doesn’t. Enterprise IT is stodgy because IT systems are most often implemented in nonflexible silos that were designed primarily to automate individual business processes. Silos in Enterprise IT are not designed to let information flow, and enterprise IT silos are most often not designed to let people collaborate and build relationships with one another. In short, enterprise IT is not cool because IT doesn’t let information flow or people collaborate.

Enterprise IT is obligated to automate business processes. We gotta do that. The business expects enterprise IT to automate their work. But business process automation should not be all that we do in enterprise IT. The business needs more from enterprise IT than process automation. In fact, if process automation is all that we do in enterprise IT, the business will think we are stodgy, because we are.

To find information about how to let people collaborate and build relationships with one another, see the CCS blog.

Letting information flow in enterprise IT requires two things:
1. Lowering the level of information model pre-agreement
2. Managing identifiers

The Web is a terrific platform for sharing information because the level of information model pre-agreement is low. A web client and a web server need only understand the HTTP protocol and a few basic document types. Typical data integration between enterprise IT systems requires a high degree of information model pre-agreement, and that makes data integration difficult, and that makes enterprise IT stodgy. What many IT professionals fail to realize is that when you are integrating data that is to be consumed by humans, you can use the Web model, and you don’t need to have this high degree of information model pre-agreement.

When identifiers are managed properly, enterprise IT gets less stodgy. For example, UPS and FedEx can easily track parcels, using tracking numbers. People with common names can be removed from terrorist “watch lists”, because of the RFID tags in their passports. Automobiles are easily distinguished using their license plate and/or their VIN. And airplanes can be readily identified using their tail number. When enterprise IT manages identifiers improperly, which sadly is quite common, the business can never get a complete view of their interactions with a particular customer, because that customer is identified differently in different systems. The business can never manage any long-lived and valuable assets through their useful life, because each long-lived asset is identified differently in different systems. And businesspeople cannot find the information they need on a wide variety of vital business topics, because information about those topics is buried deep within silos and is irretrievable because it is misidentified. And this makes enterprise IT stodgy.

Let's fix IT's stodginess by enabling people to collaborate and build relationships, by lowing the level of information model pre-agreement, and by managing identifiers properly.

COTS, SaaS, XML, and Pixie Dust

Blogger: Lyn Robison

Back in 2007, I wrote a Burton Group overview entitled, “Build, Buy, or Borrow: Choosing Custom Development Software, Open Source Software, Commercial Off-the-Shelf Software, or Software as a Service”. In this blog post, I will build on the ideas I presented in that paper and offer some refinements to my original recommendations.

These newly refined recommendations provide more detail and emphasis on COTS and SaaS, and add two new elements. These newly refined recommendations offer a path to an enterprise IT infrastructure and application portfolio that can be both radically simple and more effective than the burdensome patchwork of clunky legacy systems that you find in enterprises today. Here are my updated recommendations. Enterprises should use:

1. COTS software for the front-end
2. SaaS for the back-end
3. XML for the data
4. Pixie dust (in the form of identifiers and data services) to make information float from system to system

1. COTS for the Frontend

COTS software offers flexibility with low cost, which is readily apparent when user interfaces are implemented using COTS software packages on PCs and mobile devices. Certain COTS packages (namely browsers, media players, word processing programs, and spreadsheet software) can become low-cost, flexible user interfaces for enterprise systems. They can display all kinds of information to businesspeople, including narrative text, graphics, photographs, video, audio, structured or tabular data, charts, highly structured documents, electronic forms, etc. And these off-the-shelf UIs can go beyond simply displaying data -- they can let businesspeople edit data and submit it to backend systems for data processing and storage.

COTS packages such as browsers and productivity applications are low cost UIs because IT developers don’t have to write much, if any, UI code. These COTS packages also sport interfaces with which businesspeople are already familiar. Browsers and productivity applications run on multiple clients, including PCs, Macs, and lots of mobile devices. Clearly, if the data is HTML, OOXML, ODF, or some other readily-consumable format, browsers and productivity application software can make highly flexible, low-cost UIs for enterprise information systems.

For a demonstration of how well browsers can display various kinds of information, use your favorite browser to look at the World Wide Web. For demonstrations of how well COTS software can provide highly-flexible UIs for enterprise systems, see the templates here and here. In the SLA example, IT and businesspeople could use this SLA form to create a new SLA or update an existing SLA, which could then be saved to a backend database that stores OOXML data, and that could be queried by IT professionals whenever they need to know what service levels are expected of them. The Expense Report example is a template that could be customized and used as the UI for an expense system where employees could fill out the spreadsheet that they then save to a backend database that can validate OOXML data and process the expense reports.

2. Saas for the Backend

Saas offers the fastest time to value for generic business processes. SaaS is most appropriate for those business areas where the processes are standardized. I’ve seen lots of enterprises where businesspeople believe their processes are “different” and cannot be automated using standardized business software. In many cases, however, these processes are “different” because they are actually substandard. IOW, if businesspeople would adopt standardized business processes and the pre-built SaaS packages that can automate those standardized processes, they would significantly improve their business. In those cases where the data is too sensitive to trust offsite with a SaaS vendor, then the SaaS application often can be hosted within the enterprise’s data center. SaaS is a part of cloud computing, and enterprises should consider internal, private, and public options for cloud computing, as their security requirements dictate.

3. XML for the Data

When data is in XML, thousands of existing software packages can handle that data. Data in XML is universally approachable, accessible, and usable. Many IT professionals think of XML only as a format for transferring data between applications, which is a counter-productive perspective. More than just data transfer, XML is a powerful format for data storage. XML is a useful compromise -- XML may not be perfect for structured data or for unstructured data, but it is a useful compromise that can handle both. The Microsoft Office suite and the Open Office suite store their data (their documents) in XML, which clearly demonstrates that XML works great for narrative text storage as well as for structured data storage. As further evidence of how far XML can go, check out a demo that drew gasps at TED2010. The file format for Photosynth and Deep Zoom is XML-based.

4. Pixie Dust (in the form of identifiers and data services) to make Data float from System to System

This is best illustrated by a couple of examples. Since Gartner’s acquisition of Burton Group, we have been getting up to speed on Gartner’s internal systems. I have learned that I have at least three IDs with which I must log in to various Gartner IT systems. If Gartner ever needs to combine the data about me that resides within these various systems, they would have a tough time doing it. How do you do cross-system joins of important data when the identifiers are not consistent between the systems? It ain’t easy.

Here is another example. Within Burton Group, we have multiple systems that contain customer data. Two examples are our CRM system and our Catalyst registration system. If I want to find out the customers who are attending Catalyst and who have had recent dialogs with my team, I immediately encounter the problem of customer identifiers that don’t match between those two systems. I can try to use email addresses as identifiers -– which is about my only option, but that makes my task far from easy. If we used uniform identifiers throughout our disparate customer systems, I could use simple UI tools such as my browser or even Microsoft Excel to find the customers who are attending Catalyst and with whom we have had dialogs recently, and I could do it without building an elaborate data warehouse and an expensive set of BI applications.

We would need a data service or two to be written that would enable me to use COTS software to query the CRM system and the Catalyst registration system based on customer ID. Managing IDs and writing data services that work with COTS UIs is what MODS and the XQuery development stack are all about.

Compared to the World Wide Web, enterprise information systems are clunky and kludgey. If I want to get information from CNN, I don’t have to know any details about CNN’s computer systems. I just point my browser to http://www.cnn.com/ and I get ready access to the info I need. It is ironic that every enterprise has complete and ultimate control over its own information systems, and yet, enterprise information systems are clunky and kludgey. For example, to get information about Burton Group clients, I have to know the intimate details of our CRM system, our Catalyst registration system, and each of our other systems that might contain customer data. I cannot easily combine the data between these systems and I cannot use my browser or productivity applications with which I am already familiar to do it.

Enterprise information systems are clunky and kludgey, but they do not need to be. Enterprise information systems can and should be better than what you can find on the Web. The approach I am suggesting is to use COTS software on the front-end, use SaaS applications on the backend, use XML to store the data, and have IT developers focus on bringing data together for businesspeople, instead of having IT developers continually build more clunky applications, which only add to the growing crowd of aging legacy systems that imprison valuable data within silos and that seem to remain forever on life support in the data center.

Must IT solutions always be based on software?

Blogger: Lyn Robison

When businesspeople ask for an IT solution, the assumption is that they asking for a piece of software. When IT professionals gather the requirements for a new IT solution, they focus on gathering software requirements. I mean, IT professionals don’t start with electricity requirements; they start with the software requirements. Sure, there are process requirements and data requirements, but software will implement those processes and that data, so software requirements are the overarching concern. Clearly, in enterprise IT today, “requirements gathering” primarily involves gathering the requirements for a piece of software.

To design and build custom IT solutions, enterprise IT groups have user interface designers, software developers, and software testers. Software is the thing that the IT solution team implements in order to deliver the IT solutions that the businesspeople asked for.

Let’s not forget the computer hardware. The software requirements are what determine the hardware requirements. The software gets the hardware that it needs to satisfy its functional and nonfunctional requirements. Hardware requirements are determined by software requirements. (And the electricity requirements are then determined by the hardware requirements.)

Software does the data, software executes the processes, and software determines the hardware. It is apparent that today’s IT solutions are based on software. My question is: must they be?

Must IT solutions always be based on software? They didn’t use to be. Back in the 60’s and 70’s, IT solutions were based on hardware – mainframe hardware. You picked your mainframe, and the software and the data came along with it, almost as accessories. The focus of the solution was the mainframe hardware. Then with the advent of PCs, computer hardware became commoditized. With PCs, you could decide on your software, and then be confident that you could find whatever commodity hardware the software requires. IT solutions have been based on software ever since.

But here’s a question: what will happen when software becomes a commodity? What will IT solutions be based on then? Here’s my answer: information. When software becomes a commodity, IT solutions will be based on information requirements. When businesspeople ask for a new IT solution, IT solution teams will gather information requirements, and the information requirements will drive the software requirements, which will then drive the hardware requirements.

Some might think that software will never become a commodity. It has already. Here is a portion clipped from the Salesforce.com homepage.

It looks to me like the Salesforce folks are saying that software need no longer be the basis for IT solutions any more. Enterprises can get all of the software they need from the cloud. Software is a plentiful commodity. IT solutions can be based now on information, not on software. Gather your information requirements, then select your software, etc.

The only catch is the fact that your data is stuck within various data silos in the cloud. The real work is in figuring out how to migrate your data, integrate your data, protect your data, and manage your data. Data management has a bright future.

Can data be merely the servant of the software?

Blogger: Lyn Robison

Whenever a business needs to manage something, they usually need an information system or two to help them keep track of it. The IT professionals who implement these systems need to ask the businesspeople a fundamental question: do you need to know the quantity of this thing or the quality of this thing?

If the business is tracking 3/8 inch sheet metal screws, the answer is probably “quantity”. All sheet metal screws of the same grade, type, and class have the same attributes, so quantity is the primary considerations. When mere quantity is the consideration, you can get away with the data merely being the servant of the software. The IT people gather the software requirements, and off they go to build the system.

On the other hand, if the business is tracking customers, the answer is probably “quality”, meaning they want to track the qualities of each individual customer. Each customer is distinct and not interchangeable, and they each have unique attributes that must be tracked for business reasons. These attributes will need to be assembled from multiple information systems, which means that the data cannot be the servant of any one particular piece of software. So the IT people must gather information requirements, in place of or in addition to software requirements.

Building an information system that tracks quantity is very different from building an information system that tracks quality. If you are tracking quantity, you are no doubt tracking a fungible commodity, and your information can safely treat the data itself as a fungible commodity that can be easily aggregated. OTOH, if you are tracking quality, you must not treat the data as a fungible commodity. You must manage identities beyond the boundaries of any particular data silo.

Speaking of sheet metal screws, I was asked an interesting question by an astute reader of this blog. He asked, “let's use your 3/8 inch screws...the billions of 3/8 inch screws worldwide are certainly fungible, but if I put them in a blisterpack of twenty, give them a UPC and place them in inventory at a Home Hardware, are they non-fungible assets worth tracking, comparing, etc?”

My answer is, one blisterpack of screws is indeed the same as the one on the shelf next to it, but is different from the one in another store from a different supplier only if the screws are of a different grade, type, or class. For example, stainless steel 18-8 Phillips pan head sheet metal screws are all the same no matter the supplier, the retailer, or the packaging. Even then, all sheet metal screws are fungible (interchangeable) within their class. Likewise, one gallon of gasoline is the same as another, no matter what oil company it comes from. “Chevron with Techron” is an effort to differentiate the grade, type, or class. But one gallon of Chevron with Techron is the same as another gallon of Chevron with Techron, so Chevron gasoline is still fungible.

With fungible commodities, your IT systems only have to track the quantity within each grade, type, or class. With non-fungible assets, your IT systems must track identity, and must do so beyond the bounds of any particular application or data silo. This means that data cannot be the mere servant of a software package. With non-fungible assets, the software must bow to the needs of the data.