This post is the fifth in my “5 Keys” to EIM series, covering the core principles of data management. Please check out the others:
1 – Introduction to the “5 Keys” Series
2 – The “5 Keys” to Data Governance
3 – The “5 Keys” to Data Architecture Management
4 – The “5 Keys” to Data Development
This time I’m looking at database operations management. This is one of the more technical disciplines of data management, but I’ll try to keep it interesting. Let’s pull back the curtain on the mysterious world of
Oz database operations management!
Figure 1: 10 Data Management Disciplines (adapted from DAMA.org)
KEY 1: Your DBA is overworked, underappreciated, and is barely able to keep things functioning
I’ve performed database administrator (DBA) roles and known countless other DBAs. Without exception, DBAs routinely perform circus-caliber juggling acts to keep your company’s data safe and available. It fascinates me how programmers quickly turn into the same type of “I need it yesterday” people that they normally complain about. If IT is the black sheep of your company, then your DBA is the black sheep of IT! Seriously, buy them lunch or something. They work far harder than you probably think they do.
KEY 2: There are 2 kinds of high-quality DBAs: those that do data development, and those that go DEEEEP
My former DBA responsibilities always came from designing and building databases that I then had to support. These databases would be part of larger applications that were also my team’s responsibility. I performed the database operations necessary to keep the databases from breaking the applications, but my heart wasn’t really in the DBA world. This is your Type 1 DBA – born from necessity.
The Type 2 DBA is a much more elusive creature: it is someone who actually LIKES being a DBA! Signature characteristics include: energetically talking about “replication”, bringing up water-cooler conversations about “query execution paths”, and ranting about “enforcing key constraints”. Type 2 DBAs can get so into the inner-workings of your data universe they may be among the most important technologists you employ. If you have one of these rare breeds in your environment, approach them slowly and carefully – and then chain them to their desk so they can never escape! If this is frowned upon by your HR department, alternatively consider buying more lunches. Or doughnuts. Type 2 DBAs love doughnuts. But then, who doesn’t?
KEY 3: Cloud-based databases are not an excuse to ignore database operations…
Moving data to a cloud-based structure creates some tradeoffs. Instead of worrying about the physical server environment, DBAs must now spend more time optimizing queries for server-side computation. If it is an external cloud, security access and encryption mechanisms must be more robust. The ease of spinning up new virtual servers and databases leads to such proliferation that it increases the administrative complexity and costs.
You may also find that some solutions are more prone to performance issues in a cloud-based environment – and more complexity and potential bottlenecks between data and consumer can lead to more difficult diagnostic challenges. But if your company lacks Type 2 DBA resources, cloud-based solutions may be a better set of problems to tackle.
KEY 4: …and neither is unstructured data
DAMA specifically calls out structured data in their definition of data operations management, but I think that it isn’t as relevant today as when the specification was written. Unstructured data (“big data” often falls into this category) relies upon a new set of tools to get at the value. These tools are commonly based on key/value pairs that help make some estimations and assumptions about the data. This negates the need for some of the traditional DBA activities.
That said, it can be challenging to run a typical financial report off of unstructured data. Things with a lot of strict rules (like accounting) don’t belong in unstructured data systems. Database operations should consider how unstructured/structured data interact, and develop the appropriate architecture and reporting mechanisms. If anything, adding unstructured data to your data landscape makes the broader structure more complicated!
KEY 5: You shouldn’t build what you can’t support
As a consultant, it is awfully tempting to build something AWESOME when I won’t have to support it day-in and day-out. Even a developer that sees their DBA every morning at the doughnut shop may have trouble resisting. As a result, I’m sure some organizations have overbuilt their data environments.
This scenario leads to potentially problematic issues: higher overhead, inefficient support, or overextended personnel. Typically all three. Technology staff is notoriously good at overcoming these – until they can’t. Restoring balance happens either through colossal system failure, employee turnover, or the most common complaint: slowness.
In the end, the specific tasks and activities of your DBA or other database operations staff are not your primary concern. The most important thing to you is that you are able to access your data, and it is acceptable to have trusted personnel handling the technical details. If you choose instead to ignore your database operations, sooner or later your data will let you down.
Anthony J. Algmin is a Manager in the Business Intelligence Practice at West Monroe Partners.