Consolidation of data

I’ve been reading research papers recently (I must be in a “deep thoughts” kinda mood) and one from IDC floated across my screen. It discussed the consolidation of data as a by-product of organizations looking for more accurate and more timely analysis. As an example, the nation’s health data is spread out over five large databases.

Oak Ridge National Lab has submitted a proposal to unify all these databases and perform fraud detection using one of the world’s fastest supercomputers, their Cray system nicknamed “Jaguar.” This solution could save $50 billion a year by analyzing the data in near-real time.

It’s all about pattern recognition: trying to stop the terrorist before s/he strikes or spotting fraud at financial institutions or with Medicare. But it’s not just stopping the bad guys, it’s also about combining every larger databases about our behaviors and knowing what we are going to do before we do it. The more data points, the more accurate the predictions.

Having all that data in one place will create serious security concerns. This is one of the reasons the ID card debate in the UK has been consumed with turmoil. Of course, imagine feeding all the data about real-time cyber attacks and network activity into one database that looks at patterns. We could spot the emergence of new worms and viruses quite quickly and respond shortly thereafter with pinpoint patches.

If information is power, the concentration of all that power into one place is unnerving. The creators of the US Constitution new this and spread out power across three branches. Do we want government or private organizations to have that much power over us?

As Schneier puts it, it’s all about liberty v. control. Or how I like to put it: job security.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s