Editorials, Ethics, SQL Server

Building Data Estates… and Your Legacy as Data Folk

I’ve been listening (and re-listening) to a number of MS Ignite presentations – I love getting insight into the direction, the goals and the overall vision that they present when they’re talking about big over-arching issues.  Things like “tweak this statement” fade away and we’re left with directional, vision-oriented things.  That’s the good stuff.  It’s in the one of the opening talks on data platform stuff that they started talking about SQL Server, (and specifically Azure-related elements) where I think you can really start to see this shine through.

The term they were using that caught my eye was “Data Estate” – this was used to talk about the whole of your business data needs.  “Estate” may not be new to you, but it is to me.  I love the idea that the things that are being built today are the things that must survive.  I also think the all-inclusive feel of that is right on.

I was a bit worried, too, by the calls that indicate AI is a mainstream, board-room level requirement.  Not because AI isn’t important (I’m a fan) but because I fear that it, like Big Data and even waaaaay back to “client server” are terms that, as we’ve seen time and time again, can become cliche’ and pretty meaningless.  In short, they become pointy-haired-boss objectives.

Don’g get me wrong; I think applied AI will be an incredibly good thing and will bring stunning leverage to startups, mid-size companies and large enterprises alike.  I think that with the right systems in place, AI will show the way to new things, better ways, automation that is finely tuned and all of that.  I’m also first in line for self-driving cars.  But “AI” as an objective will be tough to deliver on when it comes to evaluation of projects undertaken.

I hope that we can quickly get to the point where it’s AI as a component of a solution.  “We’ll use Watson to win all Jeopardy matches and keep all profits to ourselves” – that type of thing.

But the real value in AI as a tool comes from some of the things that Microsoft talks about later on in the talk.  They’re feeding all sorts of telemetry data into their analysis engine – information on SQL Server (Azure SQL Database, sorry) usage on the cloud, how people are accessing the systems, etc.  Even nicer, they’re starting to use some of that same capability to optimize processes running in the SQL engine – to the point where it can recognize a query that was anticipated and optimized to run a certain way isn’t working as well as it could be… and do a course correction while things are going on.  That’s a great use of AI, and it’s only the start.

Machine learning is something that will help expose a number of new things, from automation to better ways to do normal things to trends and so-on.  In another mention, there was talk of this being the age of the customer.  Of transparency, of customization, of being responsive to what people need and want from your services and products.  These are really tough things to deliver on with information, data, normal reporting and such.  It takes things like AI and machine learning and feeding in significant data sets to see the patterns and deliver on these.

Bringing this all home to data folk, and to the title of this post – I do believe that your Data Estate is your Data Legacy.  Building things out with the right tools and capabilities now, thinking through keeping your options open and helping others see what’s possible and helping them get there – that’s the duty of Data People.  Without being on the lookout for the infrastructure, tools, capacity, functionality, platforms and such that you need to fulfill these legacy, it’ll be much more difficult to survive in very short order IMHO.

Oh, and security too.  Don’t forget data security for all of this.  (You knew I had throw that in there)