Big, Clean, Valuable: 12 Steps To Better Data and Quality Insights

Do you have data dragons? Is your data a mess? Is data stored everywhere and you don’t know who uses what? Is analytics taking forever because your data quality is poor or non-existent?

Is your data holding you back?

It’s rare that a board of directors, especially in small and medium enterprises, signs off funds to straighten out creaky data. Frequently seen as housekeeping and a problem for IT, data cleansing, modelling and data governance aren’t tackled until the situation becomes untenable. It doesn’t have to be that way!

Here’s 12 simple steps to a better data platform, quality data and more valuable insight:

1. Admit you have a problem

As a data geek, I see a lot of problem data but rarely is the problem acknowledged. If you can identify with any of the question above, admitting your organisation has a data problem is the first step to fixing it.

2. Make sure you can resolve it

If you aren’t in a position to influence change within your department, organisation or team, don’t bang your head against a brick wall. While you can make changes by stealth, without the support of the right people, you’ll be bailing out your leaking canoe with a teaspoon. Not fun!

3. Make a positive choice to resolve it

That’s right, a choice. Choosing change is a decision everyone involved has to buy into. The change needn’t be a huge, expensive project but the appetite for change must be there or your efforts will come to nought.

4. Take a living inventory

Take a simple, high level inventory of where you are and keep it up-to-date. Don’t attempt to make it so detailed and hard to maintain that no-one will touch it or even worse, it becomes “your baby”. This goes back to buy-in. Keep it simple, keep it alive.

5. Find the root cause

Understand why you’re here: Poor practices? Lack of knowledge? You won’t be surprised to learn that having dedicated data professionals who understand data modelling, logical and physical design is not a priority in many organisations.

Data development is inherently different from procedural or object development. Knowing how to code in sql isn’t the same skill as wrangling or modelling data. The skill of translating the real world domain to good models however, can be learnt.

6. Understand you can’t fix it overnight

It’ll take time, so be prepared to iterate and learn from each pass. This is a marathon not a sprint.

7. Start modelling

This is essential to good database design. Now you have your inventory and have raised your skill levels, start modelling to understand what entities (people, places, things, events, objects and concepts) you record data about and how they interact.

8. Know your pipeline

A model isn’t the be all and end all of a database. You also need to understand how the data flows around your organisation — the data pipeline if you will. This is where you’ll discover common events and meeting points. This can save time, boost understanding outside tech teams and really aid development efforts.

9. Apply good practice

Don’t just tick off a list: unit testing, integration testing, agile methodology etc. Actually integrate them correctly into your daily operations. At a minimum, test and review regularly.

Make sure you understand good practice (no such thing as best!) and why it is useful or not for your situation. Poor practice in development becomes a pain in support, don’t make it someone else’s problem.

10. Continuously review

Ask smart questions: how are we doing? could we do this better? what was the lesson?

11. Incorporate lessons learned

If you aren’t applying the lesson to the next or ongoing iterations, you’re wasting several golden opportunities to improve your data and avoid pitfalls.

12. Toot your horn

Keep everyone appraised about how you’re doing — elevator-pitch style. Don’t overwhelm people with details but make sure improvements are seen and heard. Who knows? You might even get support for a bigger project.

Good luck!

Need help with your data strategy? Hire me.

image: janneke staaks – Research Data Management

Published by


Consultant: Data Science & Data Analysis – Making the complex, simple

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s