What’s there? What’s missing? Quick guide to understanding data completeness

When we talk about coverage or completeness, we want to know a couple of things. First, what’s there? and second, what’s missing? We want to survey the land and get a short but complete overview. How do we do this? We look at our data from more than one angle.

A map is not the territory…

Data is a tool

It represents something we’re interested in. That thing could be cars, loans, flowers, or cups. Whatever it is, we want to record or review information about it. Knowing can help us sell the right cars, guide our clients to the right loans, report on the state of the flower industry or manufacture more instragrammable cups.

A white cup in focus on a table with a blue tablecloth in the background
Reality

Data describes concepts

It represents ideas we’re sharing. There are many styles and shapes of cups in the world, but the icon of a cup is pretty much universally understood. I may not know the style or shape of your cup but I understand “cup-ness”.

Cup Icon by Design Revision
Concept

How does this help us understand completeness?

Let’s take a step back. We’re unlikely to be interested in every cup that ever existed, so we have a scope. Let’s say we’re interested in cups we make and sell. Our universe of cups is limited to just those cups.

We want to know a few things about our cups: the materials we used, how large or small they are, that sort of thing. So we decide on headings or columns for each of the attributes (information about cups) that we’re interested in.

cup schema
Schema

This list of things about cups is the schema. It’s a template that describes what we want to know about cups. It isn’t our data on cups (we’ll add that under the headings) but it gives us some direction about what to record.

Unlike the concept of a cup, the schema of a cup isn’t intuitive. We’d struggle to instantly recognise “cup-ness” by looking over this list. We’ve taken reality, abstracted it to a concept then made that into a schema which is the container for our data.

So back to completeness. When we talk about completeness, we could be talking about the concept or the schema. These are different questions but together gives us insight into the state of our cup data.

  • Concept – How many cups are we reporting?
  • Schema – How many cup attributes are we reporting?

Concepts & Schemas: How are they different?

In general, when we talk about a concept of a cup, we have a list of information we need to understand “cup-ness”. So we may agree it’s not a “cup” unless we have these things: cup#,  name and type. That’s close enough to our concept of a cup that we can ask questions about the number of cups. This is the sort of information we use to plan campaigns, make strategic decisions and launch new cups to the market.

In reality, we don’t record everything diligently. We miss things out for a host of reasons. This is even more obvious when we aren’t recording the data ourselves.

Data has gaps

Understanding where those gaps are is important. Gaps affect how we report on concepts. If we’re missing cup names, that reduces the number of cups we report. We use information about gaps to improve our data collection so that we can make better strategic and planning decisions.

Takeaway

The upshot? To understand how complete our data is, we survey our data landscape in two ways: by concepts and by schema. We can count conceptual cups or count cup attributes to find what’s there and what’s missing. The two strands help us understand what’s going on in our data.

  • Some things (or attributes) are more important than others, they map to concepts;
  • Some things are conceptual (“cup-ness”) others are schematic (the cup attributes);
  • Some things are more useful for planning and strategy (concept) and others for improving data quality (schema).

 

 

 

 

Have a good open data policy

Can I Trust Your Open Data?

You want people to use your data. They want confidence that they can trust your data and rely on it, now and in the future. A good open data policy can help with that.

An open data policy sets out your commitment to your open data ecosystem. It should detail how you will collect, process, publish and share data. It will set expectations for anyone using your open data and if you stick to it, lead to confidence about what to expect.

You can create your own open data policy from the Open Data Services  open data policy template, check out the Sunlight Foundation guidelines or Socrata’s How to develop your open data policy article. Here’s some open data policies in the wild:

Remember: It’s not enough to have a policy, you have to stick it to build trust and confidence in you as an open data publisher and in your open data.

Make It Play Well With Other Data

How do I make my open data as useful as possible? How do I connect it with other data to boost insight? How do I answer really tough questions with open data? Make it play well with other data – make it interoperable.

interoperable

(ˌɪntərˈɒprəbəl)

adj

(Computer Science) of or relating to the ability to share data between different computer systems, esp on different machines: interoperable network management systems.

 Why should you care about this?

If you want your open data to help answer questions, solve problems, boost the economy by fuelling innovation or used in research, you need to go beyond names and places.
Do these mean the same company?
  • ACME
  • ACME Limited
  • A.C.M.E
How about now?
  • GB-COH-123456: ACME
  • GB-COH-123456: ACME Limited
  • GB-COH-123456: A.C.M.E
Bit more confident? You can take that code 123456* and find the company on Companies House (Hint, that’s what the GB-COH- tells the machine using your open data!). Go you, you’ve just opened up a whole new world of information! This example is using a shared standard way of talking about organisations, find out more on org-id.guide.
(* P.S This is just an example, ACME doesn’t really exist!)

Now what?

You can start to answer question like this:
Answer tough questions with good quality open data
Answer tough questions with good quality open data
These codes or Identifiers are  a gold mine. Every country has agencies that give codes to businesses, charities, non profits and more. Use those codes where you can.

Can I share codes for anything else?

Of course! You can identify places, things, categories, types and much, much more.

Tip: Make your open data more useful by making it easy to connect with other data.

See all the tips in one place: Good Quality Open Data

More on: interoperable
Courtesy of Collins English Dictionary – Complete and Unabridged, 12th Edition 2014 © HarperCollins Publishers 1991, 1994, 1998, 2000, 2003, 2006, 2007, 2009, 2011, 2014

Make It Easy To Get Hold Of

Good quality open data is accessible; easy to get hold of and easy to use.

Accessible means different things depending on your audience and how much data we’re talking about.

Mostly Human
Mostly Human

Is your audience mostly human?

Publish files humans can use, like spreadsheets for information in rows and columns, and shapefiles for geography.

Tip: Make sure the files aren’t too big to open on an everyday person’s computer.

Mostly Machines
Mostly Machines

Is your audience mostly machine?

Machines can work with spreadsheets or formats like JSON or xml, designed for exchanging data.

Do you have a little data? Start with files but think about making it even easier for machines with an API – a way of accessing information. Think of API as a bartender that serves up your open data.

Do you have a lot of data? You definitely want a good API that lets machines ask for a little data or a lot, ask for new data or just what’s changed, depending on their needs.

Bulk Is Good!
Bulk Is Good!

For everyone

Make it easy to get all your open data in one go – a bulk download. If you’ve got far too much for one download, break your open data into manageable chunks. Don’t forget to squish those files down as much as possible by zipping them up.

See all the tips in one place: Good Quality Open Data

Do not fear blanks!

Sometimes information is missing. Maybe it was never collected, maybe it was wrong. Whatever the case, let blanks be blank.

Are you missing something?
Are you missing something?

Don’t use placeholders when you mean “I don’t know”. Using abbreviations like “N/K” or even words like “Unknown” can seem helpful but are only really useful for humans who speak your language (and understand what the abbreviation really means!)

Don't use placeholders
Don’t use placeholders

Placeholders make it harder to work out that there is actually something missing. Think about all the ways you can write “Not Known”! It’s far better to leave a blank. Blanks are familiar and can be picked up by lots of tools.

Tip: Blanks are fine for numbers too.

See all the tips in one place: Good Quality Open Data

Have an open license

Open data without a license isn’t open.

Licensed to thrill?
Licensed to thrill?

Why?

The license, a text that describes how data can and can’t be used, is a must for open data. Without that clear statement of use, it’s impossible to tell if the information isn’t subject to copyright or other restrictions, so you’re using it at risk of litigation and legal challenges.

A good open data license has few restrictions – it allows as many people as possible to use the data with as few conditions as possible. The more conditions on your data, the harder it is to use it with other data.

 

 

open government license
Open government license

Let’s see some examples of excellent open data licenses:

 

Make it visible
Make it visible

Your license should be clear and clearly visible. Where possible, put your license on a page that links to your open data.

Tip: If you use a tool to manage access to your open data like CKAN, make sure you set the license.

See all the tips in one place: Good Quality Open Data

Be consistent

The golden rule for open data that’s useful is consistency.

Consistent filenames
Consistent filenames

Consistency means picking a naming strategy for your files then sticking to it. This makes it easy to spot that files are missing or out of place.

Consistent headers
Consistent headers

You’ll also want to keep your table headers the same for each new file so that anyone using your data, for example combining files, can do that easily. Changes to your headers break code and make your files harder to use.

Consistent content
Consistent content

Finally, keep your contents the same. ’12’ and ‘twelve’ aren’t the same thing. This makes it harder to use the information for analysis [see Tidy Data by Hadley Wickham].

Tip: If you can’t do maths on it, it’s text not a number.

See all the tips in one place: Good Quality Open Data