BY CHRIS ASHFORD

This article describes two very similar suggestions for data distribution that are currently being put forward independently by users of Census data. The first arose at a workshop on 20 March 2007 attended by representatives of ONS, GROS, NISRA, and a selection of data distributors with long-term experience of managing Census data. The second was presented in April 2007 by Eurostat to an EU Task Force formed of representatives from Member States to consider options for providing future Census data to Eurostat.

At this stage no commitment is being made by the UK Census Offices. However, these suggestions will be taken into account in the analysis of user requirements, after wider and fuller consultation, and in the development of a strategy for output.

Background

Significant advances were made in the distribution of Census data from the 2001 UK Census. All three UK Census Offices established web-based systems that enabled data users to access a vast range of statistics and employ various mapping and analytical tools. In addition, a number of established data distributors were able to extend and enhance their services, systems and software to better meet the needs of their user communities. The period after the 2001 Census also saw an increase in the distribution of census data via the Internet by other organisations, particularly local authorities.

The UK Census Offices supplied all 2001 Census data on CD and DVD, and a large amount of this was provided in a format designed specifically for loading into database systems. However, users raised concerns that the format was not consistent across the three Census Offices, and data distributors who managed UK data had to reformat some data themselves.

The Registrars General for England and Wales, Scotland, and Northern Ireland have published an Agreement that states their intention to promote UK harmonisation in the 2011 UK Census, and to produce consistent and coherent outputs for the UK and for each component country.

Ideas From UK Data Distributors

The three UK Census Offices are therefore considering ways in which 2011 Census data might best be made available to systems that distribute data to users.

It is important that the Census Offices understand the technical and cultural environment that is likely to exist when the 2011 Census data is released over
the period 2012-2014 and subsequent years. No-one can know for certain but there are trends developing, particularly in the use of the Internet, that are likely to strengthen in the next few years.

A workshop was therefore conducted in March 2007, to establish basic principles for the supply of data to organisations intending to distribute it through their own systems. This brought together the UK Census Offices with a number of distributors of Census data to discuss, without prejudice or commitment, how 2011 Census data might be accessed and used by the different user communities. The meeting provided an opportunity to share thoughts about future dissemination and identify the issues involved in developing an efficient and cost effective delivery of 2011 Census data.

The workshop concluded that 2011 Census dissemination should make significant use of the Internet, as at present, but that the development of new systems would most likely rely on open source approaches that share software and applications. This would require data to be supplied from the Census Offices in a format that is compatible with a range of software. It was suggested that the Internet practice of providing live data feeds could be used allowing all data to be stored at the three Census Offices and avoiding the need for distribution on physical media.

There was much support for the development of prototypes and pilots using 2001 Census data, and the three UK Census Offices will be considering how best to be involved in such projects, taking cognisance of available resources. Some data distributors are carrying out further discussion and investigations and will be contacting the Census Offices with their findings.

Eurostat proposal: a Census European Hub

ONS, GROS, and NISRA also operate within an European perspective, and there are a number of initiatives that may influence the eventual dissemination model used to supply 2011 UK Census data. One of these is the Census European Hub, which could have a significant impact on the way in which the Census Offices meet their obligation to supply data to Eurostat. The initiative shares many features of the type of dissemination model discussed at the UK workshop.

The Census European Hub is a concept, developed by The Statistical Office of the European Community (Eurostat), of a new system to publish European Census data on the Eurostat website. Although the system is still conceptual, prototypes and pilots are planned to take place soon.

How The Hub Would Work

Eurostat are proposing that each national statistics institute (NSI) creates a set of non-disclosive ‘cubes’ of data. These would be available for Eurostat to use as the base for their Census dissemination system. Each NSI would provide web access to their data according to standard processes, formats and technologies, and applying appropriate levels of statistical disclosure control.

Data providers would be able to make data available directly from their systems through a querying system managed by the hub, notifying the hub of any new sets of data and corresponding metadata. Users of the dissemination system would browse the hub to define a dataset of interest and receive the dataset from the hub, the latter having retrieved the data from the appropriate NSIs.

Eurostat envisage the Census European Hub using standards set out by the Statistical Data and Metadata Exchange (SDMX) initiative, launched in 2001 by Eurostat together with six other sponsors: the Bank for International Settlements (BIS), the European Central Bank (ECB), the International Monetary Fund (IMF), the Organisation for Economic Co-operation and Development (OECD), the United Nations Statistical Division (UNSD) and the World Bank. The stated aim was to develop and implement standards and guidelines for a more efficient transmission and dissemination of statistics, including both data and metadata.

Eurostat are intending to start the development of the Census European Hub by involving three NSIs in a pilot phase throughout 2007 and 2008. A complete implementation is planned for late 2010.

For further details please contact Chris Ashford, ONS, at chris.ashford@ons.gsi.gov.uk

Return to top