Anonymising open data

Here is the next in our occasional series about open and linked data. I wrote in a previous post that we are worki8ngt on developing an application for visualising Labour market Information for use in careers guidance.

One of the major issues we face is the anonymity of the data. fairly obviously, the mo0re sources of data are linked, the more possible it may become to identify people through the data. The UK information Commissioner’s Office has recently published a code of practice on “Anonymisation: managing data protection risk” and set up an Anonymisation Network. In the foreword to the code of practice they say:

The UK is putting more and more data into the public domain.

The government’s open data agenda allows us to find out more than ever about the performance of public bodies. We can piece together a picture that gives us a far better understanding of how our society operates and how things could be improved. However, there is also a risk that we will be able to piece together a picture of individuals’ private lives too. With ever increasing amounts of personal information in the public domain, it is important that organisations have a structured and methodical approach to assessing the risks.

The key points about the code are listed as:

  • Data protection law does not apply to data rendered anonymous in such a way that the data subject is no longer identifiable. Fewer legal restrictions apply to anonymised data.
  • The anonymisation of personal data is possible and can help service society’s information needs in a privacy-friendly way.
  • The code will help all organisations that need to anonymise personal data, for whatever purpose.
  • The code will help you to identify the issues you need to consider to ensure the anonymisation of personal data is effective.
  • The code focuses on the legal tests required in the Data Protection Act
Particularly useful are the Appendices which presents a list of key anonymisation techniques, examples and case studies and a discussion of the advantages and disadvantages of each. These include:
  • Partial data removal
  • Data quarantining
  • Pseudonymisation
  • Aggregation
  • Derived data items and banding
The report is well worth reading for anyone interested in open and linked data – even if you are not from the UK. Note for some reason files are downloading with an ashx suffix. But if you just change this locally to pdf they will  open fine.

Open data and Careers Choices

A number of readers have asked me about our ongoing work on using data for careers guidance. I am happy to say that after our initial ‘proof of process’ or prototype project undertaken for the UK Commission for Employment and Skills (UKCES), we have been awarded a new contract as part of a consortium to develop a database and open APi. The project is called LMI4All and we will work with colleagues from the University of Warwick and Raycom.

The database will draw on various sources of labour market data including the Office of National Statistics (ONS) Labour Force Survey (LFS) and the Annual survey of Hours and Earnings (ASHE). Although we will be developing some sample clients and will be organising a hackday and a modding day with external developers, it is hoped that the availability of an open API will encourage other organisations and developers to design and develop their own apps.

Despite the support for open data at a policy level in the UK and the launch of a series of measures to support the development of an open data community, projects such as this face a number of barriers. In the coming weeks, I will write a short series of articles looking at some of these issues.

In the meantime, here is an extract from the UKCES Briefing Paper about the project. You can download the full press release (PDF) at the bottom of this post. And if you would like to be informed about progress with the project, or better still are interested in being involved as a tester or early adapter, please get in touch.

What is LMI for All?

LMI for All is a data tool that the UK Commission for Employment and Skills is developing to bring together existing sources of labour market information (LMI) that can inform people’s decisions about their careers.

The outcome won’t be a new website for individuals to access but a tool that seeks to make the data freely available and to encourage open use by applications and websites which can bring the data to life for varying audiences.

At heart this is an open data project, which will support the wider government agenda to encourage use and re-use of government data sets.

What will the benefits be?

The data tool will put people in touch with some of the most robust LMI from our national surveys/sources therefore providing a common and consistent baseline for people to use alongside wider intelligence.

The data tool will have an access layer which will include guidance for developers about what the different data sources mean and how they can be used without compromising quality or confidentiality. This will help ensure that data is used appropriately and encourage the use of data in a form that suits a non-technical audience.

What LMI sources will be included?

The data tool will include LMI that can answer the questions people commonly ask when thinking about their careers, including ‘what do people get paid?’ and ‘what type of person does that job?’. It will include data about characteristics of people who work in different occupations, what qualifications they have, how much they get paid, and allow people to make comparisons across different jobs.

The first release of the data tool will include information from the Labour Force Survey and the Annual Survey of Hours and Earnings. We will be consulting with other organisations that own data during the project to extend the range of LMI available through the data tool.

LMI for All Briefing Paper

The problem – a major shortage of jobs

We are constantly being told that we have to improve our employability skills and qualifications for finding employment. Yet whilst more qualifications may help in getting a job, from a policy perspective if ignores the obvious. There is simply a shortage of work.

New research from the UK Joseph Rowntree Foundation explores the difficulty of job searching for young people seeking low-skilled work in three areas in England and Wales. Their overall finding is that the main problem for disadvantaged young people looking for work is fundamental – a major shortage of jobs.

Other key findings include:

  • Over two-thirds of applications (69%) received no response at all.
  • 78% of the jobs applied for paid under £7 an hour, while 54% offered the minimum wage. Only 24% of the vacancies offered full-time, daytime work.
  • In the weak labour market, 10 jobseekers chased every job compared to five jobseekers in the strong one.
  • Jobseekers who do not have high-speed internet at home are at a substantial disadvantage and can only search for jobs sporadically, rather than the daily basis that is required.
  • Applications sent a week after jobs were first advertised were half as likely to receive positive responses as those sent in the first three days.
  • The research found there was strong evidence that good-quality applicants from neighbourhoods with poor reputations were not more likely to be rejected by employers.
  • However, employers expressed a preference for local candidates with easy journeys to work.

Developing a response to youth unemployment

Since I wrote my last article on ‘What is the answer to youth unemployment?‘, elections in Greece, France and Germany have seen a decisive rejection of European austerity politics. This is hardly surprising. It doesn’t take a genius to work out that ever deeper cuts and austerity, whilst ultimately cutting the real cost of labour and thus boosting corporate profits, are unlikely to boost growth, jobs or individual prosperity in any way.

The EU reaction has been to call for a new strategy for growth, although details of what that might entail are pretty hazy.

As I wrote in the previous article, one of the main results of the recession has been a massive increase in youth unemployment and, in particular, a substantial increase in graduate unemployment. At the same time companies are increasingly requiring work experience prior to employment resulting in increasing pressure for new graduates to undertake low paid of unpaid internships. Pretty clearly new policies are needed for education and training but there seems little public discussion of this, let alone of what such policies might be. The prevailing EU policy is more of the same and try harder.

To rethink policies for education and training requires looking back at how we got where we are now. And it requires looking at more than just education and training policy – we need to examine the relationship between education and training, labour market policy and economic policy. here I am going to look at just a few aspects of such policies and hope to develop this a little more in the next week or so.

For the last decade – or even longer – economic policy has been driven by a liberal free market approach. In turn labour market policy has similarly been based on deregulating labour markets and removing protection for workers (interestingly, Germany, the one country in Europe where the economy is growing, has probably one of the highest levels of labour market regulation). At a European level, education and training policy has been dominated by a drive to make qualifications more transparent and thus comparable in order to promote the mobility of labour. Employers have been given a greater role in determining the content and form of qualifications. Employability has become a key theme, with individuals being made responsible for keeping their knowledge and skills up to date, often as considerable personal expense. A number of countries have tried to liberalise education and training systems by reducing subsidies for public education and introducing individual voucher schemes.

At them same time the rather ridiculous EU Lisbon declaration, declared the aim to make the EU “the most competitive and dynamic knowledge-based economy in the world capable of sustainable economic growth with more and better jobs and greater social cohesion”, by 2010. Obviously this failed. But in line with such thinking most countries in Europe saw the way forward as moving from old fashioned vocational training to mass university education to cater for the demand for the thousands of new knowledge jobs. These jobs never materialised (except in countries such as the UK in the deregulated financial services sector which ultimately triggered the economic meltdown). As Wikipedia notes:

Much of the initial theorizing about the advent of a fundamentally new era in which economic activity is increasingly ‘abstract’, i.e., disconnected from land, labour, and physical capital (machines and industrial infrastructure) was associated with the ‘business management’ literature of the ‘new economy’ NASDAQ bubble, which collapsed in 2001 (but slowly recovered, albeit, in a leaner format, throughout the 2000s). This literature was initially known more for its hyperbole and faddishness than for its academic/empirical integrity.

In reality, many of the new degree courses were vocational in orientation – such as in the new Universities in the UK or in the Fachshule in Germany. These courses were either for new occupations – for instance in computing or simply replaced traditional vocational qualifications. It is arguable whether such a policy was financially sustainable or even desirable. It is certainly arguable whether an academic programme of learning is more effective for such subjects than traditional forms of work related learning.

To further policies associated with the obsession with the knowledge economy were the raising of the school leaving age and the so called lifelong learning policy. Longer schooling was needed, it was argued, to cope with the needs for higher levels of knowledge and skills for the knowledge rich jobs of the future. And lifelong learning was needed for the learning economies in which knowledge is the crucial resource and learning is the most important process.

At them same time the EU and national governments identified a number of key sectors which were felt to be crucial and which were then promoted through he education systems. In the late 1990s, there were dire predications of a massive shortage of computer programmers which never came to pass. And in the last five years or so EU and national governments have promoted the importance of STEM subjects – Science, Technology, Engineering and Maths as key to the future of employment and economies. Such priorities were based on a business driven policy of skills-matching promoting the “involvement of businesses in forecasting skills needs, through an employers’ survey tool and qualitative studies on the skills needs of business” (EU New Skills, New Jobs policy).

It is clear such policies have failed  and exhorting governments and agencies to try harder will go nowhere. What is needed is a fundamental rethink. As Professor Phillip Brown points out, the Lisbon Strategy was based on the idea that the technological lead then enjoyed by advanced industrial economies would be maintained with an increasing polarisation between highly skilled and well paid jobs in those countries and low paid low skilled manufacturing jobs being undertaken in developing countries. For a variety of reasons, including rapid technology transfer and a massive expansion of public education systems in countries like China and India, this hasn’t happened.

Indeed it may be the very manufacturing sector which was downgraded by EU policy which is the future for jobs in Europe especially in Small and Medium enterprises. For all the talk of high tech, knowledge based jobs. The construction industry is the biggest industrial employer in Europe with 13,9 million operatives making up 6,6% of the total employment in EU27. In addition it has a substantial influence on other industries represented by a multiplier effect. According to a study by the European Commission, 1 person working in the construction industry is responsible for 2 further persons working in other sectors. Therefore, it is estimated that 41,7 million workers in the EU depend, directly or indirectly, on the construction sector. Out of the 3,1 million enterprises 95% are SMEs with fewer than 20 and 93% with fewer than 10 operatives (pdf file). And manufacturing makes up almost 25 percent of the German economy, as opposed to only 11 percent in the United States. German mittelstands – small, family-owned and mid-size manufacturing companies – are key to the manufacturing sector. Rather than relying on university graduates for skills and knowledge, the mittelsands tend to employ graduates from the Dual apprenticeship system.

Indeed, many countries are promoting apprenticeships as one way out of the present mess. The present English coalition government boasts of the increase in the number of apprenticeship places. But in truth most of these places are apprenticeships only in name. The supermarket chain, Morrisons is the largest apprenticeship provider in the UK with many apprenticeship consisting of short induction training courses. To deliver the skills and knowledge for workers in a manufacturing economy through apprenticeship requires high quality training and the active involvement of employers and train unions alike. Moreover it requires social (and financial) recognition fo the value of apprenticeships. that seems a long way away.

To overcome the present crisis of youth unemployment requires a series of radical and interlinked policy initiative involving economic and labour market policies rather than just tinkering with education and training curricula. At a macro econ0omic level it means developing manufacturing industry rather than merely relying on financial services and the high tech knowledge industry sector. It means making sure companies provide high quality training, rather than forcing individuals to be responsible for their own employability. It means making sure that those who have gained vocational qualifications have opportunities to use those skills and knowledge and are properly rewarded for their learning. It means freeing up capital for starting small companies. It means proper financing for vocational schools and providing alternatives to young people rather than just more school and expensive university courses. It means abandoning skills matching and planning for future societal skills needs.

In other words we have to abandon liberalisation and free market ideologies and to recognise that economies and employment are a social function. As such society has to plan for the future of employment and the provision of jobs for young people. Is this too much to ask?

 

 

What is the answer to youth unemployment?

According to the Guardian newspaper, Labour MP Hazel Blears, a member of a cross party parliamentary group of MPs looking at social mobility, says that seven out of 10 people get their next job from someone they know. She said “We need to ensure that young people from working-class backgrounds, whose parents don’t have the same exclusive networks as some in the City of London, are given the opportunities to achieve. This means ending unpaid internships and opening up opportunities as well as education and support.:

I am not quite sure what she means by opening up opportunities. But her claim that seven our of 10 people get their job from someone they know certainly rings true to anecdotal evidence. And although the UK has a national employment service, Job Centre Plus, a quick inspection shows that the jobs advertised tend be public sector or low paid and low skills jobs. There is no requirement in the UK to advertise jobs through government employment services and many of the higher paid jobs are advertised on different commercial online services.

One effect of the recession appears to be that whilst employers are not shedding jobs in the numbers feared (at least in soem countries), they are cutting back on employment by not employing young people.

Increasingly those companies who do take on young people are demanding work experience. Once more in the UK (regulations and practices vary across Europe) there has been an large increase in internship, especially for recent graduates. However, many of those posts are lowly paid if paid at all, restricting access to those who can afford to work for no pay and thus reinforcing the issues around social mobility (or lack of it). And once more, in reality the ‘best’ internships are going to those with contacts. Last year the Conservative party even auctioned an internship with a large accountancy company.

But however grim things may be in the UK, the situation in many European countries is much worse for young people. In Spain, youth unemployment is something like 55 per cent.

Last week I was at an EU Presidency conference which brought together ministers and civil servants responsible for employment and education and training policy from most EU countries (not my usual sort of conference, but they invited me as an ‘expert’ on new technologies). What soon became very apparent is that despite all the concern for what is happening, there were few if any ideas of what to do about it. It was very much business as usual but we have to try harder.

The most interesting contribution was a keynote presentation from Professor Phillip Brown from Cardiff University. He argues that the problem of equality of opportunity based on class,gender or race, has been see as “one of raising absolute standards of achievement to enable all to take advantage of new opportunities for skilled work which the globalisation of labour markets is seen to present (Reich 1991).” In his book ‘The Global Auction’, he argues that Western societies in particular have invested in human capital development, and individuals have taken on high levels of debt, on the understanding that both society as a whole and the individuals concerned will be well rewarded. But the “opportunity bargain” has not been kept.

Firstly it was based on assumption that the advanced industrial countries could grow richer through their lead in the use of advanced technology and a more highly skilled workforce, whilst other countries would rely on low paid jobs for cheap, mass production. That hasn’t  happened with countries like South Korea and China leapfrogging previous production modes and technologies. At the same time India and China are investing hugely in education, particularly in education in STEM subjects (Science, Technology, Engineering and Maths). Secondly rather than see the rise of new well paid, knowledge based jobs in advanced countries, instead , he says we have seen a new wave of “digital taylorisation:.

In a review of the book Peter Wilby says:

Digital Taylorism makes jobs easier to export but, crucially, changes the nature of much professional work. Aspirant graduates face the prospect not only of lower wages, smaller pensions and less job security than their parents enjoyed but also of less satisfying careers. True, every profession and company will retain a cadre of thinkers and decision-makers at the top – perhaps 10% or 15% of the total – but the mass of employees, whether or not they hold high qualifications, will perform routine functions for modest wages. Only for those with elite qualifications from elite universities (not all in Europe or America) will education deliver the promised rewards.

Thus doing more of the same is not an option. Neither is trying to sit out the recession and hope everything will return to normal. At a policy level it is not enough just to tinker with education systems to try to turn out more people with degrees. We need to rethink the relationship between economy, labour market and education and training. Maybe the idea that manufacturing was somehow old fashioned and was being replaced by the knowledge economy was not so clever.