Is this the right way to use machine learning in education?

An article ‘Predicting Employment through Machine Learning‘ by Linsey S. Hugo on the National Association of Colleges and Employers web site,confirms some of my worries about the use of machine learning in education.

The article presents a scenario which it is said “illustrates the role that machine learning, a form of predictive analytics, can play in supporting student career outcomes.” It is based on a recent study at Ohio University (OHIO) which  leveraged machine learning to forecast successful job offers before graduation with 87 percent accuracy. “The study used data from first-destination surveys and registrar reports for undergraduate business school graduates from the 2016-2017 and 2017-2018 academic years. The study included data from 846 students for which outcomes were known; these data were then used in predicting outcomes for 212 students.”

A key step in the project was “identifying employability signals” based on the idea that “it is well-recognized that employers desire particular skills from undergraduate students, such as a strong work ethic, critical thinking, adept communication, and teamwork.” These signals were adapted as proxies for the “well recognised”skills.

The data were used to develop numerous machine learning models, from commonly recognized methodologies, such as logistic regression, to advanced, non-linear models, such as a support-vector machine. Following the development of the models, new student data points were added to determine if the model could predict those students’ employment status at graduation. It correctly predicted that 107 students would be employed at graduation and 78 students would not be employed at graduation—185 correct predictions out of 212 student records, an 87 percent accuracy rate.

Additionally, this research assessed sensitivity, identifying which input variables were most predictive. In this study, internships were the most predictive variable, followed by specific majors and then co-curricular activities.

As in many learning analytics applications the data could then be used as a basis for intervention to support students employability on gradation. If they has not already undertaken a summer internship then they could be supported in this and so on.

Now on the one hand this is an impressive development of learning analytics to support over worked careers advisers and to improve the chances of graduates finding a job. Also the detailed testing of different machine learning and AI approaches is both exemplary and unusually well documented.

However I still find myself uneasy with the project. Firstly it reduces the purpose of degree level education to employment. Secondly it accepts that employers call the shots through proxies based on unquestioned and unchallenged “well recognised skills” demanded by employers. It may be “well recognised” that employers are biased against certain social groups or have a preference for upper class students. Should this be incorporated in the algorithm. Thirdly it places responsibility for employability on the individual students, rather than looking more closely at societal factors in employment. It is also noted that participation in unpaid interneships is also an increasing factor in employment in the UK: fairly obviously the financial ability to undertake such unpaid work is the preserve of the more wealthy. And suppose that all students are assisted in achieving the “predictive input variable”. Does that mean they would all achieve employment on graduation? Graduate unemployment is not only predicated on individual student achievement (whatever variables are taken into account) but also on the availability of graduate jobs. In teh UK  many graduates are employed in what are classified as non graduate jobs (the classification system is something I will return to in another blog). But is this because they fail to develop their employability signals or simply because there simply are not enough jobs?

Having said all this, I remain optimistic about the role of learning analytics and AI in education and in careers guidance. But there are many issues to be discussed and pitfalls to overcome.

 

Jobs of the Future

There is a lot of speculation at the moment as to the jobs of the future. On the one hand, it is said that we are educating young people for jobs which do not yet exist; on the other hand there are dire predictions that up to of existing 55 per cent of jobs may disappear to automation in the next five years.

If it is hard as a researcher who works with labour market data to make sense of all this, imagine what it is like for young people trying to plan a career (and if doing a degree in the UK, running up major debt).

However, there is beginning to appear some more nuanced research on the future of jobs. Michael Chui, James Manyika, and Mehdi Miremadi have just published the initial report on a research project looking at how automation will affect future employment. The report, entitled ‘Where machines could replace humans—and where they can’t (yet)’, is based on detailed analysis of 2,000-plus work activities for more than 800 occupations. Using data from the US Bureau of Labor Statistics and O*Net, they have quantified both the amount of time spent on these activities across the economy of the United States and the technical feasibility of automating each of them.

Their overall finding is that while automation will eliminate very few occupations entirely in the next decade, it will affect portions of almost all jobs to a greater or lesser degree, depending on the type of work they entail.
automation
Each whole occupation is made up of multiple types of activities, each with varying degrees of technical feasibility. In practice, they explain, automation will depend on more than just technical feasibility. Five factors are involved: technical feasibility, costs to automate, the relative scarcity, skills and costs of workers who might otherwise do the activity, benefits (e.g. superior performance) of automation beyond labour costs substitution and regulatory and social acceptance considerations.
The likelihood and ease of automation depends on the types of activities organised on a continuum of less susceptible to automation to more susceptible to automation: managing others, applying expertise,  stakeholder interactions, unpredictable physical work, data collection, processing data, predictable physical work. Thus occupations like accommodation, food service and manufacturing which include a large amount of predictable physical work are likely to be automated, similarly work in finance and insurance which involves much processing of data. On the other hand jobs in construction and in agriculture which comprise predominantly unpredictable physical work are unlikely to be automated, at least at present. And there is good news for teachers: “the importance of human interaction is evident in two sectors that, so far, have a relatively low technical potential for automation: healthcare and education.”

LMI for All API released

I have written periodic updates on the work we have been doing for the UKCES on open data, developing an open API to provide access to Labour Market Information. Although the APi is specifically targeted towards careers guidance organisations and towards end users looking for data to help in careers choices, in the longer term it may be of interest to others involved in labour market analysis and planning and for those working in economic, education and social planning.

The project has had to overcome a number of barriers, especially around the issues of disclosure, confidentiality and statistical reliability. The first public release of the API is now available. The following text is based on an email sent to interested individuals and organisations. Get in touch if you would like more information or would like to develop applications based on the API.

The screenshot above is of one of the ten applications developed at a hack day organised by one of our partners in the project, Rewired State. You can see all ten on their website.

The first pilot release of LMI for All is now available and to send you some details about this. Although this is a pilot version, it is fully functional and it would be great if you could test it as a pilot and let us know what is working well and what needs to be improved.

The main LMI for All site is at http://www.lmiforall.org.uk/.  This contains information about LMI for All and how it can be used.

The APi web explorer for developers can be accessed at http://api.lmiforall.org.uk/.  The APi is currently open for you to test and explore the potential for  development. If you wish to deploy the APi in your web site or application please email us at graham10 [at] mac [dot] com and we will supply you with an APi key.

For technical details and details about the data go to our wiki at http://collab.lmiforall.org.uk/.  This includes all the documentation including details about what data LMI for All includes and how this can be used.  There is also a frequently asked questions section.

Ongoing feedback from your organisation is an important part of the ongoing development of this data tool because we want to ensure that future improvements to LMI for All are based on feedback from people who have used it. To enable us to integrate this feedback into the development process, if you use LMI for All we will want to contact you about every four to six months to ask how things are progressing with the data tool. Additionally, to help with the promotion and roll out of LMI for All towards the end of the development period (second half of 2014), we may ask you for your permission to showcase particular LMI applications that your organisation chooses to develop.

If you have any questions, or need any further help, please use the FAQ space initially. However, if you have any specific questions which cannot be answered here, please use the LMI for All email address lmiforall [at] ukces [dot] org [dot] uk.

 

What is a knowledge worker?

I was at a meeting earlier this week discussing our ideas for a project using mobile devices for work based learning in the construction industry (see previous blog entry). We have emphasised the importance of interaction with physical objects in the workplace, which I think has generally been underestimated or even ignored in most elearning research and applications, at least outside the e-science domain.

We were asked whether the ideas we were putting forward were applicable to knowledge workers.

According to Wikipedia:

Knowledge workers in today’s workforceare individuals who are valued for their ability to act and communicate with knowledge within a specific subject area. They will often advance the overall understanding of that subject through focused analysis, design and/or development. They use research skills to define problems and to identify alternatives. Fueled by their expertise and insight, they work to solve those problems, in an effort to influence company decisions, priorities and strategies. What differentiates knowledge work from other forms of work is its primary task of “non-routine” problem solving that requires a combination of convergent, divergent, and creative thinking (Reinhardt et al., 2011).[1] Also, despite the amount of research and literature on knowledge work there is yet to be a succinct definition of the term (Pyöriä, 2005)

I am not sure that the concept of knowledge workers is very helpful. In reality many jobs today are requiring research skills and non routine problem solving as well as creative thinking. And that goes well beyond people who spend most of their days working in front of a computer or what used to be called ‘white collar’ jobs.

Indeed one of the big issues in the building and construction industry appears to be rapidly increasing needs for higher levels of skills and knowledge, driven largely by new (and especially green) technologies and work processes. Traditional course based further training does not scale well – and may not be particularly effective when not linked to workplace practice.

Proving this ‘hypothesis’ is not so easy and of course leads us back to the issue of what constitutes knowledge in a work based context. But in November last year I attended a fascinating (at least to me :) ) seminar hosted by the LLAKES project at the Institute of Education in London where Any Dickerson  discussed work undertaken for the UKCES on:

the development of a new and comprehensive set of detailed, multi-dimensional occupational skills profiles for the UK by combining the US-based Occupational Information Network (O*NET) system with the UK Standard Occupational Classification (SOC2010). This enables the multi-dimensional O*NET system to be used to generate comprehensive occupational skills profiles for the UK, providing a much more detailed depiction of skills utilisation, and changes in utilisation, than is currently available for the UK.

The project report “Developing occupational skills profiles for the UK : a feasibility study” provides detailed information about the methodology and findings. And I suspect, with a little more detailed analysis, it should be possible to draw some conclusions about changing skills and knowledge components in different occupations.

Why is this important? Obviously it has implications for economies and employment. But from the point of view of teaching and learning – and especially developing learning opportunities – we should be training for the future not the past or even the present. To do this we need a detailed understanding of what is happening in different occupations. And we need to get beyond policy rhetoric about the knowledge economy and knowledge workers.

Technology and Careers Education

I am ever more interested in how we can use technology for supporting young people (and not just young people) in making future careers choices. Talking to a focus group of young people last year, they  make use of the internet, particuarly using Google to search for details about possible future education choices, jobs and careers. I also asked them how far they explored the results of a Google search and not surpisingly they told me that they usually looked at the first three or four results. Try doing this yourself – type in your job of choice and see what comes up. All too often the results can be highly misleading.

Anyway we are working on new tools to asisst young people in choosing their future careers (more details to follow quite soon, I hope). And here is a conference paper, prepared by my colleague Sally Anne Barnes from the University of Warwick and describing the emprirical research we undertook on the use of technology in careers education in the UK (requires html5 compliant browser to view). Comments welcome – I would especially like to know what is happening in other countries than the UK.

The Potential Role of Technology in Careers Education in the UK