Pontydysgu is very happy to be part of a consortium, led by DMH Associates, selected as a finalist for the CareerTech Challenge Prize!
The project is called CareerChat and the ‘pitch’ video above expalisn the ideas behind the project. CareerChat is a chatbot providing a personalised, guided career journey experience for working adults aged 24 to 65 in low skilled jobs in three major cities: Bristol, Derby and Newcastle. It offers informed, friendly and flexible high-quality, local contextual and national labour market information including specific course/training opportunities, and job vacancies to support adults within ‘at risk’ sectors and occupations
CareerChat incorporates advanced AI technologies, database applications and Natural Language Processing and can be accessed on computers, mobile phones and devices. It allows users to reflect, explore, find out and identify pathways and access to new training and work opportunities.
Nesta is delivering the CareerTech Challenge in partnership with the Department for Education as part of their National Retraining Scheme
Nesta research suggests that more than six million people in the UK are currently employed in occupations that are likely to radically change or entirely disappear by 2030 due to automation, population aging, urbanisation and the rise of the green economy.
In the nearer-term, the coronavirus crisis has intensified the importance of this problem. Recent warnings suggest that a prolonged lockdown could result in 6.5 million people losing their jobs. [1] Of these workers, nearly 80% do not have a university degree. [2]
The solutions being funded through the CareerTech Challenge are designed to support people who will be hit the hardest by an insecure job market over the coming years. This includes those without a degree, and working in sectors such as retail, manufacturing, construction and transport.
As part of the AI and vocational education and training project funded through the EU Erasmus plus project we are producing a series of case studies of the use of AI in VET in five European countries. Here is my first case study – the Ada chatbot developed at Bolton College.
About Bolton College
Bolton College is one of the leading vocational education and training providers in the North West of England, specialising in delivering training – locally, regionally and nationally – to school leavers, adults and employers. The college employs over 550 staff members who teach over 14,500 full and part time students across a range of centres around Bolton. The college’s Learning Technology Team has a proven reputation for the use of learning analytics, machine learning and adaptive learning to support students as they progress with their studies.
The Ada Chatbot
The Learning Technology Team has developed a digital assistant called Ada which went live in April 2017. Ada, which uses the IBM Watson AI engine, can respond to a wide range of student inquiries across multiple domains. The college’s Learning Technology Lead, Aftab Hussain, says “It transforms the way students get information and insights that support them with their studies.” He explains: “It can be hard to find information on the campus. We have an information overload. We have lots of data but it is hard to manage. We don’t have the tools to manage it – this includes teachers, managers and students.” Ada was first developed to overcome the complexity of accessing information and data.
Student questions
Ada is able to respond to student questions including:
General inquiries from students about the college (for example: semester dates, library opening hours, exam office locations, campus activities, deadline for applying for university and more);
Specific questions from students about their studies (for example: What lessons do I have today/this afternoon/tomorrow? Who are my teachers? What’s my attendance like? When is my next exam? When and where is my work placement? What qualifications do I have? What courses am I enrolled in? etc.)
Subject specific inquiries from students. Bolton College is teaching Ada to respond to questions relating to GCSE Maths, GCSE English and the employability curriculum.
Personalised and contextualised learning
Aftab Hussein explains: “We are connecting all campus data sets. Ada can reply to questions contextually. She recognises who you are and is personalised according to who you are and where you are in the student life cycle. The home page uses Natural Language Processing and the Watson AI engine. It can reply to 25000 questions around issues such as mental health or library opening times etc. It also includes subject specific enquiries including around English, Mathematics and business and employability. All teachers have been invited to submit the top 20 queries they receive. Machine learning can recognise the questions. The technical process is easy.” However, he acknowledges that inputting data into the system can be time consuming and they are looking at ways of automatically reading course documentation and presentations.
All the technical development has been undertaken in house. As well as being accessible through the web, Ada, has both IOS and Android apps and can also be queried though smart speakers.
The system also links to the college Moodle installation and can provide access to assignments, college information services and curriculum materials. The system is increasingly being used in online tutorials providing both questions for participants and access to learning materials for instance videos including for health and social care.
It is personalised for individuals and contextualised according to what they are doing or want to find out. Aftab says: “We are looking at the transactional distance – the system provides immediate feedback reducing the transactional distance. “
Digital assessment
Work is also being undertaken in developing the use of the bot for assessment. This is initially being used for the evaluation of work experience, where students need to provide short examples of how they are meeting objectives – for example in collaboration or problem solving. Answers can uploaded, evaluated by the AI and feedback returned instantly.
Nudging
Since March 2019, the Ada service has provided nudges to students with timely and contextualised information, advice and guidance (IAG) to support their studies. The service nudges students about forthcoming exams, their work placement feedback and more. In the following example, a student receives feedback regarding his work placement from his career coach and employer.
The College is currently implementing ProMonitor, a service which will offer teachers and tutors with a scalable solution for managing and supporting the progress made by their students. Once ProMonitor is in place, Ada will be in a position to nudge students about forthcoming assignments and the grades awarded for those assignments. She will also offer students advice and guidance about staying on track with their studies. Likewise, Ada will nudge teachers and student support teams to inform them about student progress; allowing for timely support to be put in place for students across the College.
A personal lifelong learning companion
For Aftab Hussein the persona of the digital agent is important. “In the future”, he says, “every child will have a personal lifelong learning companion which will support teaching and learning.” He thinks they will probably come from the big platform suppliers. Children will have their digital assistant from age the age of 3 or 4 and will have a Personal Learner Number allowing data to be exchanged between different institutions,
Graham Attwell will be speaking at an online webinar – LiveCareerChat@Lockdown on 6 April. The webinar, organised by DMH Associates will focus on the future challenges for careers identities and careers advice and guidance
Deirdre Hughes says “During these turbulent times, we all have an opportunity for reflection, sharing ideas and offering practical advice on how best to manage career identity and changing work practices. This webinar is designed to bring people together and to listen and/or share experiences of careers support mechanisms at a time of crisis. ”
Graham Attwell will talk about the changing international labour markets and the challenges of new technologies, including AI and automation.
I have written before about the issues of interpreting sense making from Labour Market Data and the difference between Labour Market Information and labour Market Intelligence.
This is exposed dramatically in the article in Social Europe by German Bender entitled ‘The myth of job polarisation may fuel populism’. As German explains “It has become conventional wisdom since the turn of the century that labour markets are rapidly becoming polarised in many western countries. The share of medium-skilled jobs is said to be shrinking, while low- and high-skilled jobs are growing in proportion.” But as German points out: “In a research report published last May by the Stockholm-based think tank Arena Idé, Michael Tåhlin, professor of sociology at the Swedish Institute for Social Research, found no job polarisation—rather, a continuous upgrading of the labour market.”
German goes on to explain:
The main reason is that the research, as is to be expected from studies rooted in economics, has used wages as a proxy for skills: low-paying jobs are taken to be low-skilled jobs and so on. But there are direct ways of measuring skill demands in jobs, and Arena Idé’s report is based on a measure commonly used in sociology—educational requirements as classified by the International Labour Organization’s ISCO (International Standard Classification of Occupations) scheme. Using this methodology to analyse the change in skill composition yields strikingly different results for the middle of the skill distribution.
The study found that while jobs relatively low skill demands but relatively high wages—such as factory and warehouse workers, postal staff and truck drivers—have diminished, others with the same or slightly higher skill demands but lower wages—nursing assistants, personal-care workers, cooks and kindergarten teachers—have increased.
The reason is that the former jobs are male dominated whilst the jobs which have grown have a majority of female workers. Research in most countries has shown that women (and jobs in which women are the majority) are lower paid than jobs for men, regardless of skills levels.
“Put simply”, says German: “wages are a problematic way to measure skills, since they clearly reflect the discrimination toward women prevalent in most, if not all, labour markets across the world.”
A further review of two British studies from 2012 and 2013, showed a change in the composition, but not the volume, of intermediate-level jobs. “Perhaps the most important conclusion”, German says “was that ‘the evidence shows that intermediate-level jobs will remain, though they are changing in nature’.”
The implications of this interpretation of the data are profound. If lower and medium skilled jobs are declining there is little incentive to invest in vocational education and training for those occupations. Furthermore, young people may be put off entering such careers and similarly careers advisers may further mislead school leavers.
There has been a trend in many European countries towards higher level apprenticieships, rather than providing training with the skills need to enter such medium skilled jobs. But even a focus on skills, rather than wages, may also be misleading. It is interesting that jobs such as social care and teaching appear more resistant to automation and job replacement from technologies such as Artificial Intelligence. But those who are arguing that we should be teaching so called soft skills such as team building, empathy and communication are talking about the very skills increasingly demanded in the female dominated low and middle skilled occupations. It may be that we need not ony to relook at how we move away from wages as a proxy for skills, but also look at how we measure skills.
German references research by Daniel Oesch and Giorgio Piccitto, who studied occupational change in Germany, Spain, Sweden and the UK from 1992 to 2015, characterising good and bad jobs according to four alternative indicators: earnings, education, prestige and job satisfaction.
They concluded that occupations with high job quality showed by far the strongest job growth, whereas occupations with low job quality showed weak growth regardless of indicator used.
An article ‘Predicting Employment through Machine Learning‘ by Linsey S. Hugo on the National Association of Colleges and Employers web site,confirms some of my worries about the use of machine learning in education.
The article presents a scenario which it is said “illustrates the role that machine learning, a form of predictive analytics, can play in supporting student career outcomes.” It is based on a recent study at Ohio University (OHIO) which leveraged machine learning to forecast successful job offers before graduation with 87 percent accuracy. “The study used data from first-destination surveys and registrar reports for undergraduate business school graduates from the 2016-2017 and 2017-2018 academic years. The study included data from 846 students for which outcomes were known; these data were then used in predicting outcomes for 212 students.”
A key step in the project was “identifying employability signals” based on the idea that “it is well-recognized that employers desire particular skills from undergraduate students, such as a strong work ethic, critical thinking, adept communication, and teamwork.” These signals were adapted as proxies for the “well recognised”skills.
The data were used to develop numerous machine learning models, from commonly recognized methodologies, such as logistic regression, to advanced, non-linear models, such as a support-vector machine. Following the development of the models, new student data points were added to determine if the model could predict those students’ employment status at graduation. It correctly predicted that 107 students would be employed at graduation and 78 students would not be employed at graduation—185 correct predictions out of 212 student records, an 87 percent accuracy rate.
Additionally, this research assessed sensitivity, identifying which input variables were most predictive. In this study, internships were the most predictive variable, followed by specific majors and then co-curricular activities.
As in many learning analytics applications the data could then be used as a basis for intervention to support students employability on gradation. If they has not already undertaken a summer internship then they could be supported in this and so on.
Now on the one hand this is an impressive development of learning analytics to support over worked careers advisers and to improve the chances of graduates finding a job. Also the detailed testing of different machine learning and AI approaches is both exemplary and unusually well documented.
However I still find myself uneasy with the project. Firstly it reduces the purpose of degree level education to employment. Secondly it accepts that employers call the shots through proxies based on unquestioned and unchallenged “well recognised skills” demanded by employers. It may be “well recognised” that employers are biased against certain social groups or have a preference for upper class students. Should this be incorporated in the algorithm. Thirdly it places responsibility for employability on the individual students, rather than looking more closely at societal factors in employment. It is also noted that participation in unpaid interneships is also an increasing factor in employment in the UK: fairly obviously the financial ability to undertake such unpaid work is the preserve of the more wealthy. And suppose that all students are assisted in achieving the “predictive input variable”. Does that mean they would all achieve employment on graduation? Graduate unemployment is not only predicated on individual student achievement (whatever variables are taken into account) but also on the availability of graduate jobs. In teh UK many graduates are employed in what are classified as non graduate jobs (the classification system is something I will return to in another blog). But is this because they fail to develop their employability signals or simply because there simply are not enough jobs?
Having said all this, I remain optimistic about the role of learning analytics and AI in education and in careers guidance. But there are many issues to be discussed and pitfalls to overcome.
Graham Attwell will be speaking at an online webinar – LiveCareerChat@Lockdown on 6 April. The webinar, organised by DMH Associates will focus on the future challenges for careers identities and careers advice and guidance
I have written before about the issues of interpreting sense making from Labour Market Data and the difference between Labour Market Information and labour Market Intelligence.