#PLE2010 update – the outcomes of the review process

A further update on planning and preparations for the PLE2010 conference. We received 81 proposals, far more than we had expected. And whilst very welcome, this has generated a lot fo work. Each proposal was assigned two reviewers from the conference Academic Committee. This has meant some members of the Committee being asked to review six papers which is quite an effort for which we are truly grateful

One of the main points made in feedback to us from the reviewers was that a 360 word abstract is too short to make a proper judgement. And indeed some submissions did not make full use of the 360 words. We produced criteria for the submissions which were used by some reviewers. Others disagreed with this approach. Stephen Downes, commenting on my last blog post about the conference, said:

  • the stated criteria, as listed in the post above, are actually longer than many of the abstract submissions. As such, the criteria were overkill for what was actually being evaluated.
  • the criteria do not reflect academic merit. They are more like a check-off list that a a non-skilled intake worker could complete. The purpose of having academics do the review is that the academics can evaluate the work on its own merit, not against a check-off list.
  • the criteria reflect a specific theoretical perspective on the subject matter which is at odds with the subject matter. They reflect an instructivist perspective, and a theory-based (universalists, abstractivist) perspective. Personal learning environments are exactly the opposite of that.
  • In other words, it is not appropriate to ask academic reviewers to bring their expertise the material, and to then neuter that expertise with overly perspective statement of criteria.

On the whole I think I agree with Stephen. But I am still concerned with how we reach some common understandings or standards for reviewing, especially in a multi-disciplinary and multi national context.

Following the completion of the reviews, the conference organising committee met (via Skype) to discuss the outcomes of the process. We did not have time to properly consider the results of all 166 reviews and in the end accepted to unconditionally accept any paper with an average score of two or more (reviewers were asked to score each submission on a scale ranging from plus to minus three). That accounted for twenty six of the proposals. Each of the remaining proposals was reconsidered by the seven members of the organising committee in the light of the feedback from the reviewers. In many of the  cases we agreed with their reviews, in some cases we did not. 30 of the proposals were accepted but we have asked the proposers to resubmit their abstract, feeling that improvements could be made in clarity and in explaining their ideas to potential participants at the conference.

We referred nine of the proposals, in the main case because whilst they seemed interesting proposals we did not feel they has sufficiently addressed the theme of the conference ie Personal Learning Environments. These we have asked to resubmit the abstract and we will review the proposals for a second time. In a small number of cases we have recommended a change of format, particularly for research which is still at a conceptual stage which we felt would be better presented as a short paper, rather than a full proceedings paper. And, following the reviews, we did not accept five of the proposals. Once more the main reason was their failing to9 address the themes of the conference.

I am sure we will have upset some people through this process. But the review process was if nothing else rigorous. the meeting to discuss the outcome lasted late into the evening the we were concerned wherever possible to be inclusive in our approach. We also decided not to use the automatic functionality of the EasyChair system for providing feedback on the proposals. the main reason for this was that we were very concerned that feedback should be helpful and constructive for all proposers. Whilst many of the reviews were very helpful in that respect, some were less so and thus we have edited those reviews.

Four quick thoughts on all this:

  • I am not sure that people spend enough time thinking about the calls for papers. What are the themes a conference is trying to address? How does my work contribute towards those themes.
  • I wonder if many academics struggle with writing abstracts. I was surprised how many did not use their full 360 words in their proposals. Abstracts are difficult to write (at least I find them hard) and perhaps our 360 word limit constrained many. However, it was surprising how many were not really clear in focus.
  • I am still concerned with how we can develop common understandings and standards between reviewers. Maybe we need some sort of discourse process between reviewers.
  • The task of providing clear feedback and judgement about proposals whilst still proving constructive and helpful feedback to proposers is not easy. Once more, this maybe something which needs to be addressed at a community level.

Proudly announcing the Personal Learning Enviroments Conference

I am very happy to announce this conference. There’s a great bunch of people associated with the conference and we hope the conference themes and guidelines are sufficiently broad and inclusive to encourage creativity and learning. I will blog more about the conference in coming days. In the meantime check out the conference website and I hope to see you all in Barcelona.

“Personal Learning Environments (PLE) include the tools, communities, and services that constitute the individual educational platforms learners use to direct their own learning and pursue educational goals. The idea of the PLE represents a shift away from the model in which students consume information through independent channels such as the library, a textbook, or an LMS, moving instead to a model where students draw connections from a growing matrix of resources that they select and organize. Because they emphasize relationships, PLEs can promote authentic learning by incorporating expert feedback into learning activities and resources. A PLE also puts students in charge of their own learning processes, challenging them to reflect on the tools and resources that help them learn best. By design, a PLE is created from self-direction, and therefore the responsibility for organization—and thereby for learning—rests with the learner. (7 things you should know about Personal Learning Environments, Educause 2009).

The PLE Conference is intended to produce a space for researchers and practitioners to exchange ideas, experience and research around the development and implementation of PLEs including the design of environments, sociological and educational issues and their effectiveness and desirability as (informal) learning spaces.

Whilst the conference includes a traditional research paper strand, we also encourage proposals for sessions in different formats including workshops, posters, debates, cafe sessions, hands on sessions and demonstrations. We will also provide opportunities for unconferencing events, including the provision of spaces for informal meetings and discussions. Although the main conference takes place on 8th and 9th of July, there will be an informal launch event (with wine and tapas!) on the evening of Wednesday 7th.

As well as the face to face sessions, the conference will be supported by a variety of different online spaces.

Selected papers will be published by the International Virtual and Personal Learning Environments Journal.”