Thursday, 27 April 2017

Designing a cognitive training study: Success

This is the second in a series of posts that examines the key aspects of designing a cognitive training study. Post one considered the type of training programme that a researcher might design. But what does success look like for a training study? It is important to establish this during the design phase, to ensure appropriate tests are in place.

The obvious answer is that success is seen when there are improvements in performance compared to a control group. Both accuracy and response times might be important here. Improved accuracy is important in determining ability to carry out the task, but response times might be informative about underlying mechanisms. Increased speed might indicate improved automaticity or efficiency, while reduced speed might indicate greater thought prior to a response, or the use of a new strategy.

Gains in performance are most likely to be seen in the task that is being practised throughout the programme. If the training is computerised, performance can be tracked during each session, measuring both accuracy and speed. Plotting a learning curve of performance throughout the training might help to identify the number of sessions that were necessary to elicit meaningful change. We might also look for improvements in a task very similar to the trained task, indicating near transfer.

More importantly though, the hope is that improvements occur beyond the task being trained, in academic performance, demonstrating far transfer. One step further, we might also find long-term effects, whereby those who underwent training see sustained gains in academic performance. This is the holy grail of cognitive training research. Ultimately, this is of course one of the aims of our field – to improve education. But is academic improvement enough to call a training study successful? And what if a study doesn’t produce gains in academic performance?

A training study that shows no improvement in academic performance is not necessarily unsuccessful, as it may inform our cognitive theories. It might tell us that individual differences in that cognitive ability do not affect academic performance in the way we thought they did. This is clearly still a useful outcome, and will lead to new questions and hypotheses. Conversely, a training programme that has led to academic improvement might not be able to tell us anything new if the causal processes have not been considered. This may occur if many approaches have been incorporated into one study, and individual effects can't be teased apart. Training studies should be considered a tool, to help establish underlying mechanisms of learning.

Finally, it is important to think carefully about the precise aim of the study. Perhaps a strategy-based working memory training programme has been designed to improve maths performance. It may be that an overall aim of the project is to improve maths as a means of encouraging more pupils to take up maths-related subjects later in their educational careers. In this case, success might also be measured in terms of maths anxiety. The training programme may not have shown transfer to maths performance, but it may have reduced maths anxiety through providing new strategies, and this in turn might lead to the desired impact of higher enrolment to maths-related courses.

Considering what counts as success and what success means while designing the programme will help to crystallise the aims and hypotheses of the study. This will benefit in the selection of the tests used to measure success, so that the results can inform our mechanistic understanding.


Part one, on types of cognitive training, can be found here.

This post was informed by this highly recommended article:

Wednesday, 19 April 2017

An introduction to educational neuroscience: Useful resources

Every so often I get email requests for further information about educational neuroscience. I thought it would be handy to compile a list of resources and links that might be of interest. Many of my suggestions are London and UK based, so please bear this in mind and do share anything else you have come across further afield.

Websites

The Education and Neuroscience online group was set up by Lia Commissar of the Wellcome Trust, and aims to facilitate links and partnerships between teachers and researchers. There are opportunities here for sharing files, events and blog posts, as well as for getting involved with forum discussions.

In 2015, I’m a Scientist, supported by the Wellcome Trust, ran an online event where teachers could ask questions of, and engage in discussion with, scientists who research learning. Although the event is no longer open to questions, it’s a great resource for scrolling through or searching for questions that might be of interest.

The npj Science of Learning community is linked to the journal in that it aims to fulfil the journal’s aim to foster discussions across disciplines related to the science of learning. There are sections that relate to opinions, events, news, and the latest findings, and there are articles suitable for students, teachers, and researchers.

Events

Learnus is a community that aims to bring research into the classroom. Learnus hold free lectures throughout the year, and held their first conference in early 2017. Learnus also offer a free workshop for teachers, to increase awareness of the relevance of neuroscience to the classroom.

The Centre for Educational Neuroscience, a London consortium between Birkbeck, UCL, and the UCL Institute of Education host research seminars that are open to the public.

Reports

In 2014, the Wellcome Trust published the results of a teacher and parent survey that aimed to establish their views of how neuroscience can influence education.

In the same year, the Education Endowment Foundation (EEF) published a review of educational interventions that are informed by neuroscience. The EEF also have a handy toolkit that indicates the cost, strength of evidence, and impact of a range of interventions, although these are not necessarily based on neuroscience.

Books

Educational Neuroscience discusses methods (e.g. neuroimaging, computational modelling) and findings (e.g. relating to language, mathematics, executive functions), considering the relevance to education.

G is for Genes presents findings from genetics that are relevant to education, and discusses what individual differences in genetics means for educational equality.

Journals

There are a few journals that specifically publish educational neuroscience work, and these include: Mind, Brain, and Education, Trends in Neuroscience and Education, Educational Neuroscience, and the aforementioned npj Science of Learning.

Postgraduate courses - UK

Educational Neuroscience can be taken as an MSc or an MA for a joint degree from Birkbeck, University of London, and the UCL Institute of Education.

The University of Bristol offers an MSc in Education (Neuroscience and Education).

Postgraduate courses - US

Harvard Graduate School of Education offers a Master’s program in Mind, Brain, and Education.

Vanderbilt University offers postgraduate training in Educational Neuroscience.

Blogs

BOLD (blog on learning and development) is run by the Jacobs Foundation. It hosts authors who are scientists, journalists, policymakers, and practitioners.

ThInk is an educational neuroscience blog from the Wellcome Trust.

Societies

The International Mind, Brain, and Education Society (IMBES) aims to further our knowledge, as well as create and identify useful resources. IMBES also holds a conference roughly every two years that attracts researchers from around the world as well as teachers.

Special Interest Group 22 (Neuroscience and Education) is part of a wider organisation, the European Association for Research on Learning and Instruction (EARLI). A group conference is held every two years, and in the intervening years there are EARLI conferences that bring together all special interest groups.

Flux is a developmental cognitive neuroscience society that encourages translational research in education and other fields.

Wednesday, 12 April 2017

Designing a cognitive training study

Cognitive training is a hot topic in educational neuroscience. Can training a certain cognitive function lead to gains in academic performance? This is an exciting question for researchers who (a) want to see real-world impact of their research, and (b) aim to use training as a tool to further inform their theories. But what makes a good training study, and what are the key aspects to be considered throughout the design process? This is the first in a series of posts that examines the key aspects of designing a cognitive training study.

An important consideration is the type of training programme. Will the programme provide practise of difficult tasks (process-based training), or will it train a new strategy to bring to the task (strategy-based training)? Repetition of a task through process-based training may lead to increased automaticity and efficiency, while a new method learnt through strategy-based training may enable a toolkit approach where students can choose the best tool for each problem.

Taking one example, a training programme might aim to improve working memory, since this is known to be important for many academic outcomes. Process-based training would see the student practise working memory tasks, perhaps in an adaptive programme that gets harder or easier depending on performance. On the other hand, strategy-based training would provide explicit explanations of how to perform in the task.

A third approach, that can be considered a type of strategy-based training, is to train metacognitive knowledge. This time, the student might be given a mechanistic explanation of why working memory is so important in their academic studies. They might be explicitly told when to use working memory. Here the aim is not necessarily to train working memory, but rather to train the use of working memory within a certain context. Perhaps a student has adequate working memory but has not previously considered its use in this subject domain. Metacognitive training might allow the student to identify when working memory is needed, and to implement an appropriate strategy.

In the process- and strategy-based approaches we would expect to see an improvement in working memory. This in turn might lead to improvements in academic performance. Conversely we might not expect any working memory improvement through a metacognitive approach, but we might nonetheless see an academic improvement.

Perhaps then, the most effective approach would be to combine all three of the above: Provide repetition of the task, train specific strategies, and increase metacognitive awareness of the cognitive functions involved in a task or subject domain. In terms of educational outcomes, this might be the most likely to show an impact. The challenge for the researcher is that in providing all three, we are no closer to discovering what the “active ingredient” causing change is.

The final consideration in choosing the type of training study is that different methods may be effective for different learners. Perhaps some students require the process- and strategy-based training to improve their baseline working memory ability, while other students already have very good working memory but might benefit from metacognitive training to help them identify when to use this ability. Therefore the type of training programme might depend on the population that the programme targets.

These considerations highlight the importance of designing the training programme from a cognitive theory. While the ultimate aim of educational neuroscience is of course to improve education, as scientists, researchers must use their theory to choose the best type of training programme to answer a particular question. Simply providing training and hoping for a positive outcome is not enough. The outcome must drive theory forward, and the training programme must be carefully designed to enable this. For the scientist, an important result is one that can tell us about the mechanism behind change, rather than one that shows improvement without a good theory about why this change occurred.

This post was informed by this highly recommended article:

Friday, 7 April 2017

Summary of our "Neuroscience in the Classroom" conference

Head over to the Centre for Educational Neuroscience blog to see a summary of the day from Alex Hodgkiss who was the main organiser of the event. The day was a huge success, with almost 100 people in attendance, and almost a further 100 on the waiting list! There is huge appetite for new educational neuroscience research findings, from a wide range of individuals. We had researchers, teachers, and representatives from charities and organisations in attendance.

The whole event was filmed, so we will be making the videos available soon. I hope that the enthusiasm for the day will translate to more events like this, and lead to more conversations between teachers and researchers. Thank you to everyone who attended for your active engagement.

UPDATE: Videos of the day are now available here.