This is one of the articles in our National Teacher Enquiry Network December end of term newsletter (sign up here).
Not a day goes by without an educational commentator lamenting the lack of evidence-based thinking, policy or practice in our school system. If only everyone made evidence-based decisions, the thinking goes, then we would could reach some sort of optimal consensus and tread merrily along the road to educational utopia.
Teachers have been subjected to all sorts of ‘research-based’ advice, from Assessment for Learning to the use of teaching assistants, from ways to ask questions to whether to set by ability. The initial hope that this will lead to a revolution in effectiveness has been endlessly disappointing.
The reason is that every teacher, every pupil and every classroom is different from every other, and also that the methods of transmitting research have fundamentally underestimated the daunting barriers that exist when trying to persuade people to take on new ideas. The solution is allow ideas to be implemented and adapted locally while also trying to retain the characteristics of the original findings. That’s a big claim, and needs some further analysis, which I attempt to lay out in this article.
In Nutley, Walter and Davies’ excellent book Using Evidence, they set out a number of reasons that people commission research in the first place. I’ve summarised and slightly extended this list here, although this is by no means a complete list:
- to justify existing ideas or future policy
- to delay decisions
- to discover existing practice
- to understand the background behind a decision or practice
- to understand the culture of a group or organisation
- to solve a specific problem
- to evaluate the success of an approach or intervention
- to broaden understanding around a new finding
- to improve practice
- to synthesis or summarise other research
- to refine an existing idea
- to pursue an intellectual interest
The reason for commissioning immediately sets the tone of the research, and may be enough to lead some to dismiss or embrace its findings, regardless of the quality of the methodology or validity of the conclusions. However, it’s only one of a complex web of factors that affect the likelihood of people being receptive.
A key idea about research use is that values are immensely important. Weiss (1995) suggests that ideology and interests are only weakly influenced by research, and that research is less likely to be used where there is strong agreement between ideologies, interests and information from other sources. Indeed, not only are you likely to make value-judgements about the researchers’ credibility based on their previous findings and work, but you will be less likely to accept their findings if they do not align with your own ideology.
Personally, I find that this problem is rife in all discussion about evidence and research. It is easy to assume you are open to new ideas, but we all view research through our own personal lens, which is affected by:
- our ideology (is this aligned with my values?)
- own self-interest (will this finding help me or hinder me? Is it consistent with ideas or approaches that I’ve argued for? Would I lose face if I accepted this idea?)
- our experiences (does this finding match what I’ve seen or experienced?)
- our relationships with the researchers (do I know the personally, are they like me, do I trust them?)
- other people’s perceptions of the researchers (will I gain credibility by listening to their findings? Could these researchers be undermined by others?)
- our history of engaging with research (do I trust academics, do I see myself as open to research?)
- our willingness to change (affected by feelings of security, self-efficacy, stress etc.)
- the way in which the research is written (is this written in a way I can understand, using my normal mode of communication? Is this written in a compelling way that is easy for me to communicate to others?)
- the traditions and culture of the institution we work in (would this work/be acceptable here?)
- competing priorities (do I have time for this? Is it a priority to engage with this now? Do I have the resources to engage with this fully?)
- competing sources of information (would I prefer to accept a different conclusion stated elsewhere? is it easier to get information from another source?)
- the timeliness of the research (is this relevant to the decisions I’m making now, or is it now conflicting with decisions I’ve recently made?)
With such an enormous number of factors affecting the way research is viewed, it’s really not surprising that even the simplest of findings can be controversial. It also means that simply presenting research findings and assuming that this will lead to their uptake is hopelessly optimistic. Indeed, the least effective ways for implementing research findings are through sending printed resources or making people listen to lectures. Even where people are open to the new ideas, these methods are entirely insufficient to change ingrained habits of practice, challenge existing beliefs and misconceptions, establish new understanding and create new ways of viewing problems.
This shouldn’t really come as much of a surprise in education. We know that one of the best ways for children to take on a new idea is through actively engaging in problems after seeing a technique modelled, continually refining the idea with frequent and timely feedback and support, and then subjecting the new idea to increasingly challenging problems to deepen understanding and to link it with related knowledge.
This is a world away from traditional methods of disseminating education research and ‘best practice models’, and so it isn’t entirely surprising that the drive to embed AfL led to superficial ‘three part lessons and everyone knows their sub-level’ implementations, that schools are still using Pupil Premium money to reduce class sizes instead of more effective strategies to raise attainment, and that only a tiny percentage of continuing professional development provision has been found to be effective in transforming classroom practice.
People are failing to implement new ideas due to ideology, trust, time, resources, habit, or one of many other reasons. Simply announcing new ideas is insufficient.
So, how should we be embedding new ideas in teaching practice? A few key findings stand out (again from Using Evidence) to inform the process:
- Teachers generally value their colleagues’ opinions of the usefulness of research over traditional measures of research quality – simply labelling something as coming from a randomised control trial won’t necessarily improve uptake, for example.
- Teachers will tend to rate the credibility of local research more highly than other findings, and will be significantly more likely to take on ideas if they match their personal experience or that of trusted colleagues.
- Academic language is generally off-putting and acts as a barrier to teachers taking on new ideas, especially where they themselves haven’t engaged in research.
- The institutional culture is important – if there is an openness to new ideas and a celebration of change and learning then new ideas are more likely to be taken on.
- Where teachers engage with researchers then it is important to develop a trusted working relationship over a long period. It is important that teachers feel that there is a two-way dialogue, that their own findings and opinions are taken on board by researchers as well as vice-versa.
- It is more likely that a gradual evolutionary shifts in practice will lead to sustained change in teaching, instead of attempting a revolutionary shift.
- It is important that engagement with research is properly facilitated with time, resources and support. The most common barriers are heavy workloads, multiple competing pressures, continual demands for system change and a culture that doesn’t value learning or mistakes.
All of these points suggest that a successful process would be peer-led, sustained over several months, practice-based and in dialogue with a trusted external researcher. It also suggests that the very instrumental model where researchers identify optimal practice and then send it to schools to be implemented is naive. We need ongoing dialogue and continual refinement. As it happens, this is exactly the process that was also found by various EPPI studies to be the most effective form of professional development.
So, taking in to account all these findings, the characteristics of the most successful mechanisms of knowledge transfer from research to teaching practice are:
- Research from trusted sources presented in teacher-friendly language, and relevant to the values and aspirations of the participating teachers.
- Collaborative teacher enquiry used to try out, adapt and refine approaches from the research. Successful local adaptation will lead to significantly increased trust and uptake within a school.
- Adequate time, resources and support made available to the participants of the enquiry.
- An ongoing relationship with the external researcher or expert to inform the process, provide two-way dialogue, and to engage with the underlying theory
- A sustained process (lasting at least two terms) with time to repeatedly experiment with, adapt and refine ideas within a classroom setting (or other relevant practice setting such as one-to-one or small group work).
- A culture which celebrates learning and research-engagement within school and where teachers are allowed to take on ideas themselves, allowed to make mistakes, and are supported in their ongoing learning.
- Leadership that minimises endless change and demand to give space to experimentation and reflection, and which supports and nurtures effective teacher collaboration.
- A recognition that one-size does not fit all- i.e. that the effectiveness and validity of research will vary with different school contexts and with different practitioners and that there will be no ‘magic-bullet’
- A constant and ongoing tension between the need to adapt research findings to local context, practices and values and the need to change some values, habits and practices to ensure effective implementation.
- Ongoing evaluation of effectiveness to be sure that the new idea is having a genuine, positive effect, while understanding that it is extraordinarily difficult to reach statistically significant conclusions from the small samples found in schools.
Even with all of these characteristics in place, it will still be a very difficult to embed new ideas, with endless distractions, difficulties and competing priorities. The changing needs of pupils, the turnover of staff, and the constant change in externally-imposed requirements will mean that there will never be a clear end point to the implementation of any piece of research.
The only reasonable solution is to develop a culture in schools where teachers take part in collaborative planning and enquiry as a matter of course, where they engage with research to inform and inspire but maintain a focus on a small number of sustained changes rather than attempting hundreds of different ideas across the school at any one time. This needs to sit within a school culture where teachers act as professionals, driving their own practice forward with trust from leadership but balanced with enough accountability to keep the pressure to move forward and make genuine improvements without stifling innovation.
At the same time, researchers need to develop a culture where findings are not simply broadcast to schools, but where they engage with increasing numbers of schools to find out how to successfully adapt the approach in different contexts, how to overcome different challenges, and how to successfully combine the idea with other priorities in the classroom.
That’s incredibly hard, but when it works right you create a self-improving school system that drives better practice from the ground-up, where teachers become engaged with research, and where researchers become actively involved in dissemination and adaptation.
Here at the Teacher Development Trust we are developing tools and processes to support this ideal, with our GoodCPDGuide database to identify experts and ideas and our National Teacher Enquiry Network to encourage in-school collaborative enquiry. We are actively campaigning for more intelligent use of research and evidence in schools, and supporting policy-makers and schools leaders to make better choices when it comes to professional development. Please join us and contribute to these efforts.