How can evidence-based research answer the following question:
Are there forms of professional development or learning processes which, when applied to school teaching and learning contexts, make significant contributions to demonstrably improved student learning outcomes?
Some basic principles that I suggest should underpin all quality policy development in education
It is certainly true that evidence-based research must underpin any authentic response to the fundamental question posed above, and this will be the principal focus of this paper. But from a broader policy perspective, evidence-based research is an example of when what is ‘necessary’ may not be ‘sufficient’. Already completed and published research is but one of at least four inter-dependent and interrelated fundamental sources of information and understanding that need to be heeded.
The second fundamental source is scholarship – ie. the ideas, speculation, imagination, creativity, innovation and so on – generated and articulated by thinkers who would not fit into the mould of evidence-based researchers.
The third is the wisdom distilled from the reflection over their experience by excellent teachers, principals, and other school leaders who may never have undertaken evidence-based research, who may never have published in the scholarship genre, but who are able to abundantly irrigate educational theory and practice because of their own reflections on their expertise and experiences.
The fourth is practical, good old fashioned strategic nous, which might be described as that down to earth, insightful, flexible exercise of common sense, while fully aware of the complexities of the relevant context.
Some reflections on research
There is considerable educational research that merely confirms what good teachers, principals, and educators in many contexts have known or suspected for quite a while. For example, the research that has demonstrated that the quality of teaching is the most significant within-school factor in the quality of student learning, and that within-school differences are often more significant than between-school differences. These are really ‘no brainers’ these days.
When seeking to establish any compelling link between cause and effect in research, it is always important not to confuse cause with correlation.
When reading the outcomes of any particular piece of educational research, it is necessary to stress the importance of context when assessing the value of that research. For example, one should generally respond cautiously to any black and white research pontifications about the significance of any one, isolated, factor within the rich and diverse landscape that constitutes teaching and learning.
We must carefully exercise our critical powers when reading research. The questions that should arise include the following: Who undertook the research? What is their reputation? What was the purpose of this research? What was its context? What methodology was used? What were any underlying assumptions? Who funded the research? Who may have benefitted from it? What data was included? Was data excluded? How is the research intended to be used?
Teacher ‘professional development’ or teacher ‘professional learning’?
As in many areas of education, it is important not to get too caught up in semantics. For example, striving for any black and white distinction between teacher ‘professional development’ and teacher ‘professional learning’ can be counter-productive. Any professional development that does not involve professional learning – and any professional learning that does not involve professional development – is not worth very much at all. For the purpose of this short paper, I am focusing on the professional development of teachers through learning. I will use the expressions professional development, professional learning, professional development / learning and professional learning / development interchangeably in this paper.
False and misleading dichotomies
One of the features that too often bedevils educational theory and practice is asserting or imposing dichotomies where they do not exist.
What are known as the ‘Literacy wars’ provide classic examples of this very thing. Too often acolytes of ‘gurus’ hurled abuse (however scholarly phrased) at each other without fully understanding the depth and nuances of the theories or practices they were inveighing against. What one side claimed that the other side ignored, may be found, on close forensic inspection, to be untrue – and vice versa. Another tactic not infrequently used is the ‘strawperson’ extremist misrepresentation of the opponent’s position, which is then rather easily demolished. Again, not infrequently it is the campaign waged by acolytes of notable ‘gurus’ which is more aggressive, even combative, than the positions taken by the originating researcher or scholar.
Incidentally, in some of the adversarial discourses within education in general, one sometimes hears the almost dichotomous assertion – made either in defence or attack – that ‘it works all right in theory, but it doesn’t work in practice’. I have always held the view, however, that if the theory does not work in practice, then there is something wrong with the theory; or the practice has not properly applied the theory; or a combination of both.
Teacher professional learning / development – a personal view
In general, professional learning / development for experienced teachers, (early career teachers are not the focus of this paper) to assist them to become better professionals, will be characterised by at least an appropriate richness and rigour of understanding of content; expertise in engagement of students in their learning; diversity and flexibility in pedagogical practice; and a thorough understanding of the importance of purpose and context in learning. This will all be true whether the learning is being undertaken by the teacher herself or himself, or if it is being provided by the education system within which the teacher exercises her or his professionalism.
I believe that there are two major dimensions in the theory and practice of teacher professional development / learning:
• Like any member of a profession, every teacher or principal has a responsibility for their own professional development. For example, secondary school English teachers have a personal professional responsibility to keep abreast of research and scholarly developments in their field. Such professional development / learning can take a variety of forms.
• Those who employ teachers have a responsibility to provide systemic professional development / learning for their teachers and principals when significant systemic change is undertaken. For example, the NSW Department of Education recognises and accepts its responsibility to provide appropriate professional learning for those with the responsibility of implementing new systemic policies. Similarly, such professional development / learning can take a variety of forms.
The phrase ‘life-long learning’ has become almost a contemporary mantra; if not, indeed, a cliché. But it has validity. For teachers and principals this can be expressed as ‘career-long learning’. Apart from anything else, those who seek to have their students learn must be learners themselves. Learning is not a static process – either for teacher or student.
Teacher professional development / learning – analogies with student learning
As far as student learning is concerned, it is now the increasingly accepted view that there is no one silver bullet form of teaching and learning. Here are some examples of what can be components of effective pedagogy:
• ‘Stand and deliver’– for example, a teacher giving a lecture to combined classes of Year 10 students studying Macbeth – can be one perfectly legitimate component of a diversified teaching / learning strategy.
• A ‘typical’ classroom lesson – provided the teacher is both thoroughly familiar with the content and able to engage the students effectively, and which incorporates a range of teaching / learning strategies – is another string to such a flexible strategic bow.
• Small discussion groups of students – properly set up and monitored by the teacher, with clearly enunciated principles, processes of engagement and authentic forms of assessment – also feed into the mix.
• Students focusing on their work in pairs, also has a place.
• As does, of course, a student working on her or his own – in whatever learning space this may occur; whether it be the classroom or the library or under a tree, or at home.
Any one particular approach consistently applied in isolation, is not, sui generis, the silver bullet. And then when you cross-reference or irrigate this (or any more extensive mixture) with more macro pedagogical approaches – for example, problem-solving, or project-based methodologies, etc – other possibilities come into play. When the considerable array of potential learning and teaching flexibilities and synergies generated by multi-media information communication technological platforms are overlaid on all of the above, the possibilities are rich indeed. But at the same time, such technological wizardry in no way removes the timeless educational need for discernment, curiosity, knowledge, understanding, skills, values and all those other characteristics of quality education articulated and outlined, for example, in the Melbourne Declaration on Educational Goals for Young Australians.
Two perspectives from just over a decade ago on the issue of evidence-based research for identifying features of high quality professional learning
In 2004 Hilda Borko (in her paper ‘Professional development and teacher learning: mapping the terrain’) was trenchant in her criticism of the quality of professional development then available to school teachers in the United States of America. She lamented the absence of high quality, evidence-based research to underpin professional development. Indeed, she cited Sykes, who characterised the inadequacy of conventional professional development as ‘the most serious unsolved problem for policy and practice in American education today’ (Sykes 1996, p. 465). The premise of my paper is that it still remains a ’serious unsolved problem’ for educational research.
Ingvarson, Meiers and Beavis, 2005
A year after Borko published her article, the Australian scholars Lawrence Ingvarson, Marion Meiers and Catherine Beavis – in the introduction to their article ‘Factors affecting the impact of professional development programs on teachers’ knowledge, practice, student outcomes and efficacy’ (2005) – noted the vital role played by professional development in ensuring quality teaching and learning in schools, and the increased interest in research that identifies features of effective professional learning. They called for more sophisticated methods for evaluating professional development programs, and argued that the previous approach of distributing questionnaires at the door no longer suffices. However, their paper made no explicit call for evaluating the efficacy of professional development for teachers in terms of student learning outcomes; or, in the case of professional development experienced by principals, improved learning outcomes across their schools.
More recent perspectives
A reading of the articles by Borko and by Ingvarson, Meiers and Beavis raises the question as to what evidence-based research there should be to demonstrate the efficacy of various forms of professional development / learning – other than surveying teachers to see what they believe to have been the better or best forms that they have experienced? Of course this subjective assessment provides one legitimate source of information.
But it would clearly be of value if there could be some form or forms of evidence-based research that could show that improved development / learning outcomes of their students could be attributed to improved professional development / learning outcomes of their teachers.
Timperley’s booklet Teacher Professional Learning and Development (2008) is based on a synthesis of research evidence produced for the New Zealand Ministry of Education’s Iterative Best Evidence Synthesis (BES) Programme, which was designed to be a catalyst for systemic improvement and sustainable development in education.
At the outset, Helen Timperley sets out the aim of this relatively short work as follows: ’The focus of this particular booklet is on the interrelated conditions for professional learning and development that impact positively on valued student outcomes’ (pp. 6-7).
Before the concluding chapter, Professor Timperley observes that ‘Sustained improvement in student outcomes requires that teachers have sound theoretical knowledge, evidence-informed inquiry skills, and supportive organisational conditions’.
The basis of this publication consists of 10 inter-dependent principles for driving effective forms of teacher professional development / learning – each of which is explored in each of the chapters. These principles are as follows:
Chapter 10, ‘Maintaining momentum’, commences with the following highlighted statement: ’Sustained improvement in student outcomes requires that teachers have sound theoretical knowledge, evidence-informed inquiry skills, and supportive organisational conditions’ (p. 24). What follows in Timperley’s Chapter 10 warrants direct quoting:
Regrettably, most efforts to improve student outcomes through professional learning and development are short-lived. For improvement to be sustained, short-term perspectives need to be extended to more distant horizons.
Although the research base identifying the conditions associated with long-term improvement is somewhat thin, one thing does appear clear: sustainability depends both on what happens during the professional learning experience and on the organisational conditions that are in place when external support is withdrawn.
The professional learning experience
A sustained improvement in student outcomes depends firstly on teachers developing strong theoretical frameworks that provide them with a basis for making principled changes to practice in response to student needs. When confronted with specific teaching–learning challenges, teachers can go back to the theory to determine what adjustments they need to make to their practice.
Sustained improvement also depends on teachers developing professional, self-regulatory inquiry skills so that they can collect relevant evidence, use it to inquire into the effectiveness of their teaching, and make continuing adjustments to their practice. Teachers with these crucial self-regulatory skills are able to answer three vital questions: ’Where am I going?’, ’How am I doing?’, and ’Where to next?’ The answer to the ’Where am I going?’ question is sometimes referenced explicitly to national or state standards; more often it is found in, for example, improvements in students’ mathematical problem solving or text comprehension. The answer to the question, ’How am I doing?’ is a measure of how effective teaching is in terms of student progress. The answer to the ’Where to next?’ question is guided by a detailed and theoretically sophisticated knowledge of curriculum content and student progressions.
Continued forward momentum also depends on an organisational infrastructure that supports professional learning and self-regulated inquiry. It is difficult for teachers to engage in sophisticated inquiry processes unless sitebased leaders reinforce the importance of goals for student learning, assist teachers to collect and analyse relevant evidence of progress toward them, and access expert assistance when required (pp. 24-25).
Towards the end of this fairly short publication, Timperley lists six questions or issues underpinning the set of principles, each of which forms the basis of each of her preceding chapters. These questions or issues are as follows:
• What educational outcomes are valued for our students and how are our students doing in relation to those outcomes?
• What has been the impact of our changed actions on our students?
• Engagement of students in new learning experiences.
• What knowledge and skills do we as teachers need to enable our students to bridge the gap between current understandings and valued outcomes?
• How can we as leaders promote the learning of our teachers to bridge the gap for our students?
• Engagement of teachers in further learning to deepen professional knowledge and refine skills (pp. 26-27)
The following is what Professor Timperley considers to be the 10 principles for driving effective forms of teacher professional development or learning. She emphasises that the 10 principles ‘do not operate independently; rather, they are integrated to inform cycles of learning and action’ (p.28).
1. Focus on valued student outcomes
Professional learning experiences that focus on the links between particular teaching activities and valued student outcomes are associated with positive impacts on those outcomes.
2. Worthwhile content
The knowledge and skills developed are those that have been established as effective in achieving valued student outcomes.
3. Integration of knowledge and skills
The integration of essential teacher knowledge and skills promotes deep teacher learning and effective changes in practice.
4. Assessment for professional inquiry
Information about what students need to know and do is used to identify what teachers need to know and do.
5. Multiple opportunities to learn and apply information
To make significant changes to their practice, teachers need multiple opportunities to learn new information and understand its implications for practice. Furthermore, they need to encounter these opportunities in environments that offer both trust and challenge.
6. Approaches responsive to learning processes
The promotion of professional learning requires different approaches depending on whether or not new ideas are consistent with the assumptions that currently underpin practice.
7. Opportunities to process new learning with others
Collegial interaction that is focused on student outcomes can help teachers integrate new learning into existing practice.
8. Knowledgeable expertise
Expertise external to the group of participating teachers is necessary to challenge existing assumptions and develop the kinds of new knowledge and skills associated with positive outcomes for students.
9. Active leadership
Designated educational leaders have a key role in developing expectations for improved student outcomes and organising and promoting engagement in professional learning opportunities.
10. Maintaining momentum
Sustained improvement in student outcomes requires that teachers have sound theoretical knowledge, evidence-informed inquiry skills, and supportive organisational conditions (pp. 8-25).
In outlining her principles for professional development or learning, Timperley hits the nail on the head. She points directly at the need to link professional development or learning with demonstrable student learning outcomes. However, in subsequent years it has been difficult to find any comprehensive forms of evidence-based research that have been able to demonstrate improved learning outcomes of their students that could be attributed to improved professional development or learning outcomes of their teachers.
In his research report Building a high-quality teaching profession: lessons from around the world (2011), Dr Andreas Schleicher – well-known internationally as the OECD’s Special Advisor to the Secretary-General on Education Policy and Head of the Indicators and Analysis Division at the Directorate for Education – provided a critiqued collation of some evidence-based international research on the efficacy of professional development. One of the largest international surveys of teachers and school principals, the report was based on data from over 70,000 teachers and school principals from lower secondary teachers in the 23 participating countries. In reference to the report, Schleicher noted that ‘relatively few teachers participate in the kinds of professional development which they find has the largest impact on their work’ (see Figure 3.16 in his presentation - included in the bibliography).
Referring to results from the 2009 Teaching and Learning International Survey (TALIS), he noted that those professional development opportunities rated by teacher participants as having the highest impact on their work were: individual and collaborative research; qualification programs; informal dialogue to improve teaching; reading professional literature; courses and workshops; and professional development networks. But those with the highest levels of participation were informal dialogue to improve teaching; and courses and workshops.
However, the efficacy of all of these professional development programs was determined only by the aggregation of each participant’s own self-assessment of the ’impact’ of the professional development or learning on their teaching. No evidence was provided – or at least reported – on the efficacy as demonstrated by the improved learning of their students.
Early in the monograph A significant and direct impact by teachers and school leaders on student learning outcomes in literacy (2012) written by Ann McIntyre, following discussions with her colleagues in the NSW Department of Education and Training (as it was then known) Professional Learning and Leadership Development Directorate (of which she was the Director), references were made to then recent research on the impact of school leaders and teachers on student learning – and the link to professional learning or development. McIntyre noted that ’nearly 60 per cent of a school’s impact on student achievement is attributed to principal and teacher effectiveness’ (McIntyre, 2011, p. 9, citing McKinsey, 2010, p. 5, which itself cited Barber, Whelan and Clark, 2010) and that there is an interdependent link between student and school improvement and professional learning (cf. Elmore, 2006; Robinson, 2007).
McIntyre proceeded to argue that successful school improvement is dependent upon the capacity of a system to successfully undertake the following four actions:
1. Research and analyse the practices of school leaders and teachers that have the most significant impact on student learning.
2. Develop professional learning programs that articulate and promote these practices and develop teacher and school leader capacity to implement this learning in the context of their school and classrooms.
3. Analyse the learning needs of students, develop clear targets for school and classroom action and implement coherent, aligned professional learning strategies to build both teacher and school leader capacity to improve student learning.
4. Implement school improvement systems that enable the alignment of teacher and school leader learning to student learning.
(cf. McIntyre, 2011, pp. 48-49)
It is worth noting that Dr Ben Jensen, in his analysis of PISA-successful East Asian countries and cities, emphasised the importance of the practical implementation of creating a strong culture of teacher collaboration and mentoring; teachers observing and providing feedback on their colleagues’ teaching; sustained high quality professional development; and highly focused research on the learning development of students within their classrooms (Grattan Institute, 2012).
The paper Teacher quality evidence for action (ACE 2013) by Ann McIntyre is an example of a fine piece of recent research based on surveys of a large number of public school teachers (approximately 6,000) who provided their own self-assessment of the relative value of a range of professional development or learning ‘programs’ they had experienced.
The six key elements that had the greatest impact for primary teachers were, in order of influence:
• the collaborative preparation of lessons and teaching resources,
• lesson observation and observing each other’s lessons,
• the collaborative assessment and evaluation of student work,
• structured feedback meetings,
• developing evidence to demonstrate the achievement of professional teaching standards,
• team teaching.
McIntyre noted that the benefit of teachers working together highlights the importance of reframing activities within schools to ensure that schools are not only places for students to learn but also places for teachers to learn. Structuring time within schools to enable lesson observation and feedback and the collaborative development and evaluation of lessons collectively provide a significant source of professional learning for teachers.
Similar responses were given in a study that focussed on teachers who were described by their principals as being accomplished and of high quality.
While, by and large, these forms of professional development or learning can also be found in Dr Andreas Schleicher’s collation, what does not appear in this particular research undertaken by McIntyre is any reference to forms of professional development or learning delivered and/or experienced external to the school itself. This can be explained, however, by the fact that the policy of the former NSW Department of Education and Training had become committed to a model of funding professional learning programs fully devolved to, or within schools – which had to be ‘driven’ by one or more of a set of professional learning themes set down by the Department. Therefore, McIntyre’s research, which surveyed approximately 6,750 teachers – conducted as part of evaluation of use of teacher professional learning funds – had its dominant focus on within-school professional learning processes.
As a consequence, and to reiterate, none of the forms of external professional development or learning found to have an important impact on teachers in Schleicher’s research – individual and collaborative research; qualification programs; reading professional literature; courses and workshops; and professional development networks – are recorded as having a significant impact on the teachers in McIntyre’s research. However, Individual and collaborative research was one of the strategies that was evidenced in the analysis of student learning and approaches to lesson planning and observation. Within McIntyre’s research she found that Schleicher’s professional learning categories did not sit well with her research as her focus was, as it always has been, on professional learning as an outcomes-driven process rather than an input-driven process. This was made very clear in McIntyre’s (as yet unpublished) research paper Teacher learning to improve student learning: professional development policy and impact in NSW, Australia, presented at the Annual General Meeting of the American Educational Research Association, Chicago, in April, 2015.
In a subsequent piece of research undertaken for the NSW Primary Principals’ Association which commenced in 2013, Ann McIntyre set out to identify evidence to demonstrate the efficacy – or otherwise – of the professional learning experienced by the principals involved in the pilot of a new principal credential for the Association. The project was modelled on her research regarding the elements of professional learning that were most likely to impact practice. The program involved face-to-face evidence-informed learning seminars, mentoring by successful practising principals, and action learning – over the 18 months’ duration of the project. She planned that at the conclusion of the research project, participants would be required to present evidence that included the ‘School improvement challenge’, the ‘Performance and development plan’ and an ‘Executive summary’ that would outline their learning as leaders through the learning development processes they had implemented in their schools as a result of the professional learning processes. Essential to this concluding document would have to be their demonstrable evidence of efficacy through the provision of evidence aligned to both the Australian Professional Standard for Principals and the key accountabilities of their role. Results of this research were not available at the time this particular paper was written. When published it should provide demonstrable evidence of improved learning as a result of identified professional learning processes.
As indicated in the note below, this project commenced in 2014. Therefore, generally speaking, much of the research, but not all of it, that is cited in the preceding text, does not go beyond 2013. Even from the fairly brief collation of research identified and discussed in this paper, what does clearly emerge is the need for authentic, evidence-based causal (not merely correlational) links between the provision or experience of identified professional development or learning processes, and demonstrable student learning outcome effects that can be directly attributed to those professional development or learning causes.
A word from the author, 17 November 2015.
I commenced drafting this paper in August 2014. For a number of reasons, its completion was delayed. Most of the original draft remains. The first eight paragraphs of this paper can be found in my article, ‘Show an Affirming Flame: A Message to the Profession’ in the Journal of Professional Learning, https://cpl.asn.au/journal/semester-2-2015/show-an-affirming-flame-a-message-to-the-profession