Evaluating practices in relation to outcomes

This page summarises some thinking and research related to this capability. It is not a comprehensive or definitive background to the capability.

Evaluating practices in relation to outcomes underpins other leadership capabilities, for example, strategic thinking and planning, and adept management of resources.

Assessment, evaluation and inquiry 

Systems that perform better are systems in which assessment, inquiry and evaluation information is used effectively to inform learning. 
(OECD, 2013)

In New Zealand, the Education Review Office's School evaluation indicators include Domain 6: Evaluation, inquiry and knowledge building for improvement and innovation – see pp 40-41.

Evaluation means making a judgment about the quality, effectiveness or value of a policy, programme or practice in terms of its contribution to the desired outcomes. (ERO, 2015a)

Professional inquiry is an integral part of the evaluation process. Evaluation, inquiry and knowledge building processes work together to foster the use of relevant information at student, classroom, teacher and school levels to promote improvement. (ERO, 2015a)

Evaluative thinking

Evaluating practices involves thinking about:

• what evidence will be useful to demonstrate success
• what range of objectives and targets make sense to determine progress
• what processes will build knowledge and develop practical uses for it.

At the core of evaluative thinking is a process of considering the evidence, in relation to the questions that prompted its collection, and engaging in careful inquiry and interpretation. 

(Earl and Timperley, 2015) 

Using evidence

School effectiveness is associated with effective use of evidence at student, teacher, classroom and school levels.

Leaders play a key role in this. Where leaders do not make evidence use a priority, teachers are less likely to use it (Louis et al, 2010, in Ontario Ministry of Education, 2013).

An inquiry habit of mind

Earl and Katz (2003) describe the inquiry habit of mind as:

• valuing deep understanding
• reserving judgement 
• tolerating ambiguity
• taking a range of perspectives
• posing systematically more focused questions.

Data literacy

Data literacy is not just about numbers. It is a process of:

• standing back and deciding what you need to know and why
• collecting or locating the necessary data
• finding ways to link key data sources, ensuring that the data are worth considering
• being aware of the limitations of the data
• thinking about what the results mean
• systematically considering an issue from a range of perspectives so that you feel you have sufficient evidence to explain, support, and also challenge your point of view.

(Earl and Katz, 2006)

A culture of inquiry and evaluation

School leaders create an inquiry culture when they:

• make evidence use a priority
• establish the purpose(s) for collecting and using evidence
• use data in non-threatening ways
• provide planned opportunities and time for working with evidence
• engage staff in collaborative interpretation and decision making. 

(Ontario Ministry of Education, 2013)

Making sense of evidence

While data provides lenses for looking at a situation, it doesn't generate questions, answers or solutions.

Leaders and teachers need to interpret what the data might be signalling in their context. They need to learn to uncover patterns, generate hypotheses and systematically test these in a range of ways with a range of audiences.

It is not an easy job and involving a range of voices and expertise helps avoids the cognitive biases that can limit how well people respond to new information.

Collaborative sense making also involves drawing on research and using suitable frameworks or indicators to analyse data. It requires that we know what good looks like, so we can recognise our strengths and areas for improvement. (ERO, 2015b)

Related pages

Evidence-based leadership – articles and stories

Spiral of inquiry: Leaders leading learning – guide

Theory for improvement – tool

References

Earl, L. Katz, S. (2003). Leading schools in a data-rich world. 

Earl, L. Katz, S. (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Thousand Oaks, CA: Corwin Press.

Earl, L. Timperley, H. (2015). Evaluative thinking for successful educational innovation. OECD working paper.

Education Review Office. (2016). School evaluation indicators. See pp 40-41 Domain 6: Evaluation, inquiry and knowledge building for improvement and innovation.

Education Review Office. (2015b). Effective school evaluation: How to do and use internal evaluation for improvement

Education Review Office. (2016). Effective internal evaluation for improvement 

OECD. (2013). Synergies for better learning: An international perspective on evaluation and assessment

Ontario Ministry of Education. (2013). Using data: Transforming potential into practice. Ideas into Action series.

Image credit: Markus Spiske, Color, pattern, magnifier and purple, on Unsplash

Tell a colleague | Back to top