KEYNOTE SPEAKER - MARTIN WALKER
Martin Walker is involved in teaching and research into the effective use of assessment to improve learning. He has worked with Principals Academy in Singapore, delivering courses on validity in assessment and applications of test theory to schools in the region. Martin has also worked with the official regulators for testing in England, Wales, Scotland and and Northern Ireland. He advises several companies in the City of London of aspects of professional examinations in finance, banking and the pharmaceutical industry.
Martin has taught English and physics in a range of secondary schools and colleges in England, and has written numerous books on the teaching of English and English literature. As a teacher he became involved in national examinations, eventually operating at the highest level of national test development in English. During this period he noted that, “I remember thinking that we seemed simply to be making up questions and thinking that if a question felt about right, it must be good”.
With these ideas in mind, Martin studied for an MSc in educational assessment at Durham University from 2009 to 2011, leading to a PhD study of the validity of national examinations for 16 year old pupils in the UK. He helped to set up the UK Chartered Institute of Educational Assessors, and developed many of the Institute’s teaching programmes. His current interest is in exploring national education systems to examine whether what is being taught makes sense in today’s world, and whether the assessment of what has been taught provides useful information about learning.
Besides all that, Martin has climbed in large mountain ranges all over the world, still plays hockey and cricket, and you might find him singing in the bar if there is a guitar handy.
Keynote Session
Thursday 12 September, 3.30pm
Why Do We Assess Students, and What Do We Really Find Out When We Do?
If the purpose of educational assessment is to find out what students have learned and can do, there should be some clear underlying principles behind the development and use of assessments, and a good understanding of the inferences that may and may not be drawn from the results.
Assessment can support learning in various ways, for example in diagnosing specific problems that some students encounter, in checking whether a group of students has grasped what the teacher hopes they have grasped, and in certifying students in some way at the end of a course.
Assessment can be used diagnostically to inform decisions about what a student needs to do next in order to progress. This can be as simple as rectifying a basic misunderstanding that is preventing progress or it might involve exploring complex issues that interact to cause the student to grasp the subject less well than the teacher had hoped.
Whole class approaches to teaching and learning often tend to assume that, because the teacher has explained something once, all the students will have understood it. Good assessment can help to establish the real situation, which is probably that some students have understood everything, whilst others have gaps in their understanding. Poorly written tests, with raw marks simply being added together to give a “score”, provide misleading information, so teachers often need help in constructing tests that can give good information about learners. There is an established body of learning on assessment theory and practice which can help teachers understand concepts such as reliability and validity, and recognise that some academically credible approaches to test design are likely to result in better tests.
At a national level, assessment practice is varied. Many national testing systems still rely on teachers to construct tests, from questions that they have written themselves, and for which the “this is worth 3 marks” but “this is worth 8 marks” approach is common. Instead of guessing the relative difficulty of different questions it is perfectly possible to measure which things are difficult for learners and which things are easy. Constructing tests from questions that are understood, with technical properties that are known and with relative difficulties that are sensible, will result in better tests and therefore better information.
Finally, the uses to which test information is put can be rather surprising. Simple tests, consisting of questions that have not been tested, with raw marks that are simply added, and with levels of reliability that would make us question whether they should be used for any purpose at all, are nevertheless used in many countries to decide the life chances of students. If we think we have taught something, we should be able to find out if students have learned it.
This session will cover the above aspects of assessment and ask why we assess in the first place and what we tend to find out from many of our most common assessment types.
Spotlight Session
Thursday 12 September, 1.30pm
Item Response Theory for Beginners
This session will introduce you to item response theory, in particular the use of Rasch analysis to analyse the performance of test questions. The common educational practice of adding raw scores and then simply stating that student A is better than student B is often so full of error that such pronouncements about individual performance are meaningless.
We will look at ways of measuring the ability of the students and the difficulty of the questions in the test. Only if these two elements are well aligned are we likely to end up with good information about what students can and cannot do. We will consider how IRT approaches can tell us more about our students and our test than traditional methods of test development. The course will introduce you to one specific tool that IRT analysis can provide; the Wright Map. Developed by Ben Wright at the University of Chicago, the Wright Map allows us to see the relative abilities of the students next to the relative difficulties of the questions. Using the Wright Map we can see instantly whether some questions are so easy or so difficult as to be of little use in the test.
This will be a mathematically light session – if you don’t like having to do calculations in public then you will be fine. For the more mathematically minded, we can point you to resources that will explain things in more depth.