Professor Tim McNamara, of The University of Melbourne, is one of the world’s most respected language testing experts and was instrumental in the design, development and launch of OET in the late 1980s. Tim spoke about his recent in-depth research into OET at the TESOL 2019 Convention in Atlanta – we asked him more about the research and its findings.
What were the key drivers for your in-depth research into OET?
With OET, as with all language tests, there is always a concern that we might be ‘getting it wrong’ when test results are used to make decisions about individuals. For example, candidates who have not got the levels they require occasionally write to complain that the test is unfair. We also sometimes receive comments from health professional supervisors who say individuals they are responsible for have been admitted to supervised clinical settings, as a result of OET, who were not ready for the demands of clinical communication.
Years ago, my colleague Sally Jacoby and I explored the idea that if we were getting these OET-based decisions wrong, it might be because the judgements about readiness for the demands of clinical communication weren’t fully reflecting the issues of greatest concern to professional supervisors or candidates.
In her PhD, Sally had noted that professional supervisors, in particular workplace or professional contexts, orient towards specific performance criteria when giving routine feedback to junior colleagues, and she called these routine judgements and feedback, including on professional communication, ‘indigenous assessment’. Sally suggested that an in-depth exploration of indigenous assessment could support the development of more relevant – and therefore fairer – criteria for judging candidate performance in OET speaking and writing tasks.
We then wrote a paper in 1999 which suggested that an analysis of feedback content would show what was considered ‘important’ for communication in a specific setting, and this paper formed the basis of the two OET studies we undertook in Melbourne between 2009 and 2016.
Healthcare must be continuously changing in response to new technologies and processes, and to shifts in the style and culture of healthcare communication – given these changes, how important is research into the effectiveness of OET?
Rigorous research is absolutely central to making tests fair and relevant, particularly for a test as consequential for clinicians and patients as OET.
New insights into the communication demands of clinical settings suggest compelling new research questions, and as a result of our recent research, we have introduced important changes to the criteria used to evaluate performance in spoken and written OET tasks. Changes in technology will continue to affect relevant tasks and evaluation criteria, especially for written clinical communication.
We must also recognise the multilingual and multicultural character of clinical communication, with English as a lingua franca becoming the medium of communication when at least one participant is a non-native speaker of English.
What can English for Specific Purposes (ESP) practitioners outside healthcare, and the wider language testing community, gain from your research?
One of the great advantages of an ESP test such as OET, over more general academic tests such as IELTS, is that preparation for the test is also preparation for the communication demands of the workplace. This is called ‘positive washback’ and means that those teaching healthcare communication can benefit from the insights of those working on tests such as OET, where teaching and testing are integrated and both are targeting the same professional setting. The research itself, involving collaboration between language and workplace professionals, demonstrates both the value of this type of collaboration and how it can be conceptualized and set up.
Our research also has important implications for our understanding of how to develop tests and teaching materials for particular professional and work settings. It takes us out of the purely ‘language professional’ mindset and helps us realize that there are other, equally rich, understandings of communication than those in which we have been trained. As a result, we broaden our appreciation of the demands of professional communication in English and how we may best help those preparing to face such demands.
If you would like to learn more about OET, you can find more information on our Organisation Information page.