Summary: This fortnight Harden’s Blog addresses medical education for the future and the MCAT revisions. Professor Harden also gives an overview of the process of sending 2,300 AMEE 2012 abstracts out for review. Description: I wrote my last blog entry from the Ottawa Conference in Kuala Lumpur. The two final plenary presentations proved an excellent ending to the meeting. AndrĂ© de Champlain looked at some myths about standard setting and Cees van der Vleuten proposed the way forward for assessment. AndrĂ© previously did an excellent webinar on generalizability theory and we will try and persuade him to do a further webinar on standard setting. Some of Cees’ thoughts are embodied in the editorial in the March issue of Medical Teacher along with four articles on assessment in that issue. These are available as free downloads until the end of March for anyone who is not already a Medical Teacher subscriber. Both Cees’ presentation and the presentation by Brian Hodges in the first plenary suggested that we have paid too much attention to psychometrics in assessment and too little to validity and the assessment programme as a whole. During the meeting a number of references were made to the reliability of different assessment approaches. The measurement of the reliability was usually attributed to the method, ignoring the context and how well the method had been implemented. This is a problem. The reliability of an OSCE, for example, is to some extent dependent on how well it is implemented.

There were an interesting series of papers presented at the meeting on the progress test and Megan Quentin-Baxter has written an excellent summary highlighting some of the key points raised in her own blog. Megan’s blog can be found on the Higher Education Academy, Medicine, Dentistry & Veterinary Medicine website:

Preliminary feedback about the meeting has been very positive and plans already are in progress for the 16th Ottawa Conference to be held back in Ottawa. Suggestions for assessment themes that might be addressed in the meeting would be welcome. We have already had some interest expressed from potential hosts for the Ottawa Conference in 2016.

My return to Dundee from Malaysia coincided with the closing date for the submission of the AMEE 2012 abstracts. About 2,300 abstracts had been received. The priority was to identify those concerned with postgraduate education, simulation, e-learning and research, as papers in these topics will go to members of the AMEE committees for review. The other papers will go to an AMEE panel of reviewers. Each abstract will be reviewed by three individuals and a final selection made for the meeting. Looking at the plenaries which are already announced in the programme and at the abstracts we have received, I think we will undoubtedly have another useful meeting. Looking through the 2,300 abstracts is a formidable undertaking but one I find useful in that it gives one a sense of where people see the priorities in medical education and where there is current work on-going in medical education in different parts of the world.

I left on Thursday 22nd for Boston where I was contributing to the Annual Retreat of the Northeast Group on Educational Affairs of AMC. This was held at Tufts School of Medicine. The meeting had a record attendance of 270. On the first day I provocatively argued that we needed a revolution in medical education to meet the needs of the future. In voting at the end of the presentation, the majority of the audience appeared to agree but had different priorities for the direction of the revolution – adaptive learning, a continuum of education or globalisation and outsourcing an education programme. My argument was that investment and thinking in medical education has not been on changing the system but on simple refining of what we do at present. On the second day I talked about outcome-based education and nine myths. This was followed by a panel discussion. Speaking as a member of the panel Peter Katsufrakis from the NBME raised some interesting concerns about the move to outcome based education. He introduced the analogy of communism and wondered if the benefits will materialise in practice. He also wondered if we have the gumption to fail students who do not achieve outcomes, for example, in the attitudinal domain. This provoked an interesting discussion and I pointed out that in Dundee achieving the necessary level of competence in each of the domains including professionalism was required for students to graduate.

For me one of the most interesting parts of the meetings on both days was the time when I was scheduled to meet with a group of 15 students from different North American medical schools. They agreed with the thoughts I had expressed in my plenary about the need for a revolution in medical education, but felt impotent and unable to do anything about it. I suggested they were an important stakeholder and should be part of on-going discussions. The students suggested that in each of their schools there were only a very small number of outstanding lecturers whose lectures they attended. They simply did not turn up at the other ones but listened to recordings. They suggested it would be a great idea if recordings were available of the outstanding lecturers from all of the schools which could then be shared. This was part of the concept of outsourcing that I had introduced in the plenary. We also had an interesting discussion on professionalism and what students should do if they identify a problem with a colleague. One concern was that they did not trust staff sufficiently to report formally problems with a colleague, although some schools were addressing this issue.

The North American emphasis on written assessment manifested itself in reports on the selection for medical schools. An advisory committee appointed by AAMC in 2008 has just reported on a comprehensive review of the Medical College Admission Test (MCAT). This is the fifth revision since the exam was first introduced in 1928. Among the areas where a greater emphasis will be placed will be on the assessment of biochemistry concepts at the level taught in most first semester biochemistry courses and also on students’ knowledge and use of concepts in psychology, sociology taught in introductory psychology and sociology courses. There is a useful description of the revised MCAT at Terri Cameron, from AAMC, reported at the meeting some interesting work being done by the AAMC on the expected learning outcomes in medical education. Informing their work is a range of outcome frameworks, including the ACGME, CanMEDS and Scottish Doctor frameworks. It will be interesting to see the results.

We have had a good response to the two new ESME online courses first advertised last month and the final preparations are now underway for their launch on the 9th April. On Friday I leave for a very short visit to Saudi Arabia and a visit to Al-Imam Medical School in Riyadh.

Finally a puzzle. Before returning to Dundee after the Ottawa Conference and departing for Boston, I had a few days in a hotel located in the jungle in Langkawi, a small island off the west coast of Malaysia. I noticed that fixed to some trees was a small key hanging on a 6” chain about five feet off the ground. What was their purpose? The answer in my next blog 11th April!