Update 13/2/15: The candidate handbook has been released by the college, with relevant information added at the end of this post
Update 20/2/15: Some more information on the marking schema has been leaked. See Update below
Update 24/2/15: How to use the online SCQ system information added
The Official Word
Again, there is some information on the format of the exam on the ACEM website. The first part of the written is the select-choice (MCQs and EMQs), which will be done at a computer terminal. The second part is the more nebulous SAQs:
The SAQ component will use questions that are highly structured and specific, with responses that will require single words or short phrases, rather than mini-essays
You'll find some examples of the questions in the Resources tab, with an updated version of SAQ examples.
For convenience, again, I've copied across the relevant excerpt from the website (correct as of the writing of this post), but it may be updated.
The FEx (Written) will:
- Examine at consultant level, with a focus on knowledge-application
- Be a separate assessment from the FEx (Clinical)
The FEx (Written) will comprise two components:
1. Select-choice Question
- Multiple Choice Questions (MCQs) - Each MCQ comprises a stem (i.e. a short lead-in phrase) and a number of alternative options for response. The correct response is presented, along with a number of plausible distractors. The MCQs will be written to current best-practice guidelines. In the ACEM format, trainees select one best option from four options.
- Extended Matching Questions (EMQs) - Each set of EMQs comprise a theme (e.g. a particular presentation, investigation, diagnosis, or treatment); a list of possible options (i.e. options related to the theme), and a number of stems requiring a response chosen from the list. In the ACEM format, trainees select one best option from the options provided (which could be up to 25 options).
The proportion of MCQs to EMQs in each paper is not fixed. It is likely that the proportion of EMQs will expand over time. The total number of items in each exam will remain constant. The select-choice question component of the written exam will be undertaken online.
2. Short Answer Question (SAQ)
The SAQ component will use questions that are highly structured and specific, with responses that will require single words or short phrases, rather than mini-essays. The key feature of SAQs is that they allow the trainee to demonstrate the integration of their knowledge and application of this knowledge to clinical scenarios. SAQs will generally be of the format where the candidate is given a clinical or ED related scenario, followed by specific related questions that can be answered using relatively few words or short statements. When asked for a defined number of answers, additional marks will not be given for additional responses. The most relevant/highest priority responses should be listed.
The written examination will be held on a single day and comprise 2 x 180 minutes papers with a break in between. There will be no designated reading time for either parts of the Fellowship Exam (written). Each format of the paper (MCQ/ EMQ) and SAQ has been designed to incorporate the need to read the questions.
Example Fellowship Exam (written) questions are available in the Resource tab.
The Unofficial Word
Once again, some details and information have filtered down into our sweaty palms as we now sit, perched merely 2 weeks away from the great unknown of FE 2015.1. So what do we know (or can we surmise) about the upcoming SAQs?
- The ACEM exam definitions are still relevant
- All SAQs have been constructed so that they are completed in 6 mins (including reading time for each question)
- Questions will be specific in the amount of detail they require (eg, asking for doses, justification, etc)
- All SAQs have been standard set and the quality of the questions checked. The answers required to ensure safe EM practice (ie the borderline candidate) have been confirmed
- Where a number of answers is requested (x), then only the first x answers will be assessed. Any more than this (x+1) will be ignored
- You should not write more than x responses
- If you do, you need to place the most important answers in the first x responses; if correct answers are listed in the x+1 group, then they will not gain points
- The order of answers does not matter, as long as they are within the x responses
- If you write more than the specified number, you can indicate which x responses you want assessed, and which are the x+1 to be ignored (eg, by numbering or deleting)
- It is possible that writing outside the confines of the provided lines will not be assessed
- If you change your mind, you can cross out an answer and write a new one. Presumably you can write outside the provided area to do so, if your deleted answer is taking up that valuable space
- It is possible you could group answers within categories, but the question would probably prompt for this
- For example, list 4 differentials with examples:
- Sepsis - UTI, pneumonia, CNS infection
- Cardiac - IHD, arrhythmia
- Metabolic - hypoglycaemia, hyponatraemia
- Trauma - head injury
- For example, list 4 differentials with examples:
- If you miss what the question asks for (eg drug doses), then you won't get the mark for that answer (even if the drug is correct). If it asks purely for a list of something, then just the list is required
The marking schema for the SAQs is still unclear. It is likely that:
- Marks are allocated for correct answers within the requested x responses. There may still be important answers that should not be missed
- Each SAQ is marked per section of the question, and the sum of parts making the overall score. Failing one part of an SAQ may not mandate failing the whole question
- Individual question and overall exam scores may be scaled along a distribution curve
The Hot Tips
- Keep to time. If you write nothing, you will get zero marks for that question
- Read the question carefully, and answer exactly what is asked for
- Don't write more than the requested x responses. This wastes time and earns no extra marks. This is one of the few exam techniques to be disciplined about
- The format of the paper will provide cues to the number and amount of answer required; if x responses are requested, spaces numbered 1–x will be provided (see the example SAQs)
- Don't waste the valuable answer real estate with big gaps or waffle
Update 13/2/15: FE Candidate Handbook
ACEM have now released the FE Candidate Handbook for 2015.1, which confirms many of the points above. Additionally, other important notes are:
- No negative scoring for the select choice questions (so answer all of them)
- SAQ marks are allocated at 1 per response expected, to guide how many responses you need (if a specific number are not asked for) and how to split your time in the question
- "Fatal errors"—errors of commission or omission that would cause serious patient harm—will result in zero marks for that section, but not the entire question; you could still pass a SAQ despite a fatal error
The whole handbook is available on the ACEM website, or you can read just the information on marking here:
Update 20/2/15: Not all SAQs are created equal
In the candidate information above, we are told "SAQ marks are allocated at 1 per response expected". This means that each individual answer will be weighted equally, 1 mark per response.
The latest word expands on this, and suggests that SAQs will not all be worth the same number of marks. Those that ask for fewer responses will be worth fewer marks—there will not be the same number of responses required for each SAQ. Rather than "adjust" these scores to give an equal weighting to each SAQ, each will attract a raw score based on the marks allocated within.
This has implications on time management, as an SAQ worth 12 marks will likely require more time than an SAQ worth 8. The average marks per SAQ and the overall number of marks available is not specified, but estimated to be about 300 (10 per SAQ). This schema may change for the 2015.2 exam.
Read on further on the advice we have seen from David Lightfoot, Chair of the Court of Examiners:
All answers are allocated one mark. What that means is that if the subquestion (e.g. SAQ 20 part a- a random subquestion) asks for one response (e.g. what is the diagnosis?) that is worth one mark (candidate can score either 0 or 1). If it asks for four responses (e.g. what are the four most likely causes?) then each of those four responses is worth one point, so the total scored could be 0,1,2,3 or 4 depending on the number of correct responses given by the trainee.
Each required response counts as one mark. No maths is applied to the raw mark to get the trainees score. E.g. for the 30 SAQs, there may be a total of say 300 responses, each is worth one point, so the whole exam is out of 300. On the exam paper under the area reserved for answers to each subquestion it lists the marks available for that subquestion. So from my examples above, after the line ruled for the trainee to write the diagnosis it say "/1" and after the four lines numbered 1-4 for the Causes to be written it says "/4". So each response has the same weighting but some SAQs will have more required responses and so will be worth more marks.
...(the committee) tried to make the expected answering time for each SAQ as close to equal as possible, but that it cannot be done perfectly. That was the reason for not scaling the marks to be equal for each SAQ. I think the "unevenness" is not too gross, and that trainees should apportion approximately equal time per SAQ. Perhaps a better strategy is to aim to complete 10 SAQs per hour, rather than 6 minutes per individual SAQ. I think that would be easier to keep track of during the exam. Some of the individual items will be very fast to answer and some slower, so candidates need to keep that in mind.
Update 24/2/15: Online SCQ instructions released
In the continuing dribble of information about the new exam format, instructions on how to use the online SCQ system have been distributed to 2015.1 exam candidates.
Included on the last page is this little pearl about what to expect within the exam:
If you have any more questions about the exam format, talk to your DEMT or local examiner. And if you have any more tips or resources, please share them!
Good luck, and happy studying!