What’s the deal with situational judgment tests (SJTs)? Why are they used? What is their purpose? And is it true that they help identify the most fitting applicants for jobs and professional and academic programs? In today’s article, we explore these and many more questions that will help you understand the logic behind SJTs, as well as their shortcomings. We bust some common misconceptions about these tests and, most importantly, we show how students can prepare for tests like CASPer, MMI, and in the comfort of their homes. Let's dive in!
For decades, professional programs all over the world have worked on figuring out how to select the best possible candidates. When frameworks like and AAMC’s core competencies, as well as interview questions like “?” and “?” no longer produced the desired results, admissions committees needed something new to detect the right candidates.
While it may seem at first glance that having strong academic acumen, like a high GPA, a high GRE score, or a strong , and relevant extracurricular and work experience should make a candidate the perfect fit for their chosen field, this is not always the case. Having strong knowledge of biology does not necessarily make you a successful doctor, having 10 years of dog-walking experience does not lead to success in the veterinary field, and in-depth knowledge of your preferred subject will not always make your students love you as a teacher. So how can we determine whether a person will be a good fit for their chosen profession? How can we assess their personality and character? If statistics and experiences do not tell all, how do we find out if the people we admit will become successful and happy in their profession?
Enter situational judgment tests. These tests do not evaluate your knowledge or professional readiness – they claim to evaluate non-cognitive professional skills, such as your dispositions, personal characteristics, qualities, and even values. For a while, SJTs like the CASPer test and the interview became a staple in the admissions processes to prestigious and incredibly competitive fields, such as medicine, dentistry, physical therapy, social work, education, and more. In fact, they were so popular that certain professional fields started to develop their own versions of these tests, such as the AAMC PREview, formerly known as the AAMC SJT. These assessments promised to showcase candidates’ character and suitability, so they became quite popular with admissions boards in a variety of fields. But there was another important admissions improvement that SJTs promised:
At the time of the rise of SJTs to prominence, professionals realized that bias is a strong obstacle in the development and flourishment of their industries. Hence, they needed new admissions tools to fight the prejudices present in higher education. SJTs promised to help in this pursuit by providing assessments that claimed to evaluate the “inherent” character of candidates, without questioning their socio-economic background, race, gender, religion, and so on. You can understand the logic here, of course. It is true that SJTs never directly question these attributes. In fact, they never question the candidate directly about their suitability or background. But does this mean that they do not contribute to the existing bias in education? Let’s find out.
Think most current premeds genuinely want to be a doctor? Think again:
Over the years, SJTs have not exactly proven to bring significant improvements to the admissions process. Many of the issues that they set out to resolve remained. But what exactly led to this? What is the problem with the SJTs?
Problem #1: The creators and administrators claim that these tests are not coachable, but this is not true.
One of the aspects that were initially very attractive for programs with regards to SJTs is that the creators of these tests claimed that nobody can prepare for them. ? How do you tackle ? The administrators of these tests would simply say that you cannot get ready for their evaluations. In other words, students would just have to be themselves and show their character and values in the test.
We will discuss the repercussions of this claim in a minute, but it’s important to note that this claim has been disproven repeatedly. We at BeMo have shown that there is a significant and that tests like .
Let's take a look at some data. Our first graph shows the increase in students' SJT and interviews score after our medical school admissions consultants had some coaching sessions with them; this includes sessions, MMI prep sessions, , and more. As you can see, the change is significant.
In our next graph, we focus on the effect coaching had on students’ CASPer scores specifically. Take a look at the improvement students show after going through several rounds of CASPer simulations:
Last but not least, take a look at the improvement students demonstrated after BeMo’s MMI coaching.
Problem #2: Admins claim that there is no right or wrong answer, but this seems to be untrue
The claim of “uncoachability” is closely tied to another claim they make, and it’s that there is no right or wrong answer. Why would you score a test if there is no right answer? If this claim was true, nobody would fail the test, and therefore, there would be no need to score tests like CASPer and the MMI. But we know and how the MMI is evaluated, so the claim that there is no correct answer must be false.
Problem #3: SJTs claim to assess “inherent” traits and character
If we reflect on the previous claims we discussed, a very strange and disturbing claim makes an appearance. If these tests cannot be prepared for, then it seems that SJTs claim that qualities that make a good professional in medicine, dentistry, or education are “inherent”, i.e., they cannot be coached or taught. This claim is unsettling. Why? Because in other words, these tests are claiming that there are some people that cannot ever become professionals in these fields. No amount of hard work, learning, and training can make them suitable for their desired career – if you do not possess these qualities “inherently”, you are out of luck.
Why is this disturbing? Because they are saying that some people are predisposed to professionalism – and they find them via their assessments - while others are not fit for these professions by nature. The creators and admins of SJTs are saying that their assessment tools are used to weed through thousands of applicants and choose the ones that are “inherently” suitable for these professions. Because if you cannot prepare for these tests, as they claim, then you must possess the required qualities from birth. Or you must have been born to the right circumstances to be habituated to these qualities from birth or young age.
Problem #4: SJTs do not eliminate existing bias in admissions
And that’s when another huge issue pops up. These tests are built on certain cultural and socio-economic norms, so does that mean that if you were not born into these circumstances you cannot learn to operate in them as a mature professional? If you were born and raised outside of the western cultural norms, you cannot act like a professional? Or if you were born and raised in a socio-economic environment different from the expectations presented by these SJTs, you cannot be “coached” to follow them? This disposition is not only wrong but also strongly biased.
It seems that the prejudices the SJTs were meant to eliminate are actually perpetrated by them. Let’s have a look at some data. Recent studies show that the CASPer test and the MMI have not had an effect on breaking down socio-economic barriers present in medical school admissions. The graph below demonstrates that over 17% of applicants who were accepted to medical school using the MMI were from families with an annual income of over $200,000. Compare this to the applicant pool who did not use the MMI in the admissions process – the number is actually smaller, only 14.16%. What does this tell us? Firstly, that MMI does not seem to diminish the socio-economic disparity in med school admissions. Secondly, it seems to perpetuate it! More students from low-income and middle-income families were accepted to medical school without the use of MMI.
The situation with CASPer is quite similar. As you can see in the graph below, in the applicant pool who had to take CASPer, the portion of admitted students from high-income households is much higher than in the pool of students who did not take CASPer. This further demonstrates that instead of affecting positive change in admissions when it comes to socio-economic barriers, CASPer and MMI seem to have no positive effect.
Problem #5: SJTs do not predict future performance
Now that we have seen that SJTs do not solve the existing biases, but rather propagate them further, we might want to ask ourselves “But maybe there is evidence that shows that SJTs correctly predict which candidates are a good fit? Maybe there is proof that they pick candidates that have great future performance?” Actually, no successful study has shown this.
There has been only one, very small study conducted on this topic by one of the creators of such SJTs. The study shows a very small correlation (r = 0.3–0.5, to be exact) between success on SJTs and medical licensing examinations. But before we go celebrating this tiny correlation, let’s look at the facts:
The correlation is insignificant.
The study was conducted using a very small sample, which can hardly be considered a pattern.
The study shows a correlation between success on one type of test and another type of test. Medical licensing examinations like USMLE, , and MCCQE are all very coachable! A high score on one of these tests does not demonstrate a student’s on-the-job success or happiness. They simply demonstrate that a student knows how to prepare and do well on tests.
Problem #6: STJs are coachable
We are repeating ourselves here, but this is important to note for all: students and admissions boards alike. Ultimately, if institutions are looking for admissions tools that will show them candidates' character, suitability, and outlook for future success, then SJTs are not it. The behavior and qualities that SJTs identify and evaluate are highly coachable. In other words, anyone can do well on the test if they have proper preparation and feedback. In other words, it does not mean that admitted students actually possess these qualities – they simply know how to approach CASPer and MMI questions. So we are back to the same problem, how do we pick candidates? How can SJTs help us choose the right students? But if all SJTs do is select candidates with proper behavior during the test, it means that there is really no value to these tests. If students can prepare for them, then what is the purpose of still conducting SJTs?
However, for someone who is facing the obstacle of acing their SJT, the above information, while interesting, is not necessarily helpful. How can you prepare? What traits and skills should you hone? We will guide you through all the necessary steps AND share some sample SJTs scenarios, questions, and expert analysis so you can start prepping for your test right away!
And while the information and tips we provide below are failproof, and you can certainly use them to practice on your own, we strongly suggest that you work with a professional to assess whether your performance is improving. Whether we like it or now, SJTs are still used by a variety of programs, so it’s best to use professional feedback to see how your prep is going.
Learn more about SJTs in our video below:
As we already discussed, SJTs test non-cognitive skills, such as professional behavior, decision-making, communication skills, and problem-solving. The trick is that the setting of SJT scenarios can be both professional and private, but you must apply the same principles to both. BeMo has been able to identify and compile a list of 21 possible themes in SJT questions:
- Conflict of interest
- Ethical/moral dilemma
- Professional boundaries, obligations, and ethics
- Scope of practice
- Scope and current social events awareness
- Personal questions
- Autonomy support
- Informed consent
- Evidence-based practice
- Rural vs. city practice
- Legal awareness
- Alternative solutions
- Non-judgemental approach
- Conflict resolution
- Global issues related to the profession
- Cultural sensitivity
Now, we are going to give you a failproof approach to ANY SJT scenario. Follow these steps and practice on your own to learn how to tackle any question you may face.
When faced with a scenario, you should first identify what kind of issue is being covered in the scenario with which you’re presented. Most questions will touch on several of the issues we listed above.
For example, let’s examine a scenario where you are asked what you would do if you witnessed aggressive behavior from your boss towards a colleague. Start by identifying the kind of issue you are facing. In this case, the scenario includes themes like ethical/moral dilemma, professional boundaries, obligation, and ethics, legal awareness, conflict resolution, and so on. Identifying the issues will help you orient yourself in the situation and help you understand how to proceed.
After identifying the issue(s) presented by the scenario, you must determine the most pressing issue in the scenario you're facing. To do this, you must identify the most vulnerable party in the situation because, ultimately, those who are negatively affected by the situation should be protected first. In this case, your colleague's well-being is your priority. However, do not forget to consider all involved parties. Remember to always remain non-judgmental to everyone in the scenario and do not make any rash decisions. If we continue with the angry boss example, perhaps you should consider your supervisor's personal problems. He/she may be going through a very difficult divorce or a death in the family. Though this does not excuse his/her violent behavior, you should always consider the situation from different sides and withhold judgment.
You must gather as much information as possible to come up with informed solutions to the problem you are facing. Choose the best solution based on sound rational, ethical, legal, and scientific reasons using the "if/then" sentence structure. By organizing your answers this way, you will demonstrate that you considered all the parties involved and remained non-judgmental. For example, in the case of your boss, you can write:
“My priority is the well-being of my co-worker. I will make sure she is ok after the incident. However, before I make any decision to approach higher management to complain about my boss, I will speak to him/her to see if they are doing ok and try to understand what caused this angry outburst. If I learn that my boss is going through a death in the family or another difficult situation that made them act out, then I will try to calm down my colleague by bringing them some water and telling them that the boss did not mean to upset them personally. I will ask them to join me for lunch and take their mind off the incident. I will also let my boss know that she/he should probably apologize to our colleagues and let them know that they are valued and respected. If I learn that my boss does not have any external issues and simply acted unprofessionally toward my colleague, I will have to approach HR, so they can decide how we can help our boss with anger management.”
It is important to know that although many SJT situations and questions are ethically challenging, you should always try to actively engage with the problem. Doing nothing in the situation with your boss is not going to get you a high score on the test. You must at least show initiative by gathering the necessary information. If you simply ignore the altercation or write "I am not going to do anything, because it's none of my business", you will surely do poorly on your test.
Knowing the general approach is the first step, but what about all the or CASPer questions that you may face in the SJT? How can you prepare for a million different scenarios and questions you might have to face in your test? Here’s the secret: there are important approaches that you can implement to ace any SJT scenario or question type! The key is to know what categories of questions are typically included on these tests and to have a concrete answer strategy for each question category. This means that even though nobody knows what exact questions you are going to get on your test, you will have an approach for answering any questions that may come your way.
The 21 question topics I listed above typically fall into 3 major question categories:
Remember, several themes can be covered by one single scenario, so be aware of what kind of conflict you are trying to solve. You should also know that a CASPer scenario, for example, can include two or even three question categories! That is after you are given the scenario, the follow-up questions may include a scenario question (what would you do in this situation?), a policy question (what do you think about a certain policy or law), and a personal question (have you ever had a similar thing happen to you?). It is rare but possible to see all three question categories in one of the 15 CASPer scenarios.
To know how to provide an answer to questions from each category, you must have specific strategies. Having answer strategies for each question category guarantees that when you read any given prompt and read the follow-up questions, you will know how to structure your answer accordingly.
Situational or scenario questions are very common, and we discuss how to tackle them in the previous section. But what about the rest? How can you tackle the policy and personal questions so often used in SJTs?
Follow-up SJT questions from the policy category are typically included to assess your awareness of the local and global issues facing your profession and your ability to remain objective and non-critical of all sides of an issue. Again, consider the themes that the policy question is asking about, i.e., scope and current social events awareness, rural vs. city practice, ethical/moral dilemma, etc. Follow this general structure to answer SJT policy questions:
To prepare for questions in the policy category, you should become familiar with current events, hot topic social and political issues, and challenges facing your professional field in your town, state, province, or country. You can start by visiting the websites of your provincial/state and federal/national professional associations to read about the latest policies and news. You can learn about local and global issues by reading the newspaper, watching the news, and visiting the websites of global institutions. To practice answering policy questions, you can write down a list of the most important and interesting policies you encounter, write down their pros and cons, and try to formulate your own informed opinion. Practice with this strategy and know how to implement it in answering different policy questions you might face in the real test.
Many SJT questions will ask you to talk about times when you had to deal with similar issues that are presented in the scenario. They are meant to assess how you react in certain situations. The question may deal with professional boundaries, ethical/moral dilemmas, conflict resolutions, and other issues. Your answer should demonstrate to the evaluator that you know how to handle yourself in a situation.
When you answer personal questions remember these steps:
- Provide a very short context about the situation you are writing about.
- Write one or two sentences demonstrating specific actions, behaviors, or tasks you performed to deal with the situation. What did you do to fix the problem? How did you overcome a challenge? Use concrete examples.
- Write one sentence about what you learned from dealing with this situation.
Check out more CASPer practice questions in the video below:
The question categories we outline below can potentially come up during your MMI interview. Though they are less common than the questions we outlined above, they also require some preparation. Rule #1 – use the same techniques we outlined above when faced with a scenario, policy, or personal question in one of the stations we outline below. Your strategy does not change. However, you should know what to expect in such stations, so let's dive in:
Acting questions are exactly as they sound. You will need to act out a scenario or a situation that will test your reactions to real-life dilemmas. At these stations, you will be given a prompt that describes a certain situation and once you enter the room (physically or virtually) you will be asked to begin acting as if the situation is actually happening.
Make sure that you understand the prompt and stay attuned to the cues and emotional responses of the actors in the room with you. You may be faced with scenarios where someone becomes angry, loud, and impatient. In these situations, keep your cool and come up with decisions that would professionally resolve the conflict at hand. Remember our approach to scenarios? Stay non-judgmental and objective, even if you are uncomfortable. Your words and actions must demonstrate that you are staying non-critical. Make sure your words, tone, and body language are appropriate for the situation you’re facing. Always put yourself in the other person’s shoes to truly understand what they are going through and what kind of reaction is appropriate.
Acting questions also test your ability to be tactful and socially aware, as scenarios you face can involve questions relating to public outbursts, domestic abuse, child neglect, and other types of negative social interactions. Though they are unpredictable, you must remember to stay collected and gather as much information as possible to find the best solution for the problem you are facing. Whether it is a disgruntled patient or a friend who is being physically abused by her boyfriend, you should demonstrate empathy, gather information, and act, if necessary.
These types of questions are meant to assess your ability to problem-solve, strategize, and collaborate with others. Collaborative stations may include drawing assignments, building assignments, or other task-based instruction. They may also include a debate.
For a drawing task, you are typically asked to relate to another person in a form of media. For example, using verbal instructions, you may be asked to relate a drawing or a shape to another person who has to replicate it on a piece of paper. Your performance will reveal whether you approach problems in an organized, logical manner and whether you possess strong communication skills. In these kinds of stations, make sure the interviewer or the other applicant understands what is going to happen in the next 4 to 8 minutes, i.e. explain that they will need to follow your instructions to replicate the drawing before you. Ask if they have a pen and paper and explain what the drawing looks like. You can usually implement the Cartesian coordinates (XY system) to orient the person who is drawing.
You may be faced with other collaborative questions in your MMI interview. These may include solving a problem or creating a plan with another applicant, teaching one another something, arguing opposing sides of an issue, acting out a scenario, or answering individual questions while sitting next to one another. The key to succeeding in a collaborative interview station is remembering to collaborate with the other applicants so that you all succeed, rather than competing to outshine them in front of the interviewer. The interview will note your ability to work in a team and put the needs of the team before your own interests. Often, collaborative stations will not score you based on your personal performance, but rather on the performance of the group as a whole.
Some programs use 10- to 20-minute writing stations to further assess candidates' communication skills. The prompts for writing questions typically fall into the three main categories we already discussed: scenario, policy, and personal. This means that instead of verbalizing your answer, you will be writing it down using the same answer strategies we outlined above. Writing stations may also sometimes involve quote interpretation, but you can definitely use the personal question answer strategy to tackle those.
Your answer must be a coherent narrative. Before you write your first word, think of the structure of your answer and its key points. For example, if it is a policy question, summarize the policy and the issue it is meant to address, then list some advantages and disadvantages of the policy, and conclude by stating your opinion and providing an alternative solution if you disagree with the policy at hand. Implementing the same strategies will keep your essay organized and to the point. Leave yourself some time to reread and edit your essay for any grammatical mistakes. Although your grammar might not directly affect your score, a well-written essay will ultimately score higher than a disorganized, error-ridden essay.
Now, before we get to the practice SJT scenarios, we would like to share some very important tips.
What to learn more about SJTs?
Situational Question Example
You are a truck driver. After your shift, you go to the local bar where all of the drivers hang out. When you get there, you see a colleague. As you say hello, your colleague gets a call from your supervisor who is in charge of assigning routes. Your colleague has a conversation with your supervisor and agrees to a route. Your colleague, who has a few empty beer bottles and shot glasses in front of him, orders another shot and then runs out to get to his assigned route.
Try to answer on your own before you read our expert responses!
Policy Question Example
You are a teacher on your break in the teachers’ lounge. One of your colleagues rushes in and begins telling everyone about a new policy being introduced at the schools in the region. This new policy is geared toward preventing childhood obesity. As part of this policy, students will be assessed based on their physical fitness and diet. They will also be monitored throughout the year and reassessed periodically. To ensure a good diet, all the students' bags will also be searched when they come to school to ensure they are not packing unhealthy snacks.
Try to come up with your own answers first!
Personal Question Example
The field of [insert your profession] consists of inter-professional collaboration, teamwork, and lifelong learning.
Try to come up with your own answer before scrolling down!
1. What is a situational judgment test?
SJTs are aptitude tests that claim to test candidates’ suitability for a variety of professional fields, such as medicine, education, social work, and so on.
2. What are the cons of SJTs?
While SJTs claimed to improve the admissions process, there have been no real indications that they had a positive effect. For example, they do not seem to improve the representation of diverse socio-economic backgrounds in medicine. Additionally, there have been some that suggest that certain SJTs rate females higher than males.
3. What kind of SJTs are used by admissions boards for professional programs?
Currently, the most popular SJTs among admissions are CASPer and MMI. Some institutions created their own SJTs, such as the AAMC PREview.
4. Is it true that you cannot prepare for an SJT?
No, the skills these tests evaluate are highly coachable.
5. How can I prepare for an SJT?
You must have a solid strategy for every possible question type you may face. If you have an answer strategy, you will be able to structure a concise and clear response to any question. We strongly recommend using mock sessions and simulations to practice and get feedback from a professional to improve your performance.