CASPer & MMI Prep Myths Busted!

Note: The following represent the opinions of the authors based on publicly available information about such tests. 

Here are some of the topics we will discuss:

1. The strengths and weaknesses inherent in the CASPer test.

2. The strengths and weaknesses inherent in the MMI.

3. The lies & facts about effectiveness of CASPer test prep and MMI prep.

4. Preparation strategies that you can and should implement when preparing for the CASPer and MMI.

The strengths and weaknesses inherent in the CASPer test

First, note that although there has been a few publications about the efficacy of CASPer, they are mostly done by the same originating university affiliated researchers/individuals who have now turned the test into a for profit company constituting a conflict of interest.

 What is CASPer? In short, CASPer is essentially another tool that allows admissions committees to gain a better understanding of the candidates’ soft skills or non-cognitive skill sets (i.e. communication skills, interpersonal skills, ability to make ethical decisions, ability to resolve conflicts, etc.) prior to the interview. The CASPer is simply an online multiple mini interview. Instead of interacting with people as you would at an in person interview, during the CASPer, you will simply watch or read scenarios on your computer, answer some questions, and you are done. In a sense, CASPer is a dress rehearsal for the real interview, and if you do well, then you are likely to be invited to do it all over again live and in person.

Although CASPer is claimed to be a more reliable and valid measure of non-cognitive skills when compared to personal statements, short essays, biographical sketches, etc., it still has many shortcomings as an assessment tool. These shortcomings include, but are not limited to, the following based on our opinion:

1. Potential for cultural bias: While there are some universally shared beliefs about what is considered right and what is considered wrong, there are huge cross-cultural variations when it comes to expectations of professional conduct, dealing with superiors and colleagues, etc. For instance, it is common in the Israeli military to call your superiors by their nicknames, but this may not be considered professional in other cultural settings. Thus, the CASPer test might cause bias if questions are not validated across cultural backgrounds first and further independent studies are required to refute such concerns.

2. Candidates providing socially acceptable answers: Although the test is supposed to measure the true and genuine intentions of the candidates, there is a likelihood that given the various scenarios, the candidates simply provide socially acceptable answers, which of course, may or many not resonate with their true belief systems. This makes the test less reliable and valid. The social desirability bias inherent in the test is a concern and should be addressed in future versions of the test.

3. Potential for socioeconomic bias: There are numerous studies that have linked parental socioeconomic status, income, education, etc. (i.e. social capital, economic capital, cultural capital) to the development of cognitive and non-cognitive skills in their offspring. Of course, what the evidence shows is that those individuals who come from more advantaged socioeconomic backgrounds, on average, have stronger cognitive and non-cognitive skills that those from lower socioeconomic strata because of their economically-biased childhood experiences. Thus, the CASPer test can potentially discriminate against candidates who come from less advantaged socioeconomic settings and those who did not have the opportunity to develop or refine their non-cognitive skills as a result of their lesser social, economic, and cultural capital and further independent studies are required to refute this concern. This fact alone makes it essential that medical schools give candidates an equal footing and encourage them to take the right opportunities to adequately prepare for the CASPer, rather than to simply tell them “no need to prepare since there are no right or wrong answers.” Or simply, “Be yourself”. 

At the end of the day, medical schools (and other professional programs) should be selecting those candidates who have the capacity to 

(A) learn new skills, 

(B) be mindful of their own limitations, 

(C) and prepare in advance through meticulous planning, while seeking the help of others in order to fill in their knowledge gaps. 

These candidates would be as appropriate as those who were simply selected because they had the opportunity to develop non-cognitive skills as a result of the social, economic, and cultural capital bestowed upon them by their social networks.

4. Difficulties in verifying who actually wrote the test: Since the CASPer test is performed from the comfort of your own home and on your own computer, there can be cases in which other individuals aside from the actual candidate write the test. Of course, as a result of this shortcoming, the test results once again needs to be validated via an in person interview because even if the test is proctored, some can argue that it is still possible for some to fake their presence.

5. What about candidates with physical disabilities: This may seem obvious, but since the test is a written/typed test online, those candidates with poor typing skills or those with physical or cognitive disabilities may find the administration of the test unfair. However the site indicates there might be some considerations for those with disabilities however it's not clear what such accommodations would involve.

6. There is little scientifically sound evidence that formally proves MMI and CASPer's efficacy: As discussed, most of the few research that has been conducted so far has been done by the same individuals/institution that are now profiting from these tests so there is a clear conflict of interest, and in our opinion, such 'research' must be viewed with a grain of salt. More importantly, the only way one can prove whether or not CASPer test or MMI are actually better at selecting future candidates is to wait until new candidates that have been selected via CASPer and/or MMI go through their entire career (A 40 year study!), and of course, this study must be done by individuals who are not affiliated with the companies that now sell CASPer test and MMIs to admissions offices, nor the originating university.

The strengths and weaknesses inherent in the MMI Interview:

First, note that although there has been a few publications about the efficacy of multiple mini interview, they are mostly done by the same originating university affiliated researchers/individuals who have now turned this interview format into a for profit company constituting a conflict of interest.

The above arguments most certainly also applies to the MMI. The MMI and the CASPer share similar strengths and limitations. When it comes to its strengths, the MMI is said to be much more reliable and valid when compared to the traditional interview. But at the same time it contains many of the limitations inherent in its online equivalent the CASPer test. Some studies suggest that the MMI is culturally biased, socioeconomically discriminatory against certain candidates, and the social desirability bias remains an issue. But with all that said, keep in mind that both the CASPer and MMI are now being used and you can and should prepare for them in advance.

The "lies" & facts about effectiveness of preparation exposed:

Since CASPer and MMI are supposed to expose future candidates to real life scenarios, and since the purpose of the test is to see how an individual truly reacts to such scenarios, it has been argued that there is no need to prepare in advance for this test.

In fact certain arguments by some (certainly not all) admissions committees & test admins include the following inappropriate and false commentary about preparation:

Example 1. “BE YOURSELF”: “MMI is designed to see how you think on your feet, how you communicate what you believe when pondering ethical, moral and other dilemmas (which are not necessarily medically-related), and to explore who you are as a person. The best way for you to demonstrate these things during the interview is to be yourself, and answer honestly and thoughtfully.” - UBC

This claim is false in our opinion because “being yourself” implies that you should not even be interested in self-improvement, which is in fact contrary to the one common skill every good professional has: a sense of life-long learning and self-improvement. As you will see below you can and should always endeavour to be your BETTER self with appropriate training.

Example 2. “The general literature suggests that situational judgment tests are relatively immune to test preparation, i.e. that coaching is unlikely to provide benefit.” - Altus

This is false in our opinion because: 

A) the authors do not provide any evidence or reference to such “literature”. 

B) The wording implies that this is only a "relative" statement, however the authors' reference again is not clear.

 C) The "literature" in this field is not designed specifically to assess the efficacy of any preparation program such as BeMo's CASPer prep, which makes any conclusions about preparation programs and courses impossible. 

D) Most of the "literature" about CASPer has been conducted by the same individuals that are now part of the for profit company that administers this test, which constitutes a clear financial conflict of interest and the results cannot be taken seriously until they are reproduced by independent scientists.

 E) We have formal proof that situational judgment tests are indeed influenced by appropriate preparation, at least in practice.

So why would admissions committees want to "LIE" to you? 

We can't be sure and that's a question you have to ask them. In our opinion, many schools discourage their candidates from preparing for the test because they do not feel secure about these admissions tools and simply fear it will interfere with the reliability and validity of the test. And as discussed, some may have financial conflicts of interest because both CASPer and MMI have now turned into revenue generating corporations for their creators and possibly their affiliated universities. But here at BeMo we feel differently, and we think you can and should prepare for your MMI and CASPer test in advance, based on common sense reasoning plus evidence-based facts and figures.

First and foremost, it should be highlighted that although non-cognitive skills take time to develop via the various life experiences that we have, there is no reason that one cannot develop such skills through other avenues, or even learn more about these skill sets through some form of education. To argue that one cannot prepare for the CASPer or MMI is akin to arguing that one cannot prepare for an interview, or that one cannot modify the behaviour of a young child. This type of reasoning implies that non-cognitive skills are something that an individual is born with, and if they have it great, if not, well they are out of luck and they should never apply to a medical school! Well, this simply sounds ridiculous and even discriminatory! In fact, many of us have a strong sense of ethics, morality, interpersonal skills, compassion, etc., but we perhaps are unable to articulate or even identify such skill sets in ourselves and to others. Thus, it becomes important to have help in identifying such skills in ourselves, and to learn how to appropriately apply them across various settings.

Still not convinced that you can and should prepare for your CASPer and MMI? Well let’s look at some official statistics from McMaster medical school (the birth place of both MMI and CASPer) along with some statistics devised by our BeMo research team from the previous application cycle alone.

Just to give you an indication of how important the CASPer and MMI are when it comes to the admissions process, consider that last year McMaster medical school, for example, invited 552 individuals for an interview. Two hundred and six of these candidates, or 37%, were accepted into the program, while three hundred and forty six, or 63%, were rejected. The success rate at McMaster is not very high for those who make it to the interview stage and this should be alarming. 

But here is an interesting statistics that will make you feel more hopeful about your chances. Last year alone, out of those students who were enrolled into the BeMo Platinum MMI preparation, 90% were accepted into the program of their choice. This shows that with effective preparation, you can increase your chances of success for either the CASPer or the MMI.

BeMo’s CASPer and MMI preparation philosophy and strategies:

Our preparation programs have proven to be effective because they have been designed to:

1. Help you understand the deeper intention behind the CASPer and MMI and fully appreciate their rationale. We feel as though once you understand the rationale behind any phenomenon, it will become easier to deconstruct that phenomenon and to tackle it on. 

2. Help you develop a strong sense of professional ethics through our intensive online video course and in-person consultation sessions which not only expose you to important concepts within the field of medical ethics, but more importantly, will help you identify various categories of questions and/or scenarios that you may encounter during your actual CASPer or MMI. Furthermore, we will help you understand how to devise appropriate and strong responses to the questions on the CASPer or MMI, not by teaching you any tricks, but by simply helping you strengthen your non-cognitive skills and refine your sense of maturity and professionalism which will be essential to your success on the CASPer and MMI, and more importantly, as a future practicing professional.

3. Give you access to full CASPer simulation tests and MMIs which will allow you to practice the skills which you would have developed through our programs and with the help of our CASPer & MMI Interview experts who will be there to provide you comprehensive feedback about your performances.

We here at BeMo, truly believe that there is nothing that you cannot prepare for in advance. Just like your MCAT or undergraduate exams, there are appropriate and effective ways to develop the required skill sets to perform effectively and successfully on ANY test.

To your success,

Your friends at BeMo

BeMo Academic Consulting 

Disclaimer: CASPer stands for computer-based assessment for sampling personal characteristics and is claimed to be a trademark of McMaster & Altus. BeMo does not recommend, endorse nor affiliate with CASPer, Altus or McMaster and vice versa. BeMo only provides preparation services and practice tests. To take CASPer, contact Altus directly.