Three Reasons Why Massive Failure Rates Happen

Listen to the full interview about Licensing Exams with Chris Beauchamp, on Youtube or on Spotify.

An incident that raises a lot of eyebrows in Canada is the extremely high failure rates in many licensing exams. Unfortunately, this issue isn’t new for us Canadians. A similarly high failure rate occured in New Brunswick; only 32 per cent of nursing graduates from Université de Moncton passed, compared to 91 per cent in the previous year. 

To help us better understand why large failure rates happen, Dean spoke with Chris Beauchamp, Vice President at Meazure Learning. Chris and Dean walk through the three most common reasons cohorts may experience a high failure rate:

• Translation flaws or language barriers
• Curriculum misalignment
• Group characteristics of the testing cohort

1. Exam Translation Flaws Can Cause High Failure Rates

Language can lower chances of success, especially if the exam is not in the test taker’s native language. In fact, language barriers have been identified as the single most significant obstacle in the United States and Canada for nursing students who speak English as a second language (ESL). One reason is because non-native speakers must translate the question into their primary language, think of an answer, and then switch back to English to select one of the provided answers. This process not only increases a student’s cognitive load and stress levels, but significantly increases the time taken to answer each question.

The process of translating exams also carries its own risks. In Canada, a common explanation of New Brunswick’s high failure rate was that the French-translated version was translated by people “who were not qualified translators.” In the same year, over 400 people signed a petition citing there were language problems on Quebec’s English licensing exam that put them at a disadvantage. 

Additionally, translating the exam to non-official languages may put patients at risk if the nurse isn’t proficient in the region’s primary language. “If you can’t communicate in the predominant language of the region, it becomes a patient safety issue”, Chris points out.

2. Curriculum Misalignment Can Cause High Failure Rates

Sometimes, a high failure rate can occur because of curriculum misalignment. In Canada, the organizations responsible for admissions into a profession are not the same as those responsible for its education. This makes communication and collaboration more difficult. As Chris said, “Regulators are similar to quasi-governmental agencies. Healthcare, for example, falls under the Ministry of Health, whereas the education system falls under the Ministry of Education.” Both departments have their own mandate and strategic priorities. They may not necessarily be fully aligned on what is being taught in class versus what the exams are testing.

In New Brunswick, many blamed the high failure rate on the American company that created it. The exam therefore may better reflect the American healthcare system rather than the Canadian one. In an interview with CBC, the past president of the Canadian Nursing Students Association said that the exam focuses on nursing care in hospitals, which is more common in the United States, rather than primary care in family clinics or in Indigenous communities. almost two thirds of. Later, a report from the Canadian Association of Schools of Nursing found that two thirds of Canadian competencies were either not addressed at all or only partially addressed in the test plan, and many that are addressed are conceptualized and interpreted differently in the Canadian context. 

3. Specific Cohort Characteristics Can Cause High Failure Rates

A large failure rate may also be caused by the specific characteristics of that cohort. Many nurses’ learning outcomes were strongly impacted by the pandemic. Those who wrote the licensing exam in 2022 would have completed the majority of their studies partially online. “Students usually have hands-on clinical simulations and practicums in the field. For an extended period, they weren’t able to get into hospitals and facilities to do their training.” says Chris. “That was huge for learning and development, especially in the healthcare field.” Since the start of the pandemic, many findings have been published about the disadvantages that the ‘pandemic cohort’ have faced, including lower feelings of practice readiness, reduced sense of confidence, and higher stress levels.

Conclusion

As Chris said in his interview, making sure exams are fair for everyone is something that’s become more important to exam owners and regulators. It is important to not only include folks from different groups and geographical regions during the exam design, but to also get them involved in the approval process from the beginning, and ensuring the test administration process is accessible to everyone including those with physical disabilities or learning disorders.

About Dean Benard

Dean has been a professional investigator for almost 30 years, including roles as a police officer, regulatory investigator, investigations manager. He has run Benard + Associates since 2003. He and his team have conducted thousands of investigations in regulatory and workplace matters. Additionally, he is an advisor, consultant, coach, and leader in the investigations and conflict resolution communities.

His background educationally includes a nursing diploma from Fleming College, yes he was first a registered nurse working in critical care and research, before he entered the investigative field. He holds a Bachelor of Arts in Health Administration and Ethics from York University, a Masters of Laws in Alternative Dispute Resolution from Osgoode Hall Law School, and has a postgraduate certificate in Diversity and Inclusion from Cornell University. 

About Chris 

Chris has a PhD from the University of Ottawa in experimental psychology. He also has a BA (honors) from University of Ottawa, and a psychology and accounting diploma from Canadore College. Chris has years of experience in applied psychometrics in the Canadian credentialing field. He also has extensive experience in high-stakes exam development, administration, security, and statistical analysis for a full range of organizations.