In college, there is a type of exam students jokingly refer to as the "multiple guess test." Such multiple-choice assessments present the test taker with at least four potential answers to choose from. If a student doesn't know the exact answer, he can use the process of elimination to select the correct one or simply guess, and there's a good chance he'll guess correctly.
In the public safety world, if new recruits pass such tests to receive certification, those relying on these new employees believe these individuals come with a basic set of skills and knowledge. After all, they did pass an exam certifying they knew the material. But this may not be true, warns Brian Webster, vice president of marketing and business development at Knowledge Factor Inc., a Denver, Colorado, company that has developed new testing protocols to ensure assessments accurately reflect what test takers actually know.
"There are some people who are very lucky test takers, who can actually guess the correct answers on certifications without really knowing the information," Webster explains. "They may receive credit for information they don't truly hold. Now, that's not a serious deal in college, but it becomes a major risk with first responders, where quick decisions may mean the difference between life and death."
Three years ago, Knowledge Factor uncovered a learning methodology, developed by a UCLA professor over 20 years, which eliminates guesswork from multiple choice testing and helps ensure students have mastered the subject material.
The company calls this training method Confidence-Based Learning (CBL), which it acquired and patented three years ago. This technique ensures learning by measuring precisely what people know and don't know — without guesswork and doubt contaminating the results. Once this is complete, CBL rapidly remediates learners' gaps in knowledge and confidence.
"We individualize training and learning by first understanding what someone knows or doesn't know," explains Webster. "We then use these results to tailor a curriculum to the individual's specific needs."
Knowledge quality = confidence
According to Webster, CBL accomplishes learning by measuring knowledge quality in four different domains:
- Mastery. This is where individuals are both correct and confident in their knowledge. "Individuals with confidently held, accurate information will perform effectively and predictably based on that knowledge in the field," Webster says.
- Doubt. The person retains the correct knowledge but does not believe with full confidence the information is accurate and thus is not confident enough to act on it.
Misinformed. This is when people confidently believe the wrong information is correct. Confidently held misinformation is dangerous because individuals will act on that knowledge but won't have the correct outcomes because their facts are incorrect, says Kevin Warr, CEO and technology officer at Knowledge Factor.
Misinformation is absolutely unacceptable in law enforcement, Webster concurs. "We could probably call it the headlines quadrant," he says. "Chiefs of police would be reading headlines about it in the paper if their officers acted that way."
- Uninformed. This is where someone lacks any knowledge of the subject.
Under the traditional means of assessment, in order to determine whether two separate individuals, Person A and Person B, are competent in a given law enforcement discipline, an instructor would administer an exam.
Now let's say both individuals scored a 90 percent on this assessment. It would then be assumed they were equal in their abilities. But what if you learned that 96 percent of the time when Person A answered correctly, he felt confident about his answers, compared to Person B, who only felt certain of his answers 90 percent of the time and the other 10 percent of the time had guessed correctly?