Can anyone please point me to an article or research that supports 80% pass rate as good/normal practice? I'm trying to make a case for reducing our pass rates from 100%.
So glad I found this thread! I'm discussing this with someone now who, instead of opening up remediation options just wants to right "easier questions" and set the pass score at 100. *arg!*
I'd like to build on what Phil is suggesting and take this a step further. First, looking at the course content and purpose - is the score important to receiving a credits/accreditation? If not, and we are simply seeking understanding, I propose that in many cases a pass score isn't important at all. Why do we mistrust adult learners so still? I advocate for achieving deeper understanding instead of testing his or her ability to recollect. I'd like to see more answer exploration available so that the learner understands why an answer is incorrect as opposed to simply that it is incorrect. Adult learners are already seeking to understand. Most who get tripped up do so from OVER-thinking, considering the nuances of the question and its application. I am advocating that we harness and appeal to that instinct instead of punish it.
That would mean that either way, the learner can 'pass' the test, but the real learning will happen as they explore WHY their answer was right or wrong. They would still receive a grade, but the completion itself would be enough to pass the course. We would trust the adult learner to use the test as a learning tool, treating them less like a grade-school student and more like a learning partner in the course.
This style would not be applicable in all cases, but I believe strongly that we should continue to build on the momentum in adult education where we are increasingly opening content navigation, reducing restrictions, encouraging rather than discouraging, and allowing for more exploration than evaluation.
As others have noted here, clients and corporations aren't always interested in being forward thinking, and that has to be okay because it is their program. One of the hardest things in eLearning is to take progressive trends and make them appealing to traditional-thinkers. But that won't stop me from trying!
There are two other "real world" factors that that are relevant to the discussion:
1) Cultural Influences
2) Business Practicalities
In a perfect world, with a perfectly designed quiz, a perfectly homogenous learner group, perfectly uniform reading/learning levels, and perfectly appropriate content for each learner... then yes 100% is the perfect passing threshold.
How many of us actually live in that world?
Most of us do not have the luxury of creating unique courses tailored for sub-segments of learner populations and extended learning times. Instead in today's diverse and fast moving business world we have mixed groups of employees with diverse backgrounds and skill sets. Think about it this way.... If a typical university with extended classroom time/personal interaction can not get all their fourth year mechanical engineering students to achieve perfect 100% on a quiz, why do we think our stand alone 30 min e-learning course can do so for an even more diverse group?
And cultural factors are in play as well. We've commonly seen learners in China for example where they will continue to take courses/quizzes repeatedly until they achieve a full 100% score - even when already passed and there is no tangible advantage. Conversely, in other countries it's not unusual for us to get push back that there is even any sort of minimum passing score at all!
In my humble opinion, this is one of those topics where we must balance our ideal model of how it should be with the a more pragmatic view of it is in the business world today and the many factors that go into learning in the workplace.
We recently had to justify our cut score, and no arbitrary selection -- neither 80% nor 100% -- was acceptable. A psychometrician* steered us toward the modified Angoff method. A search on the term will bring up lots of sources. Here is a link to an easily understood description that helped us. https://www.questionmark.com/sites/default/files/PDF/2012_pd_uscg.pdf
* A psychometrician is a person who practices the science of measurement, or psychometrics. The term psychometrics refers to the measurement of an individual's psychological attributes, including the knowledge, skills, and abilities a professional might need to work in a particular job or profession.
29 Replies
I love that title Bruce.
So glad I found this thread! I'm discussing this with someone now who, instead of opening up remediation options just wants to right "easier questions" and set the pass score at 100. *arg!*
I'd like to build on what Phil is suggesting and take this a step further. First, looking at the course content and purpose - is the score important to receiving a credits/accreditation? If not, and we are simply seeking understanding, I propose that in many cases a pass score isn't important at all. Why do we mistrust adult learners so still? I advocate for achieving deeper understanding instead of testing his or her ability to recollect. I'd like to see more answer exploration available so that the learner understands why an answer is incorrect as opposed to simply that it is incorrect. Adult learners are already seeking to understand. Most who get tripped up do so from OVER-thinking, considering the nuances of the question and its application. I am advocating that we harness and appeal to that instinct instead of punish it.
That would mean that either way, the learner can 'pass' the test, but the real learning will happen as they explore WHY their answer was right or wrong. They would still receive a grade, but the completion itself would be enough to pass the course. We would trust the adult learner to use the test as a learning tool, treating them less like a grade-school student and more like a learning partner in the course.
This style would not be applicable in all cases, but I believe strongly that we should continue to build on the momentum in adult education where we are increasingly opening content navigation, reducing restrictions, encouraging rather than discouraging, and allowing for more exploration than evaluation.
As others have noted here, clients and corporations aren't always interested in being forward thinking, and that has to be okay because it is their program. One of the hardest things in eLearning is to take progressive trends and make them appealing to traditional-thinkers. But that won't stop me from trying!
Older conversations, but good topic...
There are two other "real world" factors that that are relevant to the discussion:
1) Cultural Influences
2) Business Practicalities
In a perfect world, with a perfectly designed quiz, a perfectly homogenous learner group, perfectly uniform reading/learning levels, and perfectly appropriate content for each learner... then yes 100% is the perfect passing threshold.
How many of us actually live in that world?
Most of us do not have the luxury of creating unique courses tailored for sub-segments of learner populations and extended learning times. Instead in today's diverse and fast moving business world we have mixed groups of employees with diverse backgrounds and skill sets. Think about it this way.... If a typical university with extended classroom time/personal interaction can not get all their fourth year mechanical engineering students to achieve perfect 100% on a quiz, why do we think our stand alone 30 min e-learning course can do so for an even more diverse group?
And cultural factors are in play as well. We've commonly seen learners in China for example where they will continue to take courses/quizzes repeatedly until they achieve a full 100% score - even when already passed and there is no tangible advantage. Conversely, in other countries it's not unusual for us to get push back that there is even any sort of minimum passing score at all!
In my humble opinion, this is one of those topics where we must balance our ideal model of how it should be with the a more pragmatic view of it is in the business world today and the many factors that go into learning in the workplace.
Bob
We recently had to justify our cut score, and no arbitrary selection -- neither 80% nor 100% -- was acceptable. A psychometrician* steered us toward the modified Angoff method. A search on the term will bring up lots of sources. Here is a link to an easily understood description that helped us. https://www.questionmark.com/sites/default/files/PDF/2012_pd_uscg.pdf
* A psychometrician is a person who practices the science of measurement, or
psychometrics. The term psychometrics refers to the measurement of an individual's
psychological attributes, including the knowledge, skills, and abilities a professional
might need to work in a particular job or profession.
This discussion is closed. You can start a new discussion or contact Articulate Support.