Al Feldzamen sparks a whole bunch of controversy with a post on his "unexpected truths" blog, the "fallacy of hard tests".The premise is that the harder the MC test the unfairer it is. Even with a couple of mistakes in the math, the post was controversial enough to raise a whole lot of debate around the effectiveness of MC testing.
From my perspective maybe we should be asking the question "is MC testing the way to go full stop?" With its emphasis only on knowledge, and with known issues around MC testing methodologies (a lot of which are highlighted in the blog post comments) is this approach good enough for the support of personnel certification programs?
I personally have recruited a few "certified" professionals (certification to remain nameless) over the last few years. To be honest they didn't know which way was up. They might have had the body of knowledge but it's application in the workplace was poor to non-existent. A complete waste of my time, $$ and the reputation of the professional organization down more than a few notches.
But what are the alternatives? Existing skills and competency based testing is expensive and inconsistent. What other tools and techniques are people looking to for the examination and testing of their employees and professionals?
Unexpected Truths: The fallacy of hard tests
... (continued from beginning of blog post)
“Was it all multiple choice?” I asked. “And how did they grade it?” I was thinking of my own exams. “Did they count only the right answers.?”When he said Yes to all the questions questions, I did not have the heart to tell him what I knew as a mathematical certainty—that the exam was, like most graduate medical exams, and large parts of legal licensing bar exams in most states , virtually a complete fraud.
The reason these tests are fraudulent—and the harder they are, the more they are fraudulent—is that for an extremely difficult test graded in that way, guessing tends to count much more than knowledge... more
