Assessing Employee Skills: Socratic Learning for the 21st Century

In the days of Socrates and Plato, tutors taught their students by systematically asking them probing questions. According to Plato, the instructor should feign ignorance of the subject, to elicit dialogue from the students. Attempts to answer questions would uncover gaps in student knowledge, contradictions, and poorly formed ideas. Socratic questioning encourages critical thinking. Plato believed that it encouraged students to develop the deepest possible knowledge of a topic.

Nowadays Socratic questioning is sometimes used by trainers in group discussions, but it cannot be used at scale for assessment. Instead, for the last 100 years, we have relied heavily on multiple-choice tests. Although the technology for delivering multiple-choice tests has changed, such tests have fundamental limitations.

Multiple-choice questions do not test learners’ ability to solve problems; they test the ability to recognize plausible answers when presented in a list. Learners who can only recognize answers have not even reached the lowest level in Bloom’s taxonomy of educational objectives. Such learners will be ill-prepared to apply what they learned on the job. Imagine what would happen if a mine worker were faced with an emergency such as loss of ventilation, and was unable to decide what to do without a multiple-choice list of options to choose from.

Another problem with multiple-choice testing is that it tolerates ignorance. Learners won’t know all the answers and are expected to guess their way through them. Socratic questioning is very different; it exposes the gaps in learners’ knowledge and understanding and motivates learners to learn more deeply so next time they will be able to answer questions successfully.

At Alelo we have developed technology and methods for avatar-based spoken assessments. Learners are presented with challenges, either as written prompts or as questions posed by the avatar. They must come up with answers on their own and are evaluated on the quality of their answers. Probing questions such as “why” and “what if” highlight gaps in the learner’s knowledge and motivate learners to go back and study some more. Online learners who skim through the material are easily tripped up when an avatar asks them probing questions.

One advantage that avatars have over Socrates is that avatars can play a variety of roles; they do not have act like an instructor feigning ignorance. Co-worker avatars give learners practice explaining what they learned (resulting in what is known as the protégé effect), or presenting their ideas. In healthcare, patient avatars give trainees practice explaining medical concepts in simple terms. Learners are not afraid of making mistakes when answering questions posed by avatars, which builds self-confidence.

Alelo is developing avatar assessments to test mastery of critical skills such as operational safety in mine operations or patient interviewing skills in healthcare. They help prepare trainees to apply what they learned on the job, resulting in fewer mistakes and greater occupational safety.