Tips for Writing Meaningful Level 1 and Level 3 Evaluations

Evaluations help measure the impact and effectiveness of learning experiences. The LMS has two levels of evaluation built in: Level 1 and Level 3. Some best practices for writing these evaluations include:

• Write from the learner’s perspective
• If you can’t fix it, don’t ask about it
• Only label each end of your Likert scale, not each step
• Use the same scale throughout and use an odd number of points
• Small numbers on the left, larger numbers on the right
• Write statements, not questions
• Always include a relevance question
• Don’t ask if you don’t need it
• Use open comment only if you need the data for a defined purpose and are going to analyze it
• Obtain a quantitative and qualitative measure overall
• Keep it short and simple

Level 1 Evaluations

Level 1 evaluations are meant to measure immediate reactions to a learning resource. A ratio of 20% questions on the experience and 80% questions on the content is recommended. To capture discursive responses, a free form text field might be included, or you might ask for a comment for any score lower than 3.

Possible Topics for Level 1 Evaluation

• Learning Objectives
• Course Materials
• Content Relevance
• Facilitator Capabilities
• Facility or Learning Environment
• Overall Evaluation

Level 3 Evaluations

Level 3 evaluations are meant to show whether the training resulted in changed behaviors on the job or in the learner's everyday environment. You are looking for the impact of the training. To do this effectively, an evaluation strategy needs to be in place, along with a standardized process for communication to appropriate decision-makers. Pre-work to conducting useful Level 3 evaluations includes:

• Learning objectives that clearly link to critical on-the-job work behaviors that are expected to change
• Pre-program/course data collection on these behaviors so that change can be measured
• Determining when behaviors will be measured (3 – 6 months between first measurement and evaluation recommended)
• Awareness of other factors that might influence behavior changes
• Using a control group, if possible

Once this is accomplished, the method for evaluation needs to be determined using these three questions as a guide:

1.  When to evaluate the first time?
2.  How often to re-evaluate? (at 12 and 18 months, for example)
3.  How to evaluate?

Options for evaluation include pre- and post-tests, comparison groups, self-reporting, peer evaluation, supervisor evaluation, trained observer evaluation, interviews, surveys, on-the-job observations.

Indicators or metrics of criteria to use:

• Goal-based behaviors
• Competency-based behaviors
• Knowledge, Skills, and Attitudes (KSA)
• Intention Goals and/or Beliefs

Related Links

Evaluation Process

Creating a New Evaluation

Evaluation Questions

How to Create a Likert Scale

Other Evaluation Options