Measure, measure, measure

When your organization or team measures a process, a product, or person, be sure to connect the measurement to something higher within the organization such as your core values, your core competencies, or certain behaviors. Without this connection, it will be challenging to show how any change impacts your ROI. You would just have random changes. Alignment before you develop drives buy-in as well since individuals can connect the expected bahavior change to a reason other than...because I said so...

Assessments measure knowledge of learners…

Knowledge and skill assessments should be based on the learning objectives for the class. The questions should be distributed proportionally across the learning objectives based on the percentage of content for each objective. The learning objectives themselves are best aligned with business objectives to determine the importance and priority of focus. The business objectives will need to be identified and developed prior to creating learning objectives.
Knowledge and skill assessments are a good source to measure what knowledge the participants have when they leave the learning environment. However, unless you conduct a pre-assessment, it can be difficult to determine if they learned it from the training or enter the class with the knowledge. Participants may not communicate their knowledge effectively in a pre-assessment even if they know the content, especially if the questions still need to be tested for construct (can the knowledge be used effectively) and criteria-related (is the knowledge being used by top performers) validity.

Application evaluations become more difficult when you try to educate a group who might be familiar with the content. How do you know when behaviors stem from a class or learning event you devised? How do you know that your content didn’t trigger an approach participants learned at an earlier stage in their careers? At times, we may not know if someone’s past experience aided him/her in understanding material better. All this should be taken into consideration before a level of competency, application, and ROI are calculated and set.

Satisfaction results are summary points…

After a training class is complete, the first thing we need to do is examine the satisfaction surveys…or, at least that is what so many training professionals do. You may skip to the end and read the comments. Or, look at one particular question or set of questions to determine if your training was successful. This would be a daunting task for even the most mathematical minded among us.

Unless you can calculate correlations and build scattergrams in your head, understanding the success of your training based on quick glances at pieces of paper will probably not occur. What does happen, however, is that we create an imaginary summary and decide to change our approach and sometimes, even modify the content for the next class. This can be hazardous to your training and driving your positive ROI.

Remember to recognize the impact of your approach, delivery, and design. If training is changed constantly and the content shifts, you will lose the largest group of reinforcement you may have…the learners themselves. You may also create more work for yourself, as the job-aids will need to be different from class 1 to class 20. It is a good idea to conduct pilot classes and change content from there. Remember though to bring backthose people when the final class structure is set. Every time you change, you have to determine what influenced the change. Basic statistics can help with this. However, you will need to decide when to use it. Is it in the revision of the training for a new hire class? Or, is it within your reinforcement plan? The goal is to minimize impact and maximize application in this situation.

People ensure learning will be successful…

By aligning your training objectives to business objectives, you help fill gaps in content from a learning perspective. You can conduct multiple needs assessments and analyze the data until the cows come home…however, since most live in a city environment now, you may be analyzing an awful long time. You can even make sure the content aligns proportionally to the greatest and least need of your organization. This still will not ensure success. It only helps it along.

Learning is a challenging task and it takes people to make it work well. Sure, you can have online, web-based, PDA learning events, but someone, somewhere, has to put the event together. An even greater number of people are needed to hold things in place after participants engage in the learning. As adults, we have to link some importance to content before it replaces something else in our short-term or long-term memory. People, whether they are leaders, trainers, or executives, need to connect the dots for us after we attend a learning event by talking about how it relates to our everyday work.

We can look toward measurements to help integrate the learning experiences. Through satisfaction measurements, we are able to understand what marketing of the training needs to take place for the next group. Through knowledge assessments, we are able to revise or reinvent how content is delivered to a group after they attend the initial training so that we can build off of it. With Level 3 surveys, we can recognize the impact of our efforts and minimize the effect of work-arounds or lack of focus on our ROI.

Measuring what is not applied creates more work for those who are measuring…