How can managers track training results? Have staff achieved the training objectives? Are you satisfied with what the training has accomplished? These questions have dogged trainers, performance improvement professionals and managers since the earliest days of industrial adult education at about the turn of this century.
Many adult-learning and management theorists have made efforts to provide tools to help answer these questions. The most practical, as well as and popular of these tools is the Kirkpatrick model by Donald Kirkpatrick.
The Kirkpatrick model presents four levels of training evaluation.
Level 1
Participant reactions to training and the training environment.
Level 2
Learning acquired by participants through training
Level 3
Application of learning to on-the-job situations.
Level 4
Impact of training on business performance.
In this model, each successive level provides more credible data. Consequently, this permits an objective evaluation of the impact of training. Also, each successive level becomes more complex and more time-consuming. The following case history uses levels 2 and 3 analysis.
A case history from the Midwest
“T”, a Dublin-based trainer was asked to develop and present leadership skills training at “M Ltd.” M Ltd was a small (20 employees) manufacturing facility in the Midwest. T trained three key employees: a production manager, a supervisor and a senior employee.
The training had three goals. Firstly, to train participants to identify and resolve interpersonal and technical problems. Secondly, to manage conflicts and disagreements more effectively. Finally, to teach participants to communicate more clearly with subordinates, with each other and with management. The average formal educational level of the trainees was Pass Leaving Cert. With the exception of one participant, none had participated in any formal skills training.
The programme was 32 hours in duration. Of this, 20 were classroom training and 12 were one-on-one coaching. Coaching was to help employees transfer the skills acquired in the classroom to the shop floor. Coaching sessions took place once a week. These covered that week’s training objectives and were personalised to the needs of each participant. “G”, the general manager of the facility, reinforced the classroom training by conducting a series of evaluation meetings.
Purpose of the Learning and the Evaluation Meeting
The evaluation meeting is fairly straightforward in both design and execution. It helps managers assess which skills employees have acquired and if they are using them on the job. It’s also allows line managers to evaluate training. Such a format clearly demonstrates top management’s interest in training’s success and support for training efforts. It also lets trainees know that management is interested in their performance during and after classroom training.
G told the participants that he wanted to meet with them two weeks after their training had ended. He wanted to determine how much they had learned. G asked the three trainees to present individual 30-minute summaries of key ideas or skills acquired during training. He told them to decide among themselves which topic and skills areas each would present.
G also offered to assist them preparing their presentations. He would help them to create outlines as well as provide them with supplies and administrative support. About a week later, G discovered that the trainees had not agreed on how to make their presentations. He asked trainees to refer to their training manuals and divide the topics among themselves by subject matter so that each participant presented approximately one-third of the training content.
Level 2: Learning and the Evaluation Meeting
At the evaluation meeting a week later, there were three presentations. The first trainee presentation covered modules one through four addressing interpersonal communication, leadership, conflict resolution and delegation. The second presentation of modules five through eight addressed motivation, discipline, problem solving, train-the-trainer skills and legal issues. Finally the third presentation dealt with the how to use the Ishikawa diagram to solve work problems.
During each presentation G asked questions: What was the most important skill you learned? What are the most common conflict-resolution styles that you see us using here at M Ltd.? How do you use the four basic communication styles with the people you supervise?
Level 3: Changes on the Job as a Result of Training
Overall, results were mostly positive. The employees reported that the training had enhanced their ability to communicate and listen. There was evidence of improved communication at work. Managers and employees spend less time locating materials, messages and equipment.
Better communication also improved message flow and reduced lost time. For example, after training G instituted 30-minute overlap meetings between day- and night-shift supervisors. Previously, shift overlap lasted no more than five minutes, which resulted in confusing or missed communication. With increased overlap, there were fewer scheduling, machine operation and personnel issues. This resulted a smoother turnover of work shifts.
The trainees introduced Monday morning meetings among the general manager and the production manager, as well as the supervisor and the senior employee. These looked at issues and anticipated problems for the upcoming week. Prior to training, these meetings were sporadic and did not always include all key personnel.
How to improve the process
T submitted a report at the conclusion of training. The trainer recommended that managers and the three trainees needed to develop formal job descriptions. These would define the trainees’ duties and responsibilities. T also noted that management and trainees needed to define performance and career-development goals.
Additional recommendations included formally training them in basic quality tools, developing work teams, and training temporary employees and other shop-floor personnel.
The evaluation meeting was a useful way to assess the overall value of training to M Ltd. Though management and those involved in the training agreed that the training was worth the investment, future evaluation meetings could be improved in a number of ways.
Recommendations
M Ltd has a number of training recommendations. Here they are.
Firstly, advise participants on how to give a presentation. This should include coaching on how to select appropriate presentation media and how to prepare an outline.
Allow participants to rehearse their presentation. This preparation is necessary if the evaluation meeting is to yield desired results.
Thirdly, discuss what each person expects from the evaluation meeting prior to the meeting. Trainees and managers should agree on expectations and prepare an evaluation meeting agenda reflecting these expectations.
Furthermore, document results from the evaluation meeting. This could also serve as the basis of discussion between management and key personnel about career development needs.
Finally, define the purpose of the coaching. The contribution of coaching to the overall effectiveness of this training effort was unclear. This was partly because the evaluation meeting did not directly address what the coaching sessions taught. Consequently, there was no format to evaluate the extent to which coaching did or did not supplement the classroom activities. Future coaching efforts be more formally and explicitly linked to classroom work. In addition, the evaluation meeting should assess the coaching’s relationship to classroom accomplishments.
Conclusion
Managers at M Ltd. concluded that the evaluation meeting was a cost-effective way to assess training impacts. They also thought that it permitted them to draw conclusions about the value of training to the organisation. In addition, the training effort clearly demonstrated management’s interest and support in training and developing employees.
————————————————————————————-
Article submitted by Aoife Rogers
Corporatetraining. ie is interested in receiving further case histories on training interventions, with suggestions on what went well and on what could be improved. Please send your articles to info@corporatetraining.ie
Comments