Training effectiveness and evaluation: Everything you need to know

What is Training effectiveness?

Training effectiveness refers to the impact that a training intervention has as measured by the achievement of the goals set before the training intervention.  To enable the assessment of whether the training was effective or not the goals for the training intervention must be agreed upon upfront.  In practice, I have noted that many organisations spend millions of dollars on training without necessarily checking whether the training is bringing the desired results or not.

Training effectiveness largely depends on how the training is designed, delivered and implemented.  The other key facet of training and its effectiveness depends on the transfer of what has been learnt to the job. This specifically speaks to whether the individual who has attended the training is doing the job better after the training and whether the trainee has more knowledge after the training than before. Without encompassing the above key facets, the training investment is wasted. What seems to be emerging is that training effectiveness is impacted by several factors before, during and after the training.

Organisations are investing a lot of money in training and development activities. Some of the training interventions are triggered by the need to adapt to the changing environment especially those changes brought by technological change.  Organisations are aware of the importance of training as it brings a competitive advantage to the business when done properly.

Even before focusing on training effectiveness, it is important to note that research shows that training works. The United States is estimated to spend $135 billion a year on individual training( Patel, 2010).  To benefit from training, it is recommended that organisations pay attention to what happens before, during and after training when designing training interventions. Put differently the effectiveness of training interventions depends on what happens before, during and after the training.

Training effectiveness: Training needs analysis matters

We can not talk of training effectiveness if the groundwork is not done during the early stage of training needs analysis. The identification of training needs lays the foundation for training evaluation and establishing the effectiveness of such training. The assessment of training needs enables some key processes such as laying out the expected training outcomes, hints of how the training should be designed and delivered, and ideas on how the training will be evaluated.  In the same assessment is it important to look at the environmental factors that are likely to impact the delivery of the training?  Here are some of the key points to take note of based on research as these affect the effectiveness of training interventions.

• Do not rely on asking people what training they need to do their jobs effectively. Research by Baddely & Longman (1978) shows that employees are not able to identify with accuracy what training they need.
• A thorough training needs based on task analysis should be the basis for identifying training needs. The advantage of this approach is that you would then be able to use the information on what to include in the training and also for establishing training standards.
• With more advanced approaches to training needs analysis such as Cognitive Task Analysis, it is now possible to identify the cognitive strategies required to perform a certain task. The findings from such an analysis can influence the design of training in a good way.
• Training needs analysis has also started covering the need to work in a team environment and this should be incorporated in the training needs analysis as most tasks are done in a team environment.
• There is a need to do an organisational analysis as part of the training needs analysis. The organisational analysis will mainly uncover issues related to the organisation’s strategic priorities, culture and any resource limitations that are likely to impact the way the training intervention will be implemented. In this same process, you should identify those roles that have been proven to have the biggest positive impact on the performance of the business.  In the organisational analysis, it is important as well to look at the general work environment and identify likely obstacles to the training intervention and work towards removing them.
• Person Analysis – At this stage, you are now identifying who cannot do the job based on the output from the Task Analysis. Other researchers recommend that at this stage you should also look at individual factors that may impact the training such as individual personality and motivation. The ultimate goal of carrying out a person analysis is to be able to ascertain the individuals most likely to benefit from the training and whether there is a need to customize the training to accommodate individual characteristics.

Evidence on training effectiveness

So far meta-analytic studies on the effectiveness of training have consistently proved that when the training intervention is designed systematically and grounded in the learning and training science, it has always shown effectiveness or has a positive impact.  Below I present the findings from several meta-analytic studies on the impact or effectiveness of various training interventions:

• A study by Bennet, Eden and Bell(2003) shows that organisational training had an overall effectiveness of 0.62 and when using result criteria the impact was 0.62. The same study showed that when assessed gains the learning criteria, the impact was 0.63.
• A study by M.J Burke and Day (1986) shows that managerial training had an impact of 0.67 on the objective results criteria and 0.38 on the objective learning criteria.
• Sales et al. (2008) show that the impact of team training on performance outcomes was 0.39.
• A study by Powell and Yalcin (2010) showed that managerial training had an impact of 0.17 on results objectives.
• A review of 66 papers on training done between 1991 and 2007 shows a positive relationship between training interventions and such outcomes as; productivity, sales growth, market share, work performance and customer satisfaction.
• A review of 36 studies on the impact of training interventions on financial performance shows that 22 had a positive effect, 8 mixed effects, 5 had none and one had a negative impact.
• A study by American Bankers (2004) on 17 banks showed that banks in the top 50% of training expenditure per employee performed better than other banks across all indicators.
• Bassi (2002) in a study of 575 public companies found that companies in the top half of training expenditure per employee had a Total Share Return of 37% versus 20% for firms in the bottom half.
• Bernthal (2006) in a study of 127 firms with high-quality leadership training performed better than others across all indicators.
• In a meta-analysis of 70 empirical studies, Burke and Day(1986) reviewed the effectiveness of management training across various indicators and the results showed a positive impact of training. In another review of 83 studies by Collins and Holton(2004), they found positive effects from management development on knowledge outcomes and actual leader behaviour. These results seem to point to the effectiveness of management and leadership development programs.

The above analysis clearly shows that when training is done properly and systematically it is very effective.  Organisations that invest huge sums of money in training would benefit from following some of the advice given above.

How to do events before the training affects training effectiveness?

• A study by Smith et al (1996) clearly shows that trainees who experienced negative pretraining events in the areas to be trained benefited from the training better than those without such exposure. To further illustrate this point, let's take for example that an employee who struggles with handling difficult customers and they experience such an incident before they enrol for training to address such a challenge, that trainee will benefit more from the training.
• In other research, it has been shown that trainees who think that their work environment is supportive of them utilizing the skills they are going to learn are likely to want to attend training and are more motivated.
• A study by Baldwin et al (1991) showed that trainees tend to hold more favourable attitudes towards training when they are allowed to contribute towards the design of the training and when they are allowed to choose when they can attend training compared to when they are commandeered to attend the training.
• When trainees think that the training is beneficial to them they are more willing to put effort into the training and are generally more motivated during the training ( Noe and Schimdt, 1986).
• Research has shown that when participants in a training program discover that the training they are attending is not useful, they lose motivation even after initially committing to attending the training.

Training effectiveness: The impact of the learning climate

• Research shows that training participants who indicate that their needs have not been met by the training tend to show lower post-training commitment(Krager and Kanar, 2009).
• The lesson from the above statement is that do not oversell any training program especially hyping the benefits and in the process create falling expectations. It is advisable to have realistic expectations on what the training will do to assist individuals to do their jobs better. Any training intervention that is sold as a miracle cure but fails to meet the expectations of the trainees will create challenges that are difficult to correct.
• How individuals are notified about the impending training matters. Baldwin and Magjuka (1991) noted that advance notifications that also indicate that there will be a formal follow-up to check on the progress of skills learnt to increase the participant’s intention to utilize skills learnt in the training.
• Ford et al 1998 noted that training that is presented as an opportunity to acquire skills useful for job performance has better outcomes compared to training that is presented as a test.
• As you organize your training interventions you must note that research evidence shows that training presented as mandatory is viewed more positively than optional, on condition that the general training climate in the organisation is positive. The impact is coming from the fact that participants are likely to evaluate training presented as mandatory as being more important.
• A study by Baldwinin et al(1991) shows that ignoring the input of employees after asking them for their input on what training they would like to attend is worse than not asking them.
• Researchers in the field of training recommend that when the training climate in the organisation is good, use the mandatory label to enhance training effectiveness.
• Research by Smith- Jentsch et al (2001) showed that reckless comments by the supervisor could wipe out the effects of a training intervention. The role of the supervisor is key here; Supervisors must be given enough information to enable them to prepare trainees for training and reinforce what has been learnt.
• One of the biggest obstacles to training effectiveness is training decay. McNelly et (1998) looked at studies on skills decay. They found that soon after training there were no incidences of skills decay found. When they looked at a year after the training they found out that trainees had lost 90% of what they had learnt. Arthur et al noted that retention levels varied by the nature of tasks. High levels of skills decay were noted in the cognitive tasks when compared to physical tasks. High levels of skills decay were noted in longer periods of nonuse or practice of the skill learnt.
• To enhance training effectiveness by preventing skills decay there are certain actions management can take. One way is to schedule training close to when the employee would be required to use the skills learnt. This will enable the employee to continuously use the skills they have learnt.
• Employees need the opportunity to utilize the skills before they lose them. One way to present skills decay and enhance training effectiveness is to offer refresher training.

Training effectiveness: The influence of trainee characteristics

It is known that when people attend training they bring with them individual characteristics that may impact the effectiveness of the training. These characteristics include things like goal orientation, motivation and self-efficacy. Many practitioners rarely look at the individual as an enabler or an obstacle to training effectiveness. There is enough scientific evidence to show that the trainee plays a big part in training effectiveness.

The importance of the post-training environment to the effectiveness of training

• On the transfer of training, research shows that a supportive environment matters in facilitating the transfer of training.
• Other research has shown that reflective goal setting after the training helps to keep the skills that have been learnt currently.
• When trainees are allowed to practice the new skills learnt when they are back on the job the outcomes are better (Sego & Smith, 1995). When trainees are not offered the right environment to practice the new skills learnt in the training, they tend to forget.

In summary, to get the best out of any training intervention there is a need to address the following issues:

Before training

• Conduct training needs analysis, which should cover actual task analysis, organisational analysis and person analysis.
• Prepare for the right learning climate; This includes paying attention to the scheduling of training. It is important to schedule training at a time when the employee’s attention to the training can be fully harnessed.

During the Training

• The ability to motivate the trainees is very important. As we have already seen in other parts of this article, when participants in a training program are not motivated, they are likely to benefit from the training.
• It is important to follow the right instructional principles as informed by learning theory. As an example training programs with an experiential component tend to be more effective than training programs that do not use this approach.
• Harness the advent of digital technologies as you deliver your training.

After the training

• Work on creating an environment that facilitates the transfer of training. This important step includes removing obstacles that would hinder the trainee from practising what they would have learnt. There is a need to ensure that the necessary resources for the employee are available.
• The last part is to evaluate the effectiveness of the training. When doing the evaluation link it to; the original purpose, and training needs and the evaluation must be done at multiple levels.
• Practitioners must remember that what matters in any training intervention is the transfer of what has been learnt to the work situation. This has been a perennial challenge for a very long time. The reason why the transfer of training is a problem emanates from challenges encountered during the whole training cycle.
• Research by Rouiller and Goldstein (1993) shows that a supportive environment is key in the transfer of skills learnt. This includes support from the immediate supervisor.
• In a study by Tracy et al (195), they discovered that post-training climate especially supervisory support was critical in the transfer of training. In the same study, they discovered that organisational culture and climate had a bearing on post-training outcomes.
• Generally, the work environment must have a structure that allows the employee to practice what they would have learnt in training.
• Ford et al (1992) noted that the transfer of learning is related to the availability of opportunities to practice.
• Over and above the opportunities to practice they also discovered that several factors in the organisation play a part in the transfer of skills learnt. These include support from the supervisor, opportunities to utilize the skills, and social and peer support.
• Tannenbaum (1997) noted that 7% to 9% of skills acquisition in organisations come from formal training. The rest of the skills come from on-the-job training.
• It has been found through research that debriefs play a key role in bringing effectiveness to training. The major value of the debriefs comes from the fact that they allow the trained employee to self-correct and reinforce whatever has been learnt. It has been reported in research findings that in the military for example those teams that used debriefing as part of their training witnessed more training effectiveness as they outperformed their comparable teams by 40%.

Training effectiveness through evaluation of training

The evaluation of training is premised on two key factors; whether the training met the training objectives and whether actual job performance changed as a result of the training intervention.  Training evaluation can be assessed at the following levels (Kraiger et al 2002):

• Does the trainee know more after the training (declarative knowledge)?
• Is the trainee now doing things better after the training?
• Attitudinal change – whether there has been an effective positive change?

The training and development community has largely depended on Kirkpatrick’s framework for the evaluation of training.  Kraiger’s model seems to be a better model for the evaluation of training outcomes.

Conclusion

Training effectiveness is a function of several factors starting with doing proper training needs assessments and creating the right environment for trainees to practice the learned skills. If these key steps are not done properly, training effectiveness is compromised.

Memory Nguwi is an Occupational Psychologist, Data Scientist, Speaker, & Managing Consultant- Industrial Psychology Consultants (Pvt) Ltd a management and human resources consulting firm.

Email:mnguwi@ipcconsultants.com or visit our website www.ipcconsultants.com