Conduct scholarly research to compose a 2-3 page APA formatted paper that addresses these issues. Review the grading rubric for expectations.
Conducting evaluation studies using full experimental designs and sophisticated data collection is almost impossible in real organizations.? Some managers argue that training evaluations can provide meaningful conclusions only when conducted using these techniques.?
They therefore conclude that in most cases training evaluations are a waste of time and money.?
Debate this conclusion:? Is it the case that training evaluations should be conducted only when it is possible to use the more sophisticated procedure??
Why do organizational leaders need to see quantifiable results and how can these results be presented in a reliable format?
This assignment will address the following Unit Learning Outcomes:
MBA635 Unit Assignment You've already rated students with this rubric. Any major changes could affect their assessment results.
- ULO 6.1 Assess the different types of training evaluation and discuss the barriers to evaluation.
- ULO 6.2 Explain how training evaluation can be quantifiable.
Content 127_1488 view longer description threshold: pts Edit criterion description
This area will be used by the assessor to leave comments related to this criterion.
Addresses specifics of assignment demonstrating critical thinking abilities by synthesizing research. A minimum of 4 scholarly outside sources used.35.0 pts blank Edit rating Delete rating Add new rating category after current rating Demonstrates understanding of content but greater deveopment needed. A minimum of 3 scholarly outside sources used.29.0 pts blank_2 Edit rating Delete rating Add new rating category after current rating Basic understanding of weekly topics. A minimum of 2 scholarly outside sources used.20.0 pts 127_1327 Edit rating Delete rating Add new rating category after current rating Opinion based commentary with one or no scholarly outside sources.0.0 pts 127_5865 Edit rating Delete rating Add new rating category after current rating pts
Chapter 11 Training Evaluation Chapter Learning Outcomes
After reading this chapter, you should be able to: ? define training evaluation and the main reasons for conducting evaluations
? discuss the barriers to evaluation and the factors that affect whether or not an
evaluation is conducted ? describe the different types of evaluations
? describe the models of training evaluation and the relationships among them
? describe the main variables to measure in a training evaluation and how they are
measured ? discuss the different types of designs for training evaluation as well as their
requirements, limits, and when they should be used NEL 323 324 Managing Performance through Training and Development Bell Canada
Years ago, when Bell Canada installed a new telephone system for its business clients, it also sent
out service advisers whose task it was to train the
employees to use the new system. These training
sessions consisted of ?show-and-tell? activities in
which the instructors demonstrated the use of the
telephone. Simple as the training was, it was expensive, costing millions of dollars annually. With the
introduction of electronic equipment, the functionality of the telephone systems?and complexity for
the users?increased exponentially.
Initially, the company attempted to use its tradi?
tional training approach with purchasers of the electronic systems. However, a training evaluation was
conducted and it showed that following the training
experience, customer knowledge of the operation
of the electronic telephones was quite low. Training
was not effective.
A number of attempts were then made to
improve the situation. Different types of training,
presented by either Bell Canada or user personnel,
were tried and evaluated. None made any significant difference in terms of training effectiveness.
However, these training evaluation studies did
detect an important fact. No matter how training
was conducted, the users? knowledge of a limited number of functions?those they used a lot?
increased after training, indicating that practice
seemed to have a significant effect on learning.
This suggested that providing end users with
an instructional aid might help them gain greater
benefit from the electronic system. To that end, a
special instruction booklet was carefully prepared
and trainees were provided with a brief instructional session teaching the users how to use the
instruction booklet. The evaluation of this approach
showed, empirically, that the use of the instruction
booklet resulted in greater user mastery than the
formal training course.
Thus, the training evaluations conducted
throughout this process demonstrated a) that the
traditional training method was ineffective, b) that
changing the instructors had no effect, but c) that
the use of a well-developed instruction booklet had
This demonstrates the two main objectives
of training evaluation: to assess the effectiveness of
training and, equally important, to identify ways of
enhancing that effectiveness. At Bell, the traditional
program was discontinued and replaced with an
inexpensive booklet that was both more effective
and considerably cheaper. Training programs are designed to have an effect on learning and behaviour.
However, as the Bell Canada story demonstrates, this is not always the case.
Fortunately, in that case, the organization launched an evaluation program that
involved several studies, the results of which served not only to assess the effectiveness of the existing training but also to identify and test different strategies
for improving the situation.
In this chapter, you will learn several training evaluation models, the many
types of evaluations, the variables to measure and how to do so, as well as the
strengths and weaknesses of different data collection designs used in the ?conduct
of evaluations. NEL Chapter 11: Training Evaluation What Is Training Evaluation?
Organizational training and development is intended to improve technical competencies (e.g., learning new software), to modify attitudes (e.g., preparing a manager
for an international assignment), and/or to modify behaviours (e.g., better communication skills). Organizations invest in this organizational function because it is
expected that training makes a positive contribution to them and to their employees.
Training evaluation is concerned with whether or not these expected
?outcomes materialize as a result of training. They are designed to assist decision
making: Should the organization cancel or continue a training program? Should
it be modified? How?
Training evaluation is a process designed to assess the value?the ?worthiness?
of training programs to employees and to organizations. Training evaluation assesses
this value by analyzing data collected from trainees, supervisors, or others familiar
with the trainees and with the job context. Using a variety of techniques, objective
and subjective information is gathered before, during, and after training to provide
the data required to estimate the value of a training program.
Training evaluation is not a single procedure. Rather, it is a continuum of
techniques, methods, and measures. At one end of the continuum are simple
evaluations that focus on trainee satisfaction: Trainees indicate on a brief questionnaire if they liked the course. Certainly simple and easy to implement, they
regrettably yield minimal information.
At the other end of the training evaluation continuum lie more elaborate
procedures and more complete questionnaires and interviews that provide managers with more information of a richer quality about the value of a training
program. These evaluations may assess job performance improvements and even
isolate the unique role that training played in it. Others assess the psychological states of trainees immediately after training: Have they learned the skills?
Are they motivated to apply them? Are they confident they can? Will the work
environment help or hinder them? The more sophisticated the evaluations, the
better the information they provide, the surer the conclusions and the greater
the ?confidence with which they can be stated. However, more sophisticated evaluation procedures are more costly and
more complex and difficult to implement. Conducting elaborate evaluations may
entail disruptions to the training program and/or to the job. Specialized consultants may be required in some cases. But these factors are not required in every case. At times a simple and cheap
evaluation procedure will do while a more complex and sophisticated approach
might be overkill.1 In other cases the reverse is true. In the end, training evaluation choices are a trade-off, balancing between quality and complexity/costs,
between the informational needs of decision makers and the difficulty and
resources required to obtain that information. Why Conduct Training Evaluations? Organizations invest in the training of employees and managers as it is a necessity for competitiveness in the current global environment. With chronic understaffing (one result of cost-cutting layoffs), the amount of time available for NEL Training evaluation
A process to assess the
of training programs
to employees and to
?organizations 325 326 Managing Performance through Training and Development training has become smaller and must be used more wisely. In this context, management has a stake in ensuring that the resources invested in training bear fruit.
Training evaluation is therefore of value to:
?? Help fulfill the managerial responsibility to improve training. ?? Assist managers in identifying the training programs most useful to
employees and to assist management in the determination of who should
be trained. ?? Determine the cost benefits of a program and to help ascertain which
program or training technique is most cost-effective (see Chapter 12). ?? Determine whether the training program has achieved the expected results
or solved the problem for which training was the anticipated solution. ?? Diagnose the strengths and weaknesses of a program and pinpoint
needed improvements. ?? Use the evaluation information to justify and reinforce, if merited, the
value and credibility of the training function to the organization. Do Organizations Conduct Training Evaluations? By the year 2000 most organizations in North America were conducting some
evaluation of most of the training programs offered to their employees.2 However,
there has been a gradual decline in the evaluation activities of organizations. The
survey of the American Society for Training and Development (ASTD) and the
latest Conference Board of Canada survey of organizations confirm this result.3
In 2002, 89 percent of Canadian organizations assessed training in some manner,
but by 2010 fewer than 50 percent still did. This significant decline occurred
even though the actual proportion of staff time invested in evaluation remained
steady (hovering between 4 percent and 10 percent). Notwithstanding the fact
that the same evaluation time is spent on fewer courses, the level of sophistication of the evaluations has not shown demonstrable growth. Most evaluations
are reaction based.
As depicted in Figure 11.1, although many organizations have abandoned
evaluation, almost half have not. Most (92 percent) of these organizations gauge
training success by soliciting the trainees? opinions (their ?reactions?), and they
do this for most (80 percent) of the courses held. Figure 11.1 also shows the
rarity of organizations that measure learning, behaviour, or job performance
improvements to evaluate training success. Such evaluations are more useful,
but they are more complex. Longitudinal data collected by the Conference Board
indicates little change to these percentages since the survey began a decade ago:
the percentage of organizations that evaluate reactions has remained the same. The new finding, however, is the confirmation that organizations are not
simply abandoning reaction measures: they are abandoning evaluation altogether! Perhaps dissatisfied with reaction-based evaluations, and unable or
willing to consider the use of better approaches, many organizations have chosen
to forgo training evaluation. NEL Chapter 11: Training Evaluation F i g u r e 11.1
The Percentage of Organisations that Evaluate Training and the
Percentage of Courses Evaluated at various Evaluation Levels
70 % of organizations that
conduct evaluations 92 % of courses
50 49 40 39 30 28 20 22 10
0 Reaction Learning Behaviour Results Source: C. Lavis, Learning and Development Outlook 2011: Are Organizations Ready for Learning
2.0? The Conference Board of Canada, 2011. Barriers to Training Evaluation
Studies of training professionals showed that some employers do not conduct
training evaluations because they are perceived to be too complicated to implement, too time consuming, and/or too expensive.4 Indeed, this may explain why
companies that conduct evaluation choose the simplest of procedures (reactionbased evaluations). In some cases, training managers resist evaluation because of
the difficulty of isolating from other variables the unique impact that training has
on employee effectiveness. In yet other cases, training managers do not conduct
evaluations because top management does not demand them, while others do
not because they may not wish to know. Thus, barriers to training evaluation fall
into two categories: pragmatic and political. Pragmatic Barriers to Training Evaluation Increasingly, training departments, as with all other departments, are expected
to demonstrate their usefulness, their contribution to job performance improvements, and ultimately to the company?s bottom line. Based on the ASTD and The
Conference Board of Canada reports, the training departments in most companies
are finding it a challenge to fulfill this obligation. Collecting training success data
that addresses management?s needs requires more complex evaluation procedures. Fundamentally, evaluation requires that perceptual and/or objective information furnished by trainees, their supervisors, and even others?such as peers,
subordinates, and/or clients?be gathered before, during, and/or after the
training session. These data gathering and analytical efforts require extensive
collaboration from the trainees, their supervisors, and so on, which is disruptive
and understandably difficult to obtain. NEL 327 328 Managing Performance through Training and Development Some training departments do not assess training for a different reason. As
you will learn in this chapter, taking on the training evaluation task requires knowledge about evaluation models, research design, measurement, questionnaire construction, and data analysis. For some, that can seem an intimidating prospect. Evaluation also costs money. As the effectiveness of a training department
is often defined by the number of classes offered or the number of participants,
siphoning budgets from this main task to the evaluation of the remaining training
programs may prove unpalatable.5 This may partially explain why the Conference
Board of Canada reports show no real change in the budgets allocated to evaluation activities by training departments.
However, training evaluation has been unduly mystified. The principles,
techniques, and procedures involved in training evaluation?many of which are
described in this chapter as well as in Chapter 12?are logical, straightforward,
and implementable. Moreover, with the advent of modern information technologies (e.g., Web-based questionnaires and computerized work-performance data)
and new evaluation models and designs, the disruptive impact and costs of data
collection can now be seriously eased. Political Barriers to Training Evaluation Evaluations are conducted when there is pressure from management to do so
(see Training Today 11.1, ?Upper Management?s Role in Training Evaluation?). In
the absence of such pressures, many training managers would rather forgo the
exercise. Clearly, management needs to stress evaluation. Tra i ning To d ay 11.1 Upper Management?s Role in Training Evaluation
A few years ago, there was a rash of serious work accidents in a large transportation company. Some of these
accidents were the direct or indirect result of operator
errors due to the consumption of drugs and alcohol. As
a result, the firm declared a zero-tolerance policy concerning the use of such substances. The policy required
that no employee use substances that may impair effective and safe job performance, whether or not these
substances are legal. The key element of the policy was
that all supervisors were directly and personally responsible for enforcing the policy. Supervisors who failed
to enforce the policy would themselves be subject to
sanctions that could include dismissal.
The training department was directed to develop
and administer a training program to all supervisory
and managerial personnel in the company that was
aimed at teaching the policy and its implementation.
However, the CEO of the company also insisted that
the training program be evaluated to ensure that it was effective. As a result, the training department, which
normally only administered ?smile sheets? to evaluate
their training programs, launched a much more sophisticated training evaluation program that included three
measurement times and the collection of information
on dozens of variables. Clearly, this effort was launched
because the training program had attained high visibility and because top management demanded it. The
training evaluation did uncover some problems with the
training program and suggested a number of changes.
However, none of these changes was ever implemented.
This was because top management showed no interest
in the results of the evaluation study, as these became
available several months after the training program was
The moral of the story is that high-level visibility
can stimulate evaluation actions. However, maintaining
that visibility is important to ensure that the evaluation
results will prove of practical use.
NEL Chapter 11: Training Evaluation But evaluation can be threatening. These studies might conclude that part of
a training program?or even an entire training approach?is not effective. While
this should be considered a valuable finding (as in the Bell Canada case), some
trainers fear that this will reflect poorly on them and/or the training function and
the service they offer. But without evaluations, managers are unable to demonstrate their value to the organization, which may be inherently more risky than
launching an evaluation system that can improve training and its effectiveness.
Other trainers do not evaluate on ethical grounds. They feel that evaluations
should be conducted by external professionals to avoid a perceived or actual
conflict of interest. How can the person doing the training also be the one responsible for evaluating its effectiveness? Conflict of interest, although always a possibility, is unlikely when training managers make use of the established methods
of evaluation?many of which are treated in this chapter. Types of Training Evaluation Most training evaluations focus on the impact of a training program on trainees? perceptions and, to a much lesser degree, behaviours. Perceptions are assessed through
questionnaire measures, while behavioural data may require a combination of techniques including self-reports, observation, and performance data. Evaluations may
be distinguished from one another with respect to the data gathered and analyzed,
and the fundamental purpose for which the evaluation is being conducted. 1. The data collected: Evaluations differ with respect to the type of
?information that is gathered and how that is accomplished. a. The most common training evaluations rely on trainee perceptions at the
conclusion of training (did the participants like it?), while more sophisticated evaluations go further to analyze the extent of trainee learning and
the post-training behaviour of trainees. b. More recently, there has been a growing emphasis on evaluation
studies that also assess the psychological forces that operate during
training programs and that impact outcome measures such as learning
and behaviour change. Research in this area has helped to identify
psychological states (affective, cognitive, and skills-based) that are
important training outcomes because of the influence they have on
learning as well as to improvements in job behaviours.6 c. Finally, information about the work environment to which the trainee
returns can be useful in evaluation.7 For example, measures of
training transfer climate and a learning culture have been developed.8
Understanding the organization?s culture and climate as well as its policies
can strongly affect training choices and effectiveness.9 The degree to
which opportunities exist for on-the-job practice of new skills or the level
of support provided by others to new learners, amongst other things,
have been found to influence training success.10 Training courses that are
strongly aligned with the firm?s strategic vision tend to be more effective. It has also been shown that training programs are more likely to improve
job performance when using the new skill improves the performance of
participants whose remuneration depends on performance.11 NEL 329 330 Managing Performance through Training and Development Formative evaluations
Provide data about various
aspects of a training
?program Summative evaluations
Provide data about the
worthiness or effectiveness
of a training program Descriptive evaluations
Provide information that
describes the trainee once
he or she has completed a
Provide information to
determine whether training
caused the post-training
behaviours 2. The purpose of the evaluation: Evaluations also differ with respect to
their purposes. Worthen and Sanders distinguished between formative
evaluation and summative evaluation.12 a. Formative evaluations are designed to help evaluators assess the value
of the training materials and processes with the key goal of identifying
improvements to the instructional experience (the clarity, complexity,
and relevance of the training contents, how they are presented, and the
training context). Hence, formative evaluation provides data that are of
special interest to training designers and instructors. b. Summative evaluations are designed to provide data about a
training program?s worthiness or effectiveness: Has the training
program resulted in payoffs for the organization? Cost?benefit
analyses (see Chapter 12) are usually summative. Economic indices
are often an integral and important part of these types of evaluations;
consequently, organizational managers show great interest in these
results. A further distinction can be made between descriptive and causal
?evaluations. Descriptive evaluations provide information describing trainees
once they have completed the program. What has the trainee learned in
training? Is the trainee more confident about using the skill? Is it used on the
job? Most evaluation designs have descriptive components. Causal evaluations are used to determine whether the training caused the post-training
behaviours. Was the performance improvement caused by the training program? Causal evaluations require more sophisticated experimental and statistical procedures. Models of Training Evaluation Models of training evaluation specify the information (the variables) that is to
be measured in training evaluations and their interrelationships. The dominant
training evaluation model is Donald Kirkpatrick?s hierarchical model.13 However,
research and practical experience has indicated that Kirkpatrick?s model can be
improved. The COMA model and the Decision-Based Evaluation model are two
recent efforts in that direction; both are discussed below.14 Kirkpatrick?s Hierarchical Model: The Four Levels
of Training Evaluation Kirkpatrick?s hierarchical model is the oldest, best known, and most frequently
used training evaluation model. For example the Conference Board of Canada
data summarized in Table 11.1 are organized using that model. The model identifies four levels of training evaluation criteria. According to this model, a training
program is ?effective? when: L1. Trainees report positive reactions to a training program (Level 1 5 ?reactions). L2. Trainees learn the training material (Level 2 5 learning). L3. Trainees apply what they learn in training on the job (Level 3 5 ?behaviours). L4. Training has a positive effect on organizational outcomes (Level 4 5 results). NEL Chapter 11: Training Evaluation Ta b l e 11.1
The Main Variables Measured in Training Evaluation
Variable Definition How Measured Reactions Trainee perceptions of the
program and/or specific aspects
of the course. Questionnaires, focus
groups, interviews. Learning Trainee acquisition of the program
material. Declarative learning
is knowing the information.
Procedural knowledge is being able
to translate that knowledge into a
behavioural sequence. Multiple choice or
and mastery tests
?(procedural). Behaviour On-the-job behaviour display,
objective performance measures. Self-reports, supervisory reports, direct and
production records. Motivation Trainee desire to learn and/or
transfer skills. Questionnaires. Self-efficacy Trainee confidence in learning
and/or behaviour display on the job. Questionnaires. Perceived and/or
anticipated support The assistance trainees obtain
and/or the assistance trainees
expect. Questionnaires. Organizational
?perceptions How trainees perceive the organization?s culture and climate for
learning and transfer. Standardized
results The impact of training on
?organizational outcomes. Organizational
records. In a more recent articulation, an additional level has been added to the
Kirkpatrick model. Level 5 refers to return on investment (ROI), which is designed
to assess the financial benefit of training to the organization. However, for simplicity?s sake we continue this description using the original four-level version
of the model proposed by Kirkpatrick (ROI and the financial benefit of training
programs is discussed in Chapter 12).
The model states that the four levels are arranged in a hierarchy, such that
each succeeding level provides more important (though more difficult to obtain)
information than the previous one. The model also assumes that all levels are
positively related to one another, each level having a causal effect on the next
level. Hence, positive trainee reactions (L1) cause trainees to learn more (L2),
which in turn leads to the behavioural display of the new skill at work (L3),
which in turn impacts on organizational effectiveness (L4)?the ultimate reason
This question was answered on: Sep 18, 2020
Buy this answer for only: $15
Solution~0001047000.zip (25.37 KB)
This attachment is locked
We have a ready expert answer for this paper which you can use for in-depth understanding, research editing or paraphrasing. You can buy it or order for a fresh, original and plagiarism-free copy from our tutoring website www.aceyourhomework.com (Deadline assured. Flexible pricing. TurnItIn Report provided)
Pay using PayPal (No PayPal account Required) or your credit card . All your purchases are securely protected by .
About this QuestionSTATUS
Sep 18, 2020EXPERT
GET INSTANT HELP/h4>
We have top-notch tutors who can do your essay/homework for you at a reasonable cost and then you can simply use that essay as a template to build your own arguments.
You can also use these solutions:
This we believe is a better way of understanding a problem and makes use of the efficiency of time of the student.
- As a reference for in-depth understanding of the subject.
- As a source of ideas / reasoning for your own research (if properly referenced)
- For editing and paraphrasing (check your institution's definition of plagiarism and recommended paraphrase).
NEW ASSIGNMENT HELP?
Order New Solution. Quick Turnaround
Click on the button below in order to Order for a New, Original and High-Quality Essay Solutions. New orders are original solutions and precise to your writing instruction requirements. Place a New Order using the button below.
WE GUARANTEE, THAT YOUR PAPER WILL BE WRITTEN FROM SCRATCH AND WITHIN YOUR SET DEADLINE.