Evaluating Programs and Human Performance

Chapters in Section III discuss evaluation in instructional design and provide you with three evaluation models: CIPP, Five-Domain, and Kirkpatrick’s Training Evaluation model. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

Merrill's First Principles of Instruction

Many current instructional design models suggest that the most effective learning environments are those that are problem-based and involve the student in four distinct phases of learning:
(1) activation of prior experience - Learning is facilitated when existing knowledge is activated as a foundation for new knowledge.
(2) demonstration of skills - Learning is facilitated when new knowledge is demonstrated to the learner.
(3) application of skills - Learning is facilitated when new knowledge is applied by the learner
(4) integration or these skills into real world activities - Learning is facilitated when learners are engaged in solving real-world problems. Learning is facilitated when new knowledge is integrated into the learner's world.

I actually use Merrill’s first principle of instruction daily.  An example of using the design is when I teach my students about money.  Because my students are in pre-k we really keep the concept of money very simple.  I first begin by asking the students how do they get things that they need.  Eventually we will arrive at the answer that they buy things that they need at the store. Next I introduce the penny to the children.  We talk about the importance of money and a penny is money.  I begin by modeling how to count money by saying the amount of money that I have in cents, and what I can buy.  I allow the students to count the money also.  I have objects on the counter labeled with price tags.  When we finish the lesson, the students can pretend to buy the objects by counting the number of pennies that they have and compare that number to the items price tag.  If the amounts match they can tell me that they can buy the item.  The price tags do not exceed 0.4 cents.  In this lesson I have activated their prior knowledge by asking how do we obtain things that we need.  I model the skill, allow the students to practice the skill, and last we pretend to purchase items at the store.

Bloom's Learning Taxonomy

Benjamin Bloom developed a classification of levels of intellectual behavior important in learning. Bloom identified six levels within the cognitive domain, from the simple recall or recognition of facts, as the lowest level, through increasingly more complex and abstract mental levels, to the highest order which is classified as evaluation.
-Knowledge: arrange, define, duplicate, label, list, memorize, name, order, recognize, relate, recall, repeat, reproduce state.
-Comprehension: classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate,
-Application: apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, write.
-Analysis: analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test.
-Synthesis: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, write.
-Evaluation: appraise, argue, assess, attach, choose compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.

I would use Blooms Taxonomy to help my students acquire skills at all of the various levels.  By doing this I am laying the proper foundation for higher levels of thinking by first assuring mastery of lower-level objectives.  Blooms Taxonomy provides a basis for developing measurement strategies to assess student performance at all levels of learning.  I use a kit called Developing Talkers for pre-k and K students.  I begin a new story by asking the students to name, recall, or repeat the new words in the story.  This would apply to Blooms lower level of instruction.  We would continue throughout the week building upon the knowledge that the students have acquired.  Before the end of the week, the students can predict why something happened  or compare the actions of two characters based on what has happened in the story.

 Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. Should return on investment and management of resources also be considered in evaluations of instructional programs? What other measures should/could be considered?

Other questions that instructional design evaluation should address are:
-Is there a need for this type of program in this context?
-Is the program conceptualized in a way that is should work?
-What factors helped produce the successful results?
-What environmental factors are helping support success, and what factors are getting in the way?

A return on investment and management of resources should be considered in evaluations of instructional program.  To ensure that monetary resources are put to best use, they must be allocated to programs, processes, and projects that yield the greatest return.  For example, if a program is designed to improve effectiveness and it does have that outcome, the assumption might be that the program was successful.  But if the program cost more than the effectiveness gains are worth, then no value has been added to the organization.  A less expensive program may have yielded the same results, possibly yielding a similar return.  Showing how a project or program is connected to the manager’s business goals and objectives can increase support. 

Other measures to be considered are enhanced budgets and building partnerships with key executes.  When a particular function is budgeted, the amount is often in direct proportion to the value that the function adds.  If little or no credible data supports the contribution, the budgets are often trimmed or they do not increase.  Bringing accountability to the return of investment level secures future funding.  Being able to communicate project results builds key partnerships with executes.  They only make an effort to build relationships with projects that help them succeed. 

Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning. Not all problems in learning and/or performance require an instructional one. Many times a non-instructional approach is a more appropriate solution. Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem. Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem?

Each year, our district requires the employees to watch a video on bloodborne pathogens and sexual harassment.  Our district also requires each employee complete a specified number of technology proficiency hours to be in compliance with district policy.  We are emailed the directions on how to complete the assignment, but we do not receive any formal instructional support. Although many of the employees in my district are familiar with technology and can complete the assignment in a timely manner.  We must collaborate with those individuals who are not familiar with technology so that they can complete their task and submit their assignment before the deadline.  I feel that this is an opportunity for informal learning because someone from the technology department could provide an optional training for those individuals who struggle with technology.