Thursday, February 24, 2011

Section 5: Trends and Issues in Various Settings

1. Rapid Prototyping
Example of Rapid Prototyping
After reading our book’s explanation for rapid prototyping and performing my own research, I found several excellent resources that helped me to better understand the concept. One article, “Understanding Rapid Prototying by Analogy: Making Paper Airplanes” by Lloyd Rieber (1994), gave an excellent everyday example and explanation of the purpose and uses for rapid prototyping in instructional design.  I have summarized his article below:

In this analogy, Rieber (1994) asks that you take a 8 ½” X 11” sheet of paper and make an airplane without doing any research. (He also suggests having a child do this project with you.) After the very first attempt of making the plane, it probably will not fly very well. What should you do? Make modifications to the plane that you think will make it fly better. Try flying the plane again. Determine how much difference that modification made. Continue to make modifications and test lots of hypotheses regarding what you think would improve the plane’s design using the same plane. Now take time to reflect on your design hypotheses. Are any of them worth pursuing further? Think about the feedback that every test flight gave you. Now take another sheet of paper and test your design again. Now you really need to consider the criteria for judging the effectiveness of your design.  Distance and accuracy are probably what most people would consider the best criteria for testing the effectiveness of the design.  Clear a “flight path” in the room and have a starting line. Perform several test flights over and over to measure and record the average distance and the percent of accurate flights.  Now take a piece of paper and wad it up. Throw it down the same “flight path” that you used for the paper airplane. Since you were using distance and accuracy as the main objectives, the wad of paper will probably do as well as or even better than the paper airplane. Although both of these criteria are important, there are other important criteria to consider, such as gliding ability, lift, and the amount of force used in each throw.  If you consider the medium paper, could adding elements like paperclips or pennies aid in the design? What about using cardboard instead of regular paper?

This activity demonstrates that design and development are intertwined and interdependent. Continual feedback through testing of early prototypes allows instruction to be enhanced through many design and development “test flights”.  The medium for the design cannot be randomly chosen. It must be appropriate for the activity. Most people start out with the traditional model for a paper airplane. The same holds true for instructional design. The traditional application of Gagne’s events of instruction would be what most designers would begin using. Rapid prototyping encourages the design and development of more creative or opposite designs.  Just as the paper airplane was first tested based on the criteria of distance and accuracy, too often instructional design is tested based on performance data rather that other sources of information, such as students’ motivation to participate.  The best design for a paper airplane allows the plane to glide through the air with only the slightest momentum. The best instructional designs usually work by intrinsically motivating the student to go as far as they can with just only slight prompting.
Rapid Prototyping in Education
Another resource I found, Educational Media and Technology Yearbook: Volume 35, gave me an even better insight for how rapid prototyping could be used in education.  In the chapter “Rapid Prototyping for Designing and Delivering Technology-Based Lesson”, Tshepo Batane (2010) explains that rapid prototyping is an excellent model to help instructors who are new to the use of innovative technologies in designing and developing technology based lessons. With the advancement and expansive list of possible Web 2.0 applications that can be used to facilitate instruction and engage students more effectively in the learning process, educators are challenged to find effective ways to integrate them into their curriculum. By using rapid prototyping as a guide, instructors can design and develop technology enhanced curriculum one lesson at a time.  When one prototype lesson is designed, it can be tested with students. By gathering data through soliciting feedback from students and through observing students as they complete the prototype lesson, instructors can refine and modify the lesson until the desired outcome is achieved.   
I recently have used rapid prototyping on a small scale in creating an Excel lesson using a Wiki. In teaching students about the many other chart types beyond the traditional types of charts (bar, line, and pie) that can be created using Excel, I divided classes into teams to explore and report their findings using a Wiki. I assigned each team a chart type. Each team member had a responsibility. One team member had to find a definition or purpose for using the chart, one had to find an example image of the chart, and both members had to find data that could be used in creating the chart using Excel. Each member input their findings into the Wiki. I have five classes of Business Information Management, and each class period became my “test flight” for the lesson.  During each period, I gathered data by observing and asking students their opinions about the lesson. Each period, I added changes to the design to make it better for the next period. For instance, after the first class period, I found that students could not save the image they found from the Internet and import using the File Import button. I allowed students to edit as guests instead of having to create an account. In doing this, the Import button was not available.  A student discovered that copy and paste would work. My second period class is a little larger. In this class I realized that if too many people were editing and saving at the same time, some edits were lost. I encouraged students to only be in edit mode when they were ready to input their information, to save often, and to refresh the page to make sure that edits were accepted before moving on to their next step. By my last class period, I felt that I had modified and revised the lesson to be effective for use in the future.
BIM Class Wiki

2. Full Spectrum Training
Situational Problem: Pretend you are hired as a consultant for the military. They want to use technology in its training, but electronic access is not always available.
Full-Spectrum Use of Training Products
Recreation of Figure 19-2
In researching full-spectrum training, I found an excellent source titled “Training for Full Spectrum Operations” created by the Army (2008). This publication gave me a better understanding of military training since I have no military experience. It reiterates that the primary mission of our military is to fight and win wars. With the constant problems arising due to conflict in other countries, it is evident that our military is and will be in a constant state of conflict for some time. It is imperative that soldiers be trained to “successfully conduct operations across the spectrum of conflict, anytime, anywhere”. Effective training is the foundation for operational success. Full-spectrum training occurs individually and collectively from the classrooms on base throughout deployment to sustain skills and to adapt to operational environment changes. Training can consist of a variety of technological training tools, including live, virtual and gaming technologies.

 Full-Spectrum Solution: Dempsey and Reiser (2007) explained the Marine Corps’ solution to full-spectrum training using Deployable Learning Resource Centers (DLRC). These mobile training centers consist of a “server, network switch, printer and rugged laptop stations” which can bring the courseware training to the soldiers anywhere they are located. The system can connect to external networks, including a LAN, MAN, or WAN, and via secure wireless systems.  I believe that instead of using laptop stations, PDAs could be utilized for improved accessibility. I found that there is a P37B PDA that is rugged and designed specifically for military use. It incorporates an LCD with touch-screen, keypad, Wifi, internal microphone, one speaker (stereo supported via earphone jack), and Bluetooth communication capability. It contains 256 MB of SDRAM and Flash ROM, and its main power source is a rechargeable smart battery, which has approximately 15% more battery life than normal laptop batteries. Distributed Learning (dL) is also a means that the army is utilizing to meet future training demands. In chapter 15 entitled “Army Training” in a publication entitled How the Army Runs, types of dL are listed to include interactive multimedia instruction, video teleconferencing, web-managed instruction, and simulations, all of which could be integrated in the DLRC. The chapter states that, “dL does not fundamentally change the way the Army trains, it enhances the way it goes about training by using current and emerging technologies for management and delivery of training to the Soldier when and where it is needed”. Together Deployable Learning Resource Centers using PDAs and Distributed Learning technologies could be combined to create an excellent full-spectrum mobile learning environment for successful operational training when access to electronic access is limited.

3. Outline for Staff Development Activity Introducing Step-Up-To-Excellence and GSTE

"If you're doing something you care deeply about and if you believe in it, it's impossible to imagine not trying to make it great." – Jim Collins (2001)

This quote came from the book From Good to Great. Our principal read this book over the summer and made “From Good to Great” our goal for the 2010-2011 school year. He even gave every faculty member a t-shirt with our district’s vision statement on the back and “From Good to Great” on the front. But this alone is not enough to necessitate a great change. Our campus is currently rated “Acceptable”, and the year before we were rated “Low Performing”. Our goal is to step up to the next higher performance level. The elementary campuses have had great success and some  are even rated “Exemplary”. I believe that not only our campus but the school district as a whole should step up to excellence by creating a systematic change.

To introduce the Guidance System for Transforming Education and Step-Up-To-Excellence to our faculty, I would prepare a few things beforehand:
1.     Incorporate the help of our high school’s Leadership Class members to complete a domino chain reaction demonstration project using some of the principles and components from the Guidance System for Transforming Education.
Phase I: The leadership sponsor and I will serve as the Support Team for the project. We will create an online survey to inquire from the faculty what they believe is working, what needs to be improved, and suggestions for improvement for our campus and our district. From the responses we can determine if the changes suggested represent only piecemeal change or systemic change.
Phase II: The Support Team will join with our principal to form the Starter Team. We will meet to find the total number of faculty members we have and explain the project’s goal of enlightening the faculty’s awareness of systematic change and methods to incorporate systematic change. 
Phase III: The Starter Team will expand into the Leadership Team (school class members). The Starter Team will become facilitators, and I will become an advisor and “critical friend”.  We will discuss the fact that dominoes will be used to create the letters SSISD for Sulphur Springs Independent School District. The number of dominoes used will be the exact number that represents the number for our faculty. The Leadership Team will gain the assistance of the community by asking community members to help donate the number of dominoes needed for the demonstration.
Phase IV: The Leadership Team will form and capacitate the building Design Teams who will construct the domino chain reaction using the shape of the letters SSISD. The dominoes will be set up in the area that the Professional Development activity will be held.
2.     Video the team during each phase of the project, and create a final video to show during the professional development activity.
3.     Create a PowerPoint presentation outlining the components of the Guidance System for Transforming Education (GSTE) and Step-Up-To-Excellence (SUTE) for systematic change.

On the day of the professional development, the Leadership Team will be present beside the domino demonstration. Faculty members will be encouraged to view the domino project before the meeting begins. After the meeting begins, the Starter Team will address the faculty to begin the meeting and direct their attention to the video screen to watch as the Leadership Team members start the chain reaction of the dominoes. (A webcam and computer will be connected to a projector for the entire audience to be able to see the demonstration.) Before the dominoes are set in motion, some of the team members will  say, “From Good”. The chain reaction will begin. When it is completed, other team members will say, “To Great!” 

The Starter Team will then conduct the professional development activities using the following outline with a PowerPoint as a visual aid:
1.     Introduction:
a.     Explain that the purpose of the professional development is to introduce systemic change and two methods of creating systemic change.
b.    Explain the difference between piecemeal change and systemic change.
2.     Show the results of the online survey suggestions for improvement. Have the faculty members form groups of four to five members. Distribute several of the suggestions written on paper. Have the teams discuss the suggestions and determine if the suggestions require piecemeal change or systemic change.
a.     Let each team share one suggestion and their interpretation of what type of change it represents.
3.     Share with the faculty examples of school districts who have undergone systemic change and that have been successful using Decatur Township and Chugach as the primary examples.
4.     Introduce the two methods for managing systemic change: Guidance System for Transforming Education (GSTE) and Step-Up-To-Excellence (SUTE).
           a.     Outline their components and similarities through using visual graphics and charts.

b.    Explain that the domino demonstration was created using several of the basic procedures used in GSTE. Show the video for the phases of the domino project.
5.     Have a member of the Leadership team read the following excerpt from the article “Step-Up-To-Excellence: A Change Navigation Protocol for Transforming School Systems” by Francis M. Duffy (2006).

Microsoft Office Clipart Image

Rolling across America is a long train called “The School Improvement Express.” The triple societal engines of standards, assessment, and accountability are pulling it. The lead engine goes by the name “The No Child Left Behind Engine That Could.” The rolling stock is composed of school systems and a myriad of contemporary school improvement models, processes, and desirable outcomes. The train has once again come to a stop at a broad and deep abyss that goes by the name “The Canyon of Systemic School Improvement.” On the far side of the abyss lies the “Land of High Performance.” The riders on the train want to go there. In fact, they have wanted to go there for years but have failed to make the crossing, and so they keep returning here to the edge of the abyss to stare across with longing in their hearts wondering how they will ever traverse it.
Standing at the edge of this great abyss, some educators see a threat while others see an opportunity. Some see an impossible crossing, while others see just another puzzle to be solved. Meanwhile, the pressure in the three great “engines” for setting standards, assessing student learning, and holding educators accountable for results continues to build and shows no sign of dissipating. The “engineers” have their hands on the brakes but they can feel the pressure of the engine trying to edge the train forward, which feels like having one foot on the brake of a car while stepping on the gas with the other foot.
Even though the train has rolled across a lot of ground and although its passengers have done good things along the way, there they stand one more time looking out over the abyss wondering how in the world they will get to the other side. Some of those standing at the edge say, “Impossible can’t be done.” Others say, “We’ve been here before and failed then.” Still others stand there and theorize about the complexity of crossing such a canyon. “It’s so hard to define the boundaries of the canyon. Just what is a system, what does it mean, is it this or is it that? We need this, this, this, and that or we’ll never cross,” they suggest, but then they take no action to do what is needed. Still others, looking backward at the long train say, “What’s behind us is the future. What we have done in the past is what we should continue to do.”
6.   Conclude with sharing with the faculty that the number of dominoes used in the project represented the number of faculty employed by SSISD. To create a systemic change, it will take each member of the faculty and the resources and support from the community for the school district as a whole to move “From Good to Great”.
7.     Pass out a survey to gain thoughts and feelings in regards to the professional development for faculty to take with them to complete and submit to the campus principal.

4. Faculty Development University Offices
1. Harvard University
What are the different names used for faculty development?
Faculty Development and Diversity

What division is it under?
Office of the Senior Vice Provost

What services does it offer?
Diversity training, development and mentoring resources, scholarship and research resources, teaching and advising resources, media and information technology services, faculty real estate services, university health services, and work-life benefits are just a few of the services offered.

How often are programs given and what specifically are they?
Numerous program events are held throughout each semester for each division of the office. The divisions listed were: Diversity, Development and Mentorying, Scholarship and Research, Appointment Polices and Practices, Teaching and Advising, Work-Life, Benefits, and Perks, and Preparing for an Academic Career.  A few of the specific programs that were listed were: Women’s Network, school specific mentoring programs, grant programs for enabling research, New England Higher Education Recruitment Consortium, Harvard Adoption Assistance Plan, Employee Assistance Program, as well as numerous events and seminars relating to particular service areas offered throughout each semester.
2.  Ohio State University

What are the different names used for faculty development?
University Center of the Advancement of Teaching (formerly called Faculty & TA Development)

What division is it under?
Office of Adademic Affairs

What services does it offer?
Professional consultants, learning communities, online teaching resources, blog posts, motivational award presentations, and a faculty library.

How often are programs given and what specifically are they?
The current workshops that are being offered are: A Curriculum for the 21st Century, Feb. 25; How Learning Works with Author Michele DiPietro, March 7; Teaching Two Courses in One, Feb. 25; and Roundtable Series on May Session, March 3. A Spring Course Design Institute will be held March 21-25, and Spring Quarter Design Institute will be held April 6-May 4. An online At-Risk Simulation Training can be taken throughout each semester. Two university partner events were listed: Emerging Trends in Higher Education: The Change Face of Academia will be offered Feb. 21, and the OSU Colloquium on Growing International Perspectives is a yearlong series of enrichment opportunities.

3. Michigan State University

What are the different names used for faculty development?
Office of Faculty and Organizational Development (F&OD)

What division is it under?
Office of Provost

What services does it offer?
Faculty learning communities, faculty social gatherings, faculty development network, Faculty Emeriti Association, instructional consultations and related services, instructional briefings, workshop and seminars opportunities, leadership resources, faculty mentoring resources, online instructional resources, SoTL information and resources, and orientations.

How often are programs given and what specifically are they?
Programs are offered throughout the year. A few of the programs that were listed were: Lilly Teaching Fellow Program, Walter and Pauline Adams Academy for Instructional Excellence and Innovation, Scholarship of Teaching and Learning Symposium, programs for academic specialists, Spring Institute on College Teaching and Learning, CIC Academic Leadership Program, Early Career Faculty Programs, and the CIC Department Executive Officer Program.

Thursday, February 17, 2011

Section 4: Human Performance Technology

1. Human Performance Improvement
In teaching Career and Technical Education courses, I have found that throughout the years the one major performance problem in my area is and has been student accountability for their own performance. To analyze this performance problem, I used the HTP model of the International Society for Performance Improvement for performance analysis.
Performance Analysis:

My classroom goal is for each student to be accountable for their own performance in order to be successful in class. The desired workplace performance is for students to be accountable by keeping track of assignments and due dates, completing all assignments on time, and coming in for tutorials when assignments are missed due to absences. The gap in the desired workplace performance and the actual state of workplace performance is that only about 60% of students in each class actually meet these performance goals. The other 40% do not keep adequate track of assignments and due dates, turn in work on time half of the time, and usually do not come in for tutorials when they have missed assignments. About 20-25% of these students are failing or on the brink of failing because of these performance issues.
Cause Analysis:
In evaluating the causes for this low performance issue, I have found that lack of information, feedback, tools, resources, expectations, and consequences have been the major contributors.
Lack of Information and Feedback: In past years, students would not ask for missed work when returning to class. I tried to rectify this problem by creating an assignments website several years ago. The problem is that most students still would not actually get on the website to get their missed assignments. Students also do not adequately keep up with their graded work and really have no clue about their grade average until the three week progress report time or the end of the six weeks.
Lack of Resources and Tools: I have found that most students do not keep track of assignments and due dates. Last year, I started requiring students to keep a class folder which contained the course outline, assignments log, notes, and graded work. Most students have utilized the folder for class notes, but still are not keeping their assignment log updated regularly.
Lack of Expectations: I feel that student expectations for their own performance in technology based classes are not always very high due to their level of experience with technology. That has improved over the years, but I still find that there are many students whose expectations are too low.
Intervention Selection and Design:
After analyzing the causes, I have found several non instructional ways to try to improve this low performance issue. They are as follows:

·      Electronic Performance Support: To get students to utilize the class assignments website, I have started putting the focus activity on the website instead of on the board. At the beginning of each class period, students must log onto the class assignments website to complete their focus activity and see their assignments for the day. Each week’s focus activities and assignments are posted daily and remain on the website. I have also added my PowerPoint lesson presentations to aid students who missed class. I also plan on including short how-to videos by either using YouTube or creating my own using Jing to demonstrate concepts taught. Students can use this as a review or as a lesson if they missed class. I have found that requiring students to access the site daily, has improved their own use of the site to get missed assignments when they have been absent.

BIM Class Web Site Home Page

·      Job Aids: Students are required to maintain a class folder which includes an assignment log. To motivate students to utilize the assignment log in keeping track of assignments, I added a column for due date and a check mark column for students to check off assignments as they had been turned in. I also added a column for students to track the grade of each assignment to have a better idea of their grade in the class. This year I have started taking a major grade (instead of just a daily grade) each six weeks on the folder. In order to get full credit, the assignment log must be up to date and complete.  This becomes a reward in a sense, because it can actually be an easy way for the student to improve their test average if they will keep their folder maintained adequately (including their assignments log).

·      Electronic Performance Support: Last year, our school implemented a system for parents to be able to access their child’s grades online. Parents have to come to the office to request their log-in and password to access the online grade book. Some parents have taken advantage of this and have even given their child the log-in and password so they can check their own grades. I have talked to our administrators about the possibility of allowing all students the ability to access their own grades online as well as their parents. I know that the students who have access to their grades have become more accountable for their own performance. I really believe that if every student had access, more students would take more initiative in becoming more accountable for the completion of their assignments. This would force them to see the effects of their performance on a daily or weekly basis rather than just at the three week or six weeks mark; which can sometimes be too late.
2. Electronic Performance Support Systems (EPSS)

In Chapter 15 Reiser and Dempsey (2007) explain the ideas and definitions of an electronic performance support system since its materialization in the late 1980’s. In researching EPSS, I found several other definitions that were beneficial in reviewing the purpose of an EPSS.
  • A former Nortel Networks executive, William Bezanson (2002) provided this definition: “A performance support system provides just-in-time, just enough training, information, tools, and help for users of a product or work environment, to enable optimum performance by those users when and where needed, thereby also enhancing the performance of the overall business.”
  • In the book Electronic Performance Support Systems: How and Why to Remake the Workplace Through the Strategic Application of Technology, Gloria Gerry (1991) provided this definition: “A system that provides electronic task guidance and support to the user at the moment of need. EPSS can provide application help, reference information, guided instructions and/or tutorials, subject matter expert advice and hints on how to perform a task more efficiently. An EPSS can combine various technologies to present the desired information. The information can be in the form of text, graphical displays, sound, and video presentations.”
  • In the book Fundamentals of Performance Technology, VanTiem, Moseley, and Dessinger (2004), had the following definition: “An electronic performance support system is a highly sophisticated technological job aid, with the following advantages: Access to large databases of information; designed to coach the user through questioning, assessing answers, evaluating responses and to offer recommendations, and user-friendly.”

My EPSS Definition Preference:
With my interest in accounting, I found that I prefer William Bezanson’s definition best. JIT (Just-In-Time) is a familiar accounting acronym that refers to a management philosophy that strives to eliminate waste by providing the right amount of inventory at the right time. I feel that an EPSS should provide an employee or student with the right information, tools, training, and help at the point that it is needed to enable them to improve their performance. I also agree with Laffey (1995) in that the EPSS should be a dynamic, ongoing synergistic input from users to keep the information that is provided current and relevant to the problems or situations at hand.

 EPSS: Past, Present, and Future:

Microsoft Office Clipart Image
Even though electronic performance support systems have been implemented by several companies, it is still a fairly new, emerging field. Gery (2002) states that job aids such as job training, peer and advisory support, and reference materials are still the predominant methods used to enhance human performance in the workplace. One of the main drawbacks of setting up an EPSS is the fear of risk associated with developing and maintaining this system that could be very expensive. Most companies that have begun using an EPSS, have not created an evaluation system to report the return on investment. Without knowing the cost effectiveness, companies are still leery of incorporating such an expensive system. Another deterring factor is simply resistance to change. With any new system, alterations must be made in the work environment to accommodate and encourage employees to use the system. This becomes a challenge for managers.  
With society relying more and more on the use of technology, the realization that electronic performance support systems are the best way to train and support human performance in the workplace will not only become fact, but the norm in the future for businesses. Already, the companies that have begun to utilize the EPSS have reported savings (Hewlett-Packard) and increased sales (Payless Shoe-Source) due to the implementation of the system. I believe that as today’s digital natives become tomorrow’s workplace managers, technology will inundate the workplace. As the EPSS evolves with new technologies, the use of an EPSS will become more efficient and effective in supporting the performance of groups and more widely recognized as an integral part of improving performance in the workplace.
3. Blended Learning Approach and Knowledge Management
The Problem
Changes in Career and Technical Education that were initiated this year have been astronomical. New TEKS, new courses, new clusters, and new ideas have some teachers scrambling to keep up. Many of the new course names are just new faces for the old courses that were offered previously, however there are a great number of courses that are brand new with no real textbooks or curriculums currently in place. Teachers who have had to start teaching these new courses this year have had to face starting the year with no textbooks and no real curriculum.
The Solution
The Career and Technical Education unit of TEA answered teachers’ cries for help in a sense by using blended learning and knowledge management to aid CTE teachers. Most CTE teachers were required to go to workshop this past summer for training in their particular cluster. In the workshop, we were given a CTE manual with the new TEKS and resources for our cluster’s courses. We were also introduced to the CTE website that was created on the University of North Texas educational site. The site provides access to each cluster with TEKS, resources, and curriculum that teachers have created. Not all courses have curriculum yet, but we have been told that as soon as it is created, it will be posted to this site. The site also offers CTE teachers a CTE Community Forum to ask questions and share ideas and lessons as well as a blog for teachers to collaborate. The CTE website has become in a sense the knowledge management system for CTE teachers. It provides access and information to all CTE teachers, collaboration through an online community forum and blog, and codification through links that contain specific information for each cluster, tools for instruction, and training programs offered through workshops at our summer conference.

Knowledge Management Components

4. My Informal Learning Experiences
In attaining my teacher certification, I was trained through formal learning in my undergraduate classes. These classes helped me learn my content and how to devise a lesson plan, but that was really the extent of my learning. I really learned how to teach through informal learning experiences. These learning experiences came through my teaching internship, mentor teachers, watching other teachers teach, talking with other teachers at conferences and workshops, gaining ideas online through looking at strategies and methods other teachers were using, and especially through trial and error. Each experience gave me insight into the art of teaching. I was engaged in learning because I wanted to become the best teacher that I could possibly be. I am still learning informally. My role was and still is that of a student who is willing and excited about learning. There were never any real instructors in any of my informal learning experiences; just individuals who had expertise in the field that I came into contact with throughout the course of my teaching career.

UIL Accounting
Although I have been a teacher for many years, I have only in the last eight years had the opportunity to be a UIL Accounting coach. When I started my current teaching position, I was told that I would be responsible for coaching a UIL event. I was given some old accounting tests without any instruction or training on what to do next. I started by taking the tests myself. This taught me what content to prepare my students for, but it didn’t teach me how to build a winning team. This I learned through informal learning experiences at UIL meets. I began talking with other UIL Accounting coaches at the meets. We shared ideas, challenges, and strategies for getting students prepared for the event. After every meet, I learned a little more about becoming a better coach. I remember meeting one particular UIL coach who has successfully had a state winning team for many years. She has shared with me her “secrets”, and she became my mentor. We communicated at meets and through email. Last year, my team won 2nd place in State. I contribute much of the success of my team to the tips and tricks that this accounting coach has shared with me through informal instruction.

Wednesday, February 9, 2011

Section 3: Evaluating, Implementing and Managing Instructional Programs and Projects

1. Two Models for Evaluation
Flashlight Triad Model
Virginia Tech uses the Flashlight Triad Model of evaluation for technology oriented evaluations. The Flashlight Model comprises five steps:

1. Overview and Confronting the Blog: This step involves brainstorming to create a comprehensive list of elements that “feed into and flow” from a lesson or course. The examples that were given were: faculty development and preparation, student prerequisites, budget, assistance with course materials development, and student attitudes and satisfaction.

2. From Blob to Issue: Consider the purpose of the evaluation. Two questions should be answered:
1) What are the three most crucial things you fear might happen as a result of using the technology?
2) What are the three most crucial things you hope will occur?
One or two top issues should be selected from the answers and become the focus of the evaluation.

3. From Issue to Triad: For each issue that was selected, several triads are created. “A triad consists of the type of technology you might employ (T), a specific activity that the technology enables (A), and outcomes expected from that type of activity (O).” After several triads are created, the ones that will be focused on are selected.

Sample Triad

4. From Triad to Data: For each triad that was selected, questions are generated to gather data through surveys, focus groups, interviews, etc. At least five types of questions can be generated: 1) Technology (related to technical infrastructure, training, function, etc.), 2) Interaction of Technology and Activity (related to technical systems and their ability to support the specific activity), 3) Activity (related to the nature of the student tasks), 4) Interaction of Activity and Outcomes (related to the capability for technology-based activities to generate predicted outcomes), and 5) Outcomes (related to student learning, satisfaction, and student suggestions)

5. From Data to Next Steps: Based on the evaluation data, decisions will be made about modification and improvement for the course/lesson.

My Implementation of the Flashlight Triad Model:  Since technology implementation is such an important part of instruction, this is an excellent tool to evaluate the use of those technologies in the learning process. I have just started to use blogging in my classroom as a means for students to express their ideas and opinions on specific topics. In my evaluation of the use of blogging using this model, I would first focus on student attitudes and satisfaction. For the issues, I would select student motivation and active learning. The triad I would use is as follows:

Expression of ideas, thoughts, and reflection to improve writing; sharing ideas to support different perspectives
Students demonstrate more ownership in their writing; are more aware of checking their writing for grammatical and spelling errors before posting; gain different perspectives of ideas through reading other blogs.

From this triad, I would generate survey questions to ask students through an online survey. Based on the data that is collected, I can make a decision as to whether blogging is being beneficial for students in my classroom.
Dessinger-Moseley Full Scope Evaluation Model
The Dessinger-Moseley (1998) model of evaluation integrates evaluation throughout the ISD model. It includes six integrated evaluation activities:

Recreation of Figure 1-1 from Confirmative Evaluation: Practical Strategies for
Valuing Continuous Improvement by Dessinger and Moseley (2004)
  • Formative Evaluation focuses on the need for training, instructional design and development processes and products, reaction during and after training, and accomplishment during training.
  • Summative Evaluation focuses on the immediate results of program implementation: reaction during and after training, accomplishment during training, and the self-reported expectation that new knowledge, skills, and attitudes with transfer to the job and affect workplace performance.
  • Confirmative Evaluation focuses on the program’s continuing impact and value, as well as the long-term transfer of knowledge and skills to the job and workplace.
  • Meta Evaluation focuses on all the evaluation types—their inputs, outputs, outcomes, and focus.
Source: Dessinger, J., & Moseley, J. (2004). Confirmative Evaluation: Practical Strategies for Valuing Continuous Improvement. Pfeiffer & Co.
My Implementation of the Dessinger-Moseley Full Scope Evaluation Model:
The use of wikis has been something that I have experimented with using in the classroom as a collaboration tool.
  • For the formative evaluation, I wanted to create a field test evaluation that would allow me to try using a wiki on a small scale. I started with a small project where students had to create an online survey using They then had to post their survey link in a wiki for other students to easily find and take the surveys. I wanted to see if the students could manage the wiki, see its relevance in use, and find areas to improve its use in the future.
  • For the summative evaluation, I asked student groups what they liked and disliked about using the wiki during the project. I had made the wiki editable only by members. So each student had to create a log-in in order to edit. Many students forgot their passwords (which seems to be the case even after we discuss the creation of log-ins and passwords), which took up time to reset passwords. I will have to have a better system in place for students to create log-ins and passwords. I am about to implement the use of wikis once again for students to create Excel charts to share with the whole class. Students will create an Excel chart comparing educational, income, and other statistics of Sulphur Springs, TX with two other cities of their choice. They will post their Excel charts to their class wiki to share and discuss findings. At this point, I will continue summative evaluations having students answer an online survey to find their reaction to the use of the wiki. I will be able to evaluate their accomplishment of using the wiki when all students successfully post and respond to the charts.
  • For the confirmative evaluation, I will create projects that will give students a choice of collaboration methods (with the wiki being one of the choices) to see if students will choose to use this as a means of group collaboration.
  • For the meta evaluation, I can use data collected from all evaluations made during the course of using class wikis to determine whether wikis should be a tool that I encourage students to use in the future.

2. Technological Innovation — Perceived Attributes

Microsoft Office Image

A technological innovation that was introduced last semester at our school was an online tardy tracker. The tardy tracker was developed by our technology specialist as a better way of keeping track of student tardies. We have used an online attendance method for several years, but teachers were still using a paper based method of tracking tardies. Teachers had to write down students’ names when they were tardy and physically take the tardy sheet to the office at the end of each day. Secretaries in the office had to then manually enter the tardies for the day into a database to be able to generate reports for the administrator that handles student tardies.
According to the concept of perceived attributes (Rogers, 2003), there are five key attributes that potential adopters will base their feelings about an innovation. They are:
Relative Advantage: In the case of the new online tardy tracking system, the immediate relative advantage was less paperwork for the teacher. Teachers were very responsive to using the new tardy system because of this attribute of the innovation.
Compatibility:  The new tardy tracking system is easy to access online through our local area network. Teachers are able to open the tardy system and our attendance system to easily work between in recording student attendance and tardies each class period.
Complexity:  The online tardy system is very simple and easy to use. It requires the same information that was needed in paper form, but is much easier to complete since a database with student information is connected to the system. The date for that day is automatically selected when the system is opened. As a teacher begins typing the student’s last name, a list of possible students appears. After selecting the correct student, the student’s ID number and grade are added to the form automatically. The teacher then selects the class period, enters the time the student arrived in class, and hits submit.
Trialability: We were not really given a trial period before the online tardy system was adopted. However, teachers had been requesting a way to enter tardies online for some time. With this request, our technology specialist created this program and tested it thoroughly before it was implemented.  
Observability: The observed benefits was not only less paperwork for teachers, but reduced work for the secretaries who had previously had to manually enter all paper tardy slips turned in by the teachers each day. The new system is accessible to everyone on the network. Reports with accumulated student tardies can be automatically generated for each student per class period and per day for the administrator to issue punishments for excessive tardies. Teachers can also look back at their own tardy report at any time to see the tardies that were entered for the day.

3. Situational Leadership

As a teacher, I completely understand resources being scarce. With budget cuts, resources are becoming slimmer and slimmer for educators.  I actually see administrators turning more to “in-house” training and staff developments to cut costs. If I were assigned the task of developing a series of professional development sessions focusing on technology use in the classroom for teachers, I would facilitate the project using the Situational Leadership model. My basic personality behavior is that of a follower. I really do not like to lead, but I love working with a team and have good interpersonal skills. From what I understand, the Situational Leadership model is based more on a side-by-side team approach to management rather than a top-down approach. To be effective, the manager has the task of achieving organizational goals by adapting the leadership style and resources based on the analysis of specific situations.  In developing the professional development sessions, I would proceed as follows:
Build the Team: First I would ask other interested colleagues to join the team. Based on the type of training, I would try to pull in those people with experience in those areas.  My team for the professional development sessions based on technology use in the classroom would consist of:

Project Team Communication Pattern
Created using PowerPoint Diagrams
 Phase 1:  After talking with each team member, I would need to decide the experience and confidence level of the team as a whole.  I feel that most of my team members are experienced and have the confidence because of their experiences to be able to work efficiently towards meeting the goals. If however in the initial planning meeting I sense that there is some uncertainty, I must assume the leadership style of being a directive and thorough manager without being too demanding.

Phase 2:  As we continue to meet to plan, organize, and develop the professional development sessions, the team should build confidence and begin to “buy-in” to the goals of the team.  In this phase, I will assume a more coaching leadership style by explaining, advising, and supporting decisions made by the team.
Phase 3:  In this phase, focusing on results is critical. My style of leadership must move to being that of a supporting motivator—encouraging the team and rewarding them for effort and production.  I would ask the client for team members to have additional time during the school day to work together as well as provide lunch for team members.
Phase 4: The team at this point should feel comfortable in working together. As the manager, I should now be able to step back and take the role of a monitor. The decision making and problem solving aspects of finalizing the project can now be proficiently completed by the team members.