ArticleSummary Maddrell - Upload a Document to Scribd

Article Reviews Running head: ARTICLE REVIEWS 1 Article Reviews Jennifer Maddrell Old Dominion University IDT 848 Evaluation Study Abstracts Dr. Adcock October 22, 2008 Article Reviews Reference Burton, J., & Aversa, F. (1979). Formative evaluation information from scripts, scratch tracks, and rough cuts: A comparison. Educational Communication and Technology Journal, 27(3), 191-194. Summary 2 Given the significant time and expense outlay involved with television course production, Burton and Aversa (1979) sought to understand how early in the televised course development process the learner content review should occur. While prior research on formative evaluation suggested that review should begin when the instructional product is still “fluid”, Burton and Aversa questioned how useful learner script review is at the early production stage and predicted early stage scripts would be too incomplete for the learner to discern the instructional message. Design Sample and Studied Variables 82 adult learner reviewers were selected from a group of potential students who fit the learner profile for the course. The students were randomly assigned to one of three treatment groups, including those who reviewed (a) the written script alone, (b) the written script and an audio scratch track, and (c) the first rough cut version of the video. The three groups were compared based on both learning outcome, as well as on learner responses to the course material categorized into three areas, including the overall appeal of the program, the learner’s affective responses to the subject matter, and the design of the structural elements of the program. Treatment Members of all three groups provided basic demographic information, including age, education level, subject background, and received the same introduction to the course entitled, Japan II: The Changing Tradition. Those in the script group read through the written script once. Article Reviews The learners in the scratch track listened to an audio recording of a single voice reading all narrations while following along with the written script. The rough cut group viewed the initial version of the video without visual effects or music. After reviewing the materials, the learners completed a 5-point Likert scale opinion questionnaire about the instructional product followed by a short answer test to assess their understanding of the presented material. Analysis and Results 3 The collected demographic information confirmed the groups did not differ significantly. Further, the differences in the mean scores across the three groups for the short answer test were not statistically significant. However, in terms of learner responses to the questionnaire, the mean differences across the three groups were statistically significant. For each learner response measures, the mean scores for the scratch test group were greater than for script group which were greater than for the rough cut group. Critical Summary This study provides support for the use of early scripts and audio scratch tests in high production courses. However, as was most striking to the researchers, the relatively harsh response to the rough cut video appears to contradict prior research. As a possible explanation for the poor learner responses in this study, the rough cut used in this study may have been too rough and too far from the finished representations to allow a viable comparison. Application This study offers support for early evaluation, especially when production time and expense is high and late term revisions would be costly. As this study suggests, learner reviewers are able to discern the instructional message in very early drafts within the development process. Reference Article Reviews Jones, T., & Richey, R. (2000). Rapid prototyping methodology in action: A developmental study. Educational Technology Research and Development, 48(2), 63-80. Summary Citing mixed findings in research literature, Jones and Richey (2000) questioned the 4 effect of rapid prototyping (RP) on instructional design development cycle time, product quality, and customer satisfaction. The purpose of their qualitative study was to gain a better understanding of how RP methods are applied, what the customer’s role is within the RP process, how (if any) concurrent completion of design tasks occur, what (if any) instructional systems process and quality enhancements result, and how customer satisfaction is impacted. Design Qualitative Details and Methodology This qualitative study was conducted at an instructional design firm with 14 employees. Several years prior to the study, the firm adopted a RP process for their custom designs that focused on three milestones, including (a) kickoff involving a customer meeting where roles, responsibilities, and schedules are determined, (b) design freeze when full agreement on product format, content, and instructional strategies is reached between the designers and customer and rapid development occurs, and (c) pilot ready when the product ready for learner pilot testing. The activities of two senior instructional designers on two separate projects, as well as one client contact per project, were examined. Both projects were one-day instructor led classes, but were delivered using different media. Data collection included reviews of designer task logs and other project data, as well as personal interviews. Data analysis focused on the nature of the RP process, attitudes about RP and the product, cycle time, and overall customer involvement. Analysis and Results While the projects were completed fairly linearly, especially in the final stages, the data Article Reviews analysis revealed that the 14 key tasks prescribed by the firm’s RP model were performed for each project with concurrent processing occurring in the completion of 10 out of the 14 key tasks. Work time varied between the two projects at the task level, but total work time was similar at 79.25 hours for Project 1 and 74.0 hours for Project 2. Both the designers and customers perceived reduced cycle times as compared with traditional instructional design. The researchers noted the relatively high degree of customer interaction in both projects. Customers were actively involved in (a) analyzing the training needs, (b) providing input, feedback and approval of content, learning activities, and the prototype, and (c) participating in the pilot. Given that learner achievement data was not collected, the researchers focused on satisfaction (of the designer and customer) and usability to the end customer. Satisfaction with 5 the project was high for both the customer and designers. Further, both projects were put into use immediately after delivery to the customer and were in use one year after which the researches deemed to be a measure of product quality from an external consultant’s perspective. Critical Summary Given the results of this limited qualitative review, the chosen RP design process resulted in acceptable production cycle time, a usable instructional product, and a satisfied customer; all good outcomes for an instructional design consultant. However, without a measure of learning outcome, effectiveness was not fully evaluated. Further, it is possible that the relatively high degree of stakeholder involvement during the entire instructional design process, not simply RP process changes, was the key factor in the project’s success. Article Reviews Application The most significant outcome of this study is the reinforcement of the need for frequent communication and buy-in from the stakeholder. As suggested in this study, a project will run 6 efficiently and result in a more satisfactory outcome if there is open communication, stakeholder input, and agreement on key decisions throughout the design process. Reference Brown, K. G. (2005). An Examination of the Structure and Nomological Network of Trainee Reactions: A Closer Look at 'Smile Sheets'. Journal of Applied Psychology, 90(5), 9911001. Summary Brown (2005) studied training reaction from an affect-based theoretical framework examining affect as a subjective state that is either positive or negative. Based on cited prior theory and research, Brown predicted: (a) affective training experiences create an overall evaluation of satisfaction which in turn influences specific reactions to the training; (b) content interest is positively related to reactions; (c) learner personality traits and orientations are related to reactions; (d) media aesthetic appeal influence reactions, and (e) reactions and learning are related. Design Sample and Studied Variables Two studies were held to examine these predictions. In the first study, 178 undergraduate business students and 101 graduate business students volunteered to answer a survey regarding a pre-recorded videotaped lecture with 64% and 58% response rates, respectively. The second study included 97 undergraduate business students who were randomly assigned to one of three groups who viewed the same lecture presented via three different technologies, including (a) a Article Reviews 7 computer-delivered presentation, (b) audio and print, and (c) video, audio, and print. Between the two studies, a host of measures were evaluated and compared, including learners’ computer experience, personality traits (extraversion, neuroticism, and openness), mastery and performance goal orientation, content interest, technology satisfaction, perception of relevance, enjoyment, overall satisfaction, engagement, and learning. Treatment In the first study, learners viewed a videotaped lecture and took a survey assessing content interest and reaction. In the second study, participants viewed the identical instruction, but via the noted technologies. At the lecture’s midpoint, a brief engagement survey was conducted. After the lecture, learners completed a reaction survey, an intention questionnaire related to future use of the technology, and a 25 item multiple choice knowledge test. Analysis and Results From the first study, a factor analysis suggested that (a) reactions are related, (b) overall satisfaction is a good predictor of other reaction measures, and (c) attitude (interest) and disposition (master goal orientation) predict reactions. Within the second study, a multivariate analysis of covariance with ACT score as the covariate showed statistically significant differences in reactions across delivery technologies with audio conditions having statistically lower satisfaction measures. In addition, regression analysis suggests reactions can predict engagement, intentions, and learning. Critical Summary While Brown’s paper presents an intriguing affect-based theoretical framework for the study of trainee reaction, he acknowledged the conflict between his research findings and prior research, especially within the suggested relationship between reaction and learning. It is Article Reviews 8 troubling that Brown is satisfied that his findings from these very short and limited single session interventions can challenge this extensive body of prior research. Application Brown’s affect-based theoretical framework is an intriguing basis for future research on trainee reactions. If his findings are correct that an overall satisfaction measure is a predictor of other reaction measures and that reaction can predict engagement, intentions, and learning, reaction surveys could be streamline to just a few items addressing overall satisfaction with the experience. Reference Kandaswamy, S., Stolovitch, H., & Thiagarajan, S. (1976). Learner verification and revision: An experimental comparison of two methods. Audio-visual Communication Review, 24(3), 316-328. Summary Kandaswamy, Stolovitch, and Thiagarajan (1976) report on one in a series of studies on learner verification and revision (LVR). Noting increased advocacy and use of learner feedback during formative evaluation, the researchers assess the generalizability of prior studies which support LVR and compare the effectiveness of tutorial LVR and group-based LVR methods. Design Sample and Studied Variables 140 eighth grade girls were randomly selected from two different schools in India. 60 girls were randomly assigned to the LVR group while the remaining 80 were included in a final summative comparison. Four teachers from the schools were randomly selected as evaluator / revisers. The studied LVR methods included (a) tutorial LVR in which the evaluator / reviser probes and monitors the learner’s nonverbal and verbal feedback while the learner completes the Article Reviews 9 material and (b) group based LVR in which the evaluator / reviser analyzes patterns of errors and predicts causes after the learner completes the material. The 60 assigned to the LVR group were stratified based on prior math achievement and one top, average, and poor student was randomly assigned to each of the four evaluator / revisers for the tutorial LVR treatment. The remaining 48 were randomly assigned to the four evaluator / revisers for the group-based LVR treatment. Treatment For the 48 learners in the group-based LVR treatment, a proctor administered a printed self-study instruction booklet which contained a pretest, instruction, and posttest. Upon completion, each of the two group-based evaluator / revisers took 12 booklets each and made independent group-based revisions. The other two evaluator / revisers conducted separate tutorial LVR sessions with the 3 students assigned to them and made independent revisions from the evaluation. In a second phase, the review and revision process was reversed. These two phases resulted in a total of eight revised versions and allowed an evaluation of the order in which the methods were used. The 80 students in summative comparison group were randomly assigned to one of eight groups and completed one of the revised pretest, instruction, and posttest materials. Analysis and Results A one-way analysis of variance of the posttests of the original and eight revised versions show statistically significant differences between each of the eight revisions and the original which supports the research prediction that learner review and revision improves the instructional material. However, there was no significant difference in outcomes between the tutorial and group methods of LVR or from the order in which the methods were used. Yet, the revisions by different evaluators and revisers did have different degrees of effectiveness supporting the prediction that not all revisions by evaluators are of equal value. Article Reviews Critical Summary This study is significant in that it suggests support for conducting learner based review. 10 Further, the findings suggest that the evaluation method and order of use of different methods are less important than the person chosen as the evaluator and reviser. Application These findings suggest that not all evaluators are equally effective at evaluation and revision. Therefore, an evaluator quality control process should be contemplated which includes a cross-comparison of revision recommendations across multiple reviewers and evaluations of the evaluators. Reference Medley-Mark, V., & Weston, C. B. (1988). A comparison of student feedback obtained from three methods of formative evaluation of instructional materials. Instructional Science, 17(1), 3-27. Summary Twelve years after the 1976 study by Kandaswamy, Stolovitch, and Thigarajan discussed above, Medley-Mark and Weston (1988) sought to quantitatively and qualitatively compare the data collected from one-to-one and small groups during learner verification and review (LVR). Given the lack of research on the characteristics of the data collected, the stated purpose of the study was to examine the identified student problems across various LVR conditions. Article Reviews Design Mixed Method Details and Methodology 24 students volunteered to participate in the evaluation of two modules in their undergraduate educational media course. The volunteers were stratified based on grade point average. From this, six students were selected based on availability and assigned into one of 11 three groups, including a one-to-one group with the student with the highest grade point average (GPA) (1-1), a small group of two students with comparable mid-range GPA’s (1-2), and a small group of three students from each of the three GPA levels (1-3). All six students participated in two sessions. In the first, the unmodified version of the prior year’s one hour lecture was given by the instructor followed by Assignment 1 which the learners completed based on their assigned condition, as discussed below. In the second session, learners viewed a video-taped lecture followed by Assignment 2, again completed based on their assigned condition. The two print based assignments included short-answer and essay exercises which were graded and evaluated. In the 1-1 session, the evaluator assumed a passive role while the learner was encouraged to think aloud during assignment completion. Students in the 1-2 session were encouraged to discuss encountered problems as they individually completed the assignments. Those in the 1-3 group were instructed to work independently and passively. After the learners completed the assignments, the evaluator in each group conducted a debriefing session with prepared questions. Analysis and Results Overall, the 1-1 condition identified the most number of problems with the greatest detail, but the evaluation process involved the most time and effort on the part of both the learner and evaluator. The 1-2 group identified the second highest number of problems, but evaluated the Article Reviews product in conditions that were most similar to actual use. The 1-3 group identified the fewest 12 number of issues, but their time to complete the assignment was closest to actual use. In addition, the groups differed based on the types of problems identified. The 1-1 and 1-2 conditions focused heavily on problems associated with the situation statements while the 1-3 group focused on the problem or the problem choice within the assignment. Critical Summary This study is valuable for its qualitative comparison of the one-to-one and small group learner evaluation. While no clear cut winner is established, that was not the point of the study. Rather, the study suggests a comparison and tradeoff between efficiency and effectiveness. While one-to-one evaluation may provide the greater efficacy in terms of problem identification, it will come at a higher cost in terms of time and energy on the part of the learner and evaluator. In contrast, the small group learner evaluation may offer a more efficient and practical evaluation method, but may not offer the most breadth and depth of problem identification. Application This study suggests that evaluators need to consider the practical efficacy and efficiency tradeoffs and implications when creating an evaluation plan. While one-to-one evaluation may lack efficiency, greater efficacy may result in terms of breadth and depth of problem identification. In contrast, small group may offer a more efficiency when the evaluator does not have the luxury of time and budget to run a series of one-on-one learner reviews, but fewer issues may be identified.

IDT873 Maddrell Procedures Abstract 8 - Upload a Document to Scribd

IDT 873 Abstracts: Procedural Skills Jennifer Maddrell Blakemore, C. L., Hilton, H. G., Harrison, J. M., Pellett, T. L., & Gresh, J. (1992). Comparison of Students Taught Basketball Skills Using Mastery and Nonmastery Learning Methods. Journal of Teaching in Physical Education, 11(3), 235-247. Research Purpose and focus. Blakemore, Hilton, Harrison, Pellett, and Gresh (1992) analyzed mastery learning as a means of teaching psychomotor skills. The study’s purpose was to compare mastery and nonmastery learning methods as means of teaching basketball skills. Methodology. Three physical education classes of seventh-grade boys were randomly selected as treatment and control groups. One treatment group of 39 boys was taught using a mastery model while the other treatment group of 32 boys was taught using nonmastery methods. The control group of 33 boys was taught soccer and hockey. The same instructor taught the three classes. Instruction lasted six weeks for 50 minutes a day five days a week. Students in the mastery treatment were taught using Bloom’s mastery learning model which is based on both individual student need and group mastery. The mastery group’s routine included warm-ups (5 minutes), diagnostic tests (10 minutes), corrective and enrichment practice with feedback (10 minutes), and competitive game play (10 minutes). Each session’s diagnostic test confirmed whether or not 80% or more of the class had achieved mastery which was the trigger to move onto a new skill unit. The tests also served as a means of evaluating and providing feedback about a student’s progress toward skill attainment. Instruction in basketball skills in the nonmastery class included the same skills taught in the same order, but followed a predetermined instructional plan and schedule that included warm-ups, skill instruction, practice, and game play with timing based on the planned schedule. Both isolated skills (dribbling, shooting, and layups) and game play ability (based on shots taken, shots made, and turnover game statistics) were assessed in pretests and posts tests for students in both groups. Results and conclusions. Pretests confirmed that the groups were of equivalent starting skill ability. From pretest to posttest, only the mastery group demonstrated statistically significant improvement in the three assessed skills. The control and nonmastery groups improved only in the ability to dribble, and the nonmastery group decreased in shooting and layups. In terms of game play, there were no significant differences between groups from pretest to posttest. Heuristics The results of this experiment suggest value in incorporating periodic diagnostic evaluation and feedback when teaching skills. As is suggested by Bloom’s mastery learning model and the results of this test, frequent evaluation with corrective and enrichment feedback provides greater instructional support than simply presentation and practice alone. Critique This study is straightforward and clearly outlined in the research report … FINALLY! These findings are important as they suggest individualized feedback with information about learner performance results and suggested corrective strategies delivered immediately following skill testing may enhance skill development beyond presentation and practice alone. However, it is unclear how the results would be impacted if learners of mixed ability were combined within the mastery session. Would higher ability learners become bored waiting for lower ability learners to reach mastery? Would lower ability learners become frustrated and embarrassed about holding back the other members of the group? Page | 1 Submitted 20081023 IDT 873 Abstracts: Procedural Skills Jennifer Maddrell Harrison, J., Preece, L., Blakemore, C., Richards, R., Wilkinson, C., & Fellingham, G. (1999). Effects of two instructional models - Skill teaching and mastery learning - On skill development, knowledge, self-efficacy, and game play in volleyball. Journal of Teaching in Physical Education, 19(1), 34-57. Research Purpose and focus. As a follow up to the 1992 study discussed above, Harrison, Preece, Blakemore, Wilkinson and Fellingham (1999) again analyzed mastery learning as a means of teaching psychomotor skills. However, in this study, volleyball skills were the subject of the instruction and Mastery Learning was compared to a Skill Teaching method. Beyond skill attainment, knowledge and self-efficacy measures were also compared. Methodology. 182 students including both males and females in six college volleyball classes participated in the study. Based on a four skill (set up, passing, serving, and spike) pretest, the students were stratified into high, medium, and low ability groups for analysis only. While all students participated in the classes as enrolled, only the high and low skilled learners (147 in total) were included in the analysis. Given prior studies in which the no instruction intervention control group showed no improvement, a control group was not included. Instruction was taught by three graduate assistants who each taught two classes, one under a Mastery Learning model and one under a Skill Teaching model. Each of the six volleyball classes was randomly assigned to one of the six courses and was taught under either the Mastery Learning model or Skill Teaching model. Instruction lasted 16 weeks with class sessions held two days a week. Students in the mastery treatment were taught using Bloom’s mastery model which is based on individual student need and group mastery. Each mastery group’s class sessions included formative testing, corrective and enrichment practice with feedback, and competitive modified game play. The formative tests confirmed whether or not 80% or more of the class had achieved mastery which was the trigger to move from Subunit I (forearm pass, set, overhand serve, spike, and mini-volley games) to Subunit II (full court games, 4-2 offense, player up defense, serve / receive, block, spike, and dive). While the focus of the Mastery Learning model was frequent diagnostic tests followed by feedback, the focus of the Skills Teaching model was hands on practice in modified game play. Instruction using the Skill Teaching model was sequenced using Rink’s system in which skills were taught using modified equipment (lower nets, non-standard balls) in classes that included warm-up, skill practice, and modified games (simplified rules and extra points for targeted skill attainment). The same skills were taught, but followed a predetermined instructional plan and schedule. Pretest, midterm, and posttest measures of isolated skills (set-up, passing, serving, and spike) were assessed. In addition, self-efficacy was assessed following the pretest, midterm, and post test assessments. A 63 item objective knowledge test on techniques, rules, and strategies was conducted at the end of the term with mastery level set at 80%. Results and conclusions. While significant pretest and posttest differences were found in both groups, the major finding was that Skill Teaching and Mastery Learning produced similar levels of improvement in the measures of isolated skill attainment, game play, self-efficacy, and knowledge. Given the few significant differences between outcomes following the use of the two Page | 2 Submitted 20081023 IDT 873 Abstracts: Procedural Skills Jennifer Maddrell models, the researchers concluded that there was no compelling finding to suggest one model over the other. Heuristics The results of these experiments suggest that designers should incorporate diagnostic evaluation followed by feedback when teaching skills. In addition, including frequent hands on practice using modified equipment and rules can help learners to build skills over time. Critique As with the first reported study, this study is straightforward and clearly outlined … yeah! These findings are important as they add to the understanding of mastery learning as a method to teach skills from the 1992 study. When considered as an extension of the 1992 study, this study suggests that both options are preferable to only presentation and practice alone. However, it is unfortunate that the researchers did not directly discuss and compare these methods and results with the 1992 study’s methods and findings. Suggestions for the similarities and differences between the two studies were not explored. Further, the researchers did not fully explore the connection between gender and skill difference and the effect that may have had on the participants in the study. The researchers noted that the majority of the high skilled group members were male and the majority of the low skilled group members were female and both groups participated together in the classes. Could gender and the comingling of skill levels within the treatment groups have played a more significant role than the researchers discussed? Did the girls (of lesser skill ability) feel intimidated playing with boys (of higher skill ability)? Page | 3 Submitted 20081023

IDT873 Maddrell Abstract Rules 7 - Upload a Document to Scribd

IDT 873 Abstracts: Rules and Principles Jennifer Maddrell Ross, S. M., & Rakow, E. A. (1981). Learner Control versus Program Control as Adaptive Strategies for Selection of Instructional Support on Math Rules. Journal of Educational Psychology, 73(5), 745-53. Research Purpose and focus. Ross and Rakow (1981) sought to extend previous research on the effect of adaptive instructional strategies on math rule attainment. They predicted that an adaptive design strategy which varied the number of practice examples based on the student’s pretest score would improve rule attainment over a nonadaptive strategy. Methodology. 124 undergraduate students volunteered to participate in the study. The students were randomly assigned to one of four treatment groups, including 1) program control (the number of examples were adapted to pretest scores), 2) learner control (the number of examples were selected by subjects), 3) nonadaptive (a constant five examples per rule were presented to learners), and 4) lecture (the nonadaptive program was presented through lecture format) groups. The learning program consisted of presentation of five introductory math rules, including inequalities, factorials, exponents, order of operations, and summation. Presentation for each rule included the rule definition, as well as complete and incomplete examples. Other than the lecture group, the instruction for all of the groups was done through self-study booklets in a lab session with three or less subjects who worked with a single proctor to complete a separate booklet for each rule. Those in the lecture group received the identical content as the nonadaptive self-study group within a presentation given by an instructor. Achievement assessment included a pretest taken a few weeks prior to the instructional phase, a posttest taken after completion of each rule booklet, and a delayed posttest given a few weeks after the instruction. The achievement tests consisted of open ended test questions. The study also incorporated an attitude survey which was taken after the completion of the last rule booklet. Results and conclusions. The results indicated statistically significant differences in posttest achievement based on the treatment strategy. The immediate posttest results indicated the mean score of the program control group was higher than all other treatments while both the lecture and the nonadaptive groups performed better than the learner control group. In the delayed posttest, the differences were the same, but more pronounced. However, the noted results for the learner control group varied based on entrance ability. While low entrance ability students performed well under program control and poorly under learner control, high entrance ability students performed well under both. Further, there was no significant difference among treatment groups on the attitude survey scores. Heuristics The results of this study suggest value in adapting instructional presentation based on learner need. Based on the findings of this study, increasing the presentation of examples for those with low entrance ability and decreasing presentation for those with high entrance ability may improve rule attainment. Page | 1 Submitted 20081015 IDT 873 Abstracts: Rules and Principles Jennifer Maddrell Critique The results of this study support prior research findings which suggest value in modifying the amount of instructional support based on individual need. Further, this study suggests entrance ability assessment may be an effective means of gauging the amount of needed instructional support. In addition, this study suggests that for low entrance ability learners, learner control in gauging an optimal presentation may not be an effective strategy. However, it is important to note that learners in the learner control treatment were required to ask the proctor for additional examples which may have made the learners uncomfortable and less likely to ask for additional examples. Wiley, J., & Voss, J. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91(2), 301-311. Research Purpose and focus. Wiley and Voss (1999) evaluated the effect of student generated arguments on learning historical subject matter. Two separate experiments were conducted. The purpose and methods were similar, namely to evaluate whether argument writing tasks promoted a deeper understanding of the to-be-learned material than other narrative, summary, or explanation writing tasks. Methodology. 24 undergraduate students participated in the second study. The students were randomly assigned to one of four treatment groups, including 1) a narrative group, 2) a summary group, 3) an explanation group, and 4) an argument group. All groups received the same information about Ireland from 1800 to 1850, either in a paper based or computer based newspaper article. After reading the material, students were asked to assume the role of historian and, based on their assigned treatment, develop either a written narrative, summary, explanation, or argument about what produced the significant changes in Ireland’s population between 1846 and 1850. Learners were given approximately 30 minutes to read the material and complete their reflective writing task. After the writing task, participants were assessed based on three learning measures including 1) a sentence verification task (10 true / false questions), 2) an inference verification task (determining if statements were true on the basis of the presented information) and 3) a principle identification task in which students indicated how similar the causes of the Irish Potato Famine were to other historical situations. In addition, the sentences in each student’s writing task were classified based on whether the sentences were a) borrowed from the original source, b) transformed, or c) added information. Results and conclusions. The results indicate little difference between whether students read the newspaper article from the computer or paper. Further, there were no significant differences across treatment groups in the recognition of sentences. However, those in the argument writing treatment demonstrated better identification of inferences and generated essay sentences with more transformed and causal information. In contrast, the other writing tasks resulted in essays with more borrowed and added sentences and less causal information. Heuristics The results of these experiments suggest the nature of the reflective writing task impacts the learner’s attainment of the to-be-learned material. Based on the conclusions of the researchers, writing tasks which require the learners to form and support arguments about causes Page | 2 Submitted 20081015 IDT 873 Abstracts: Rules and Principles Jennifer Maddrell of events may lead to better recognition of the underlying principles in the subject matter than can be achieved in writing tasks that focus on just narrative summaries or explanations. Critique In terms of the teaching of rules and principles, this study offers support for what Wiley and Voss term knowledge-transforming versus knowledge-telling tasks. Wiley and Voss argue that the findings of their study suggest argument writing is a knowledge-transforming task which requires learners to go beyond mere recall of principles to the construction and subsequent defense of arguments to explain the root causes of events. They suggest that through argument writing tasks the subject matter is learned at a deeper level given that learners must retrieve and relate more information in order to justify their positions. While this study was small in scope and scale, it offers an intriguing stepping stone for future research on the effects of various types of reflective writing tasks on rule or principle attainment. Page | 3 Submitted 20081015

IDT873 CTA Maddrell - Upload a Document to Scribd

Cognitive Task Analysis Running head: COGNITIVE TASK ANALYSIS 1 Cognitive Task Analysis Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Gary Morrison October 15, 2008 Cognitive Task Analysis Traditional Task Analysis A traditional procedural task analysis describes a task as a series of discrete actions (Jonassen, Tessmer, & Hannum, 1999). Figure 1 diagrams a procedural task analysis for the insurance underwriting submission review task. Within this triage task, the underwriter must evaluate various aspects of the new submission and decide whether to quote or decline the submission. Figure 1. Procedural Analysis of Insurance Underwriting Submission Review Task. 2 Guide to symbols: = Input and exit point; = Mental operation; = Decision Point; = Direction in Step Cognitive Task Analysis 3 As depicted in Figure 1, in completing the submission review task, the underwriter must make a series of mental operations and decisions in route to a conclusion to either a) decline the submission or b) quote the submission. These mental operations and subsequent decisions include the following: Assessing the viability of the opportunity. Upon receipt of the submission, the underwriter must make a quick review of the information provided to assess the viability of the opportunity. Given the information presented within the submission and discussions with the broker, the underwriter must judge the likelihood the account will actually leave the incumbent carrier. Critical cues to consider include prior service and claims handling problems with the incumbent carrier, time to transition the account, and completeness of the submission. If the relationship with the prior carrier has been good, there is little time to transition the account, or the broker only provided enough information to provide a price (not service) quote, it is likely the insured is not serious about moving from the incumbent carrier and the broker is just seeking comparative price quotes. However, if the insured is dissatisfied with the incumbent carrier’s service, there is ample time to transition the servicing of the account, or the submission provides a comprehensive overview of both price and service requirements, it is likely the opportunity is viable. If the assessment of the information leads to a conclusion that the chances are slim the account will move, the underwriter makes the decision to decline the account. However, if the assessments leads to a conclusion that there is a good chance of writing the account, the underwriter makes the decision to continue working on the account. Examining the employee concentrations. Given the potentially catastrophic exposure of providing casualty insurance at locations with high employee concentrations, the underwriter’s triage of the submission includes an examination of employee concentrations. If the insured has employee concentrations at any one location above company guidelines, the underwriter makes the decision to decline the account. Otherwise, the underwriter makes the decision to continue working on the account. Comparing the account’s exposures to the company’s underwriting guidelines. Upon receipt of the submission, the underwriter must compare the prospective account’s exposures to the insurance company’s underwriting guidelines. Critical to this comparison is a review of the insured’s current and prior operations. If the company is involved in any operations which result in exposures that are against the underwriting guidelines, the underwriter makes the decision to decline the account. Otherwise, the underwriter makes the decision to move forward with the quotation task (beyond the scope of this submission triage task analysis). Cognitive Task Analysis A cognitive task analysis (CTA) offers an alternative means of describing the cognitive elements of the evaluation and decision making processes involved in the task. The following provides the results of an Applied Cognitive Task Analysis (ACTA) based on interviews conducted with an underwriting subject matter expert (SME) to gain information about cognitive strategies used to complete the submission triage task (Militello & Hutton, 1998). The ACTA includes a task diagram, knowledge audit table, simulation interview, and cognitive demands table. Cognitive Task Analysis 4 Task diagram Figure 2 is the task diagram generated after an initial interview with the underwriting SME. The task diagram offers a high level overview of the submission triage task which focuses on the most difficult cognitive aspects. The SME was asked, “Think about what you do when you triage a new prospect. Can you break this task down into less than six, but more than three steps?” The SME mentioned five steps, but one was eliminated (financial approval) as it is not task performed by underwriter. Figure 2. Task Diagram for New Account Prospect Triage. Knowledge Audit Table During interviews with the SME, the interviewer probed for concrete examples, cues and strategies, and reasons why the task is often difficult for novices. The interviewer asked the SME to focus on specific examples for each aspect of expertise. Table 1 summarizes the results of the knowledge audit for the submission triage task. Simulation Interview During a simulation interview with the SME, the interviewer asked the SME to focus on the challenging aspects of a specific representative scenario associated with new submission triage. Table 2 summarizes the results of the simulation interview, including the actions, assessments, cues, and potential errors identified for each central event. Cognitive Demands Table Table 3 consolidates and synthesizes the data collected during the interview process. The cognitive demands table centers on the common themes that came from the interviews and identifies the difficult cognitive elements, common errors, and cues or strategies used by experts to overcome these challenges. Cognitive Task Analysis Table 1. Knowledge Audit Table. Aspect of expertise Past and future Example: Call from broker about account where incumbent carrier messed up on claim and insured’s legal department insisting the account must move. 5 • • Cues and strategies High level nature of incumbent mess up Level of people involved in decision (low level versus high level) • • Why Difficult? Novice may not recognize significance of messed up claim handling Novice may not link level of insured to severity of problem Novice may not link severity of problem to increased chance of writing account. Novices may not consider other issues beyond price that influence buying decision Novices do not have relationship with broker to know when you are getting the “straight” facts versus a “sales pitch” Novices may get into the minutia of the account specifics and not step back and realize the timeframe is not feasible to actually move the account Novices are focused on details within submission Novices are familiar with “outside” considerations that affect the likelihood of writing the account • • • • Big picture Example: Steps back from all the facts about the account presented by the broker to consider what is the “real” motivation behind looking for a quote? Is this prospect a true opportunity or does the broker just need a competing price quote? If it is only a need to get competing price quotes, highly unlikely the account will move. Noticing Example: Broker not soliciting TPA quotes for claim handling which would be a #1 condition of actually moving the account. Job Smarts Example: Focus on what broker said in conversation versus purely what is presented in the quote. Opportunities Example: Our unit can’t work on this account, but other units in company can. Anomalies Example: Broker doesn’t return phone calls. Shows a lack of interest. • • • • Beyond price, there service issues with prior carrier Your personal history with that broker. Time frame to release quote What other carriers are quoting • • • • • Going beyond underwriting information presented in the submission Considering conditional things that impact your quote Timeframes Others carriers being asked to quote. Reasons for leaving Understanding of underwriting appetite of other units Knowing how to access those people Timing of returned phone calls Extent of response to questions • • • • • • • • • Novices tend to be preoccupied with verifying details within submission Novices not aware of situational issues that can be “deal breakers” or “deal makers” Novices don’t know underwriting appetite of other units Novices don’t know people outside of the unit Novices may not recognize they are “getting blown off” and they continue working on submission • • Cognitive Task Analysis (either lacking or detailed) 6 • Novices don’t recognize significance of “out of sight / out of mind” which is signal if you are alive or dead Table 2. Simulation Interview. Events Discussion about prospect with broker Actions Ask probing questions about opportunity Sensing tone from broker of urgency and desire to have you quote. Assessment Answers to question make sense or not with what is in the submission Broker wants to work with you or just wants a quote for comparison purposes How much time is there between now and effective date? Are the exposures inherent in risk acceptable under our underwriting guidelines? Critical Cues Can you meet the issued There is disaffection with incumbent Openness of the broker Willingness to provide additional information Too much time signals the broker is “shopping” for an early quote. Too little time signals that broker just wants to keep current carrier “honest” “Red flag” exposures that we cannot write “Go” classes of business that we are targeting Potential Errors Being overly optimistic about any opportunity Not probing deeply for hidden facts about situation Not reading the verbal and nonverbal cues the broker is giving you. • • • • • • • • • • • Deciding whether to quote • • Evaluating time frame between quote deadline and effective date Assessing if account meets underwriting guidelines • • • • • • • • • Being so excited about the opportunity that you rush to judgment Spin wheels on accounts where there isn’t a true opportunity Don’t dig deeply enough into what the account really does or did in the past that could represent “hidden” exposures Cognitive Task Analysis 7 Cognitive Task Analysis 8 Table 3. Cognitive Demands Table. Difficult cognitive elements Assessing whether broker’s answers make sense or not with what is in the submission Considering the “real” opportunity and exposures beyond the obvious information given in the submission Comparing account’s exposure information with underwriting guidelines Why difficult Common errors Cues and strategies used • • Consider if you really know the story behind the story Get and keep the broker talking to elicit information beyond the submission Ask about reasons why account would move Consider whether timeframe to move account is realistic • • • • • • • Considering and suggesting alternatives • • Novice underwriters tend to focus on basic facts in the submission versus what the broker is telling them Brokers reluctant to voluntarily air dirty laundry about account Novices underwriters tend to focus on information given versus information needed to make decision Can be uncomfortable situation for novice underwriters to probe for answers Companies often have many types of operations which cross several classes of business Novice underwriters tend to focus on the primary business operations Novice underwriters often have difficult assigning an account to the appropriate business classification within the guidelines. Novice underwriters tend to focus on what broker is asking you to do Novice underwriters often fail to identify ways to adjust quotation options to meet guidelines • • Don’t recognize or probe for hidden “red flags” Focus exclusively on information in submission Taking the submission at “face value” Failing to engage in uncomfortable probing conversations with the broker Failing to fully capture exposures Getting lost in the details Misinterpreting underwriting data Misinterpreting the underwriting guidelines • • • • • • • • • Review account with senior underwriter Check multiple sources to evaluate exposures • • • Quote only what is asked by broker Failing to probe for alternate opportunities with the broker • • Consider ways to adjust quotation options to fit within underwriting guidelines. Consider other coverages and limits that you or other departments could quote Cognitive Task Analysis Comparison of Approaches 9 Analysis Comparison In comparing the results of the traditional task analysis with the cognitive task analysis, significant differences emerge in following areas: a) the identification and analysis of hidden cognitive processes, b) the relative level of elaboration regarding the central task elements, c) the focus on expert and novice differences. Overt behaviors versus cognitive processes. The key strength of the traditional task analysis is the ability to examine overt behaviors required to complete a task. However, as seen in this example, additional critical cognitive processes and actions were uncovered within the ACTA. Further, the ACTA offered a means of analyzing the relative significance and difficulty of the required task elements. Level of elaboration. The traditional task analysis identified the relevant processes and decision points in the submission triage task. However, by focusing on the difficult cognitive aspects of the task, the ACTA provided greater elaboration with regard to the knowledge and cognitive processes required to perform the task. As the cognitive demands table highlights, the ACTA focused attention on the difficult cognitive elements, common errors, and strategies to overcome those difficulties and errors. Unfortunately, these elements were not unearthed within the traditional task analysis. Focus on expert and novice differences. Unlike the traditional task analysis, the ACTA analysis focused on the central differences between how an expert and a novice perform the submission triage task. The result is a comparison of current state (novices) and desired state (experts), as well as strategies to take the novice to an expert level. Implications for Practice Traditional task analysis allows practitioners to target the inputs, central operations, and decision points involved in carrying out a task. While this provides a good overview of what happens as the task is carried out, it does not provide the designer with an understanding of the nature of the cognitive processes required to complete the task. Further, following a traditional task analysis, the practitioner cannot gage the relative importance of the various tasks elements or which aspect(s) of the task are harder for the novice. As seen in the results between the two analyses, the cognitive task analysis provides practitioners with a better understanding of the difficult and critical cognitive processes, as well as the and cues and strategies, which are central to successful completion of the task. When to use Traditional Task Analysis versus Cognitive Task Analysis Both a traditional task analysis and cognitive task analysis highlight key aspects of the task. However, as seen in the two analyses above, each produces different results. As noted, the cognitive task analysis offers a better analysis of the central knowledge and decision making cognitive processes. Given that each task is different, the following provides a comparison of which analysis is more appropriate based on the degree of observable behaviors, the degree of required expertise, and the relative cognitive difficulty of the task. Cognitive Task Analysis 10 Degree of observable behaviors. The difference in outcomes between the two approaches is likely less significant when the task involves primarily observable behaviors. However, if the task involves primarily mental actions that result in less observable behaviors, a cognitive task analysis is the more appropriate option. Expert versus novice differences. When little task related expertise is required to perform the task, the results of both analyses would likely be similar. However, if successful completion of the task requires knowledge that a novice would not possess, a cognitive task analysis allows the practitioner to uncover or drill down on the difficult cognitive elements. As noted, these cognitive elements are less likely to be adequately analyzed in a traditional task analysis. Relative cognitive difficulty. While a traditional task analysis provides a comprehensive outline of the steps in the task, it does not offer a relative assessment of which steps are harder or more critical to successful completion. Instead, each step in the task is considered equally. However, as seen in the cognitive demands table, some tasks hinge on a smaller number of critical or difficult elements. Therefore, the ACTA is more appropriate when successful task outcomes depend upon cognitively difficult judgments or decision. Cognitive Task Analysis 11 References Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis methods for instructional design. Mahwah, N.J.: L. Erlbaum Associates. Militello, L. G., & Hutton, R. J. B. (1998). Applied cognitive task analysis (ACTA): a practitioner’s toolkit for understanding cognitive task demands. Ergonomics, 41(11), 1618-1641.

IDT873 Maddrell Concept Abstract 6 - Upload a Document to Scribd

IDT 873 Abstracts: Concepts Jennifer Maddrell Klausmeier, H. J., & Feldman, K. V. (1975). Effects of a definition and a varying number of examples and nonexamples on concept attainment. Journal of Educational Psychology, 67(2), 174-178. Research Purpose and focus. Klausmeier and Feldman (1975) focused their research on concept attainment which they defined within their study as the ability to a) discriminate defining attributes, b) name the concept and each defining attribute, c) evaluate examples and nonexamples, and d) define the word representing the concept. In reviewing prior literature on concept attainment, they highlighted four categories of variables generally studied, including 1) a rational set of examples and nonexamples, 2) definitions of a concept (based on the relevant attributes of the concept), 3) emphasizers to facilitate discrimination, and 4) feedback. The purpose of this study was to evaluate the effect of presenting various combinations of concept definitions and rational sets. They predicted better attainment from those presented with both a rational set and a definition than those presented with either one or the other. Further, they predicted better attainment from those presented with the definition and additional different rational sets. Methodology. 134 fourth-grade students from two Wisconsin (Go Badgers!) elementary schools participated in the study. The students were stratified into high, medium and low levels based on their performance on the most recent Iowa Tests of Basic Skills test. The subject matter concept was the equilateral triangle. Students within each stratification level were randomly assigned to one of four treatment groups which included those presented with 1) a definition of the concept without examples or nonexamples, 2) a rational set of three examples and five nonexamples, 3) a combination of the same definition and rational set, and 4) a combination of the same definition and three different rational sets of three examples and five nonexamples. The treatment lesson was presented in four printed lesson booklets. Following instruction, students were given 1 minute to read each lesson page and then were instructed to turn to the next page allowing 5 minutes per lesson booklet. Immediately following the last lesson, a classification task within a printed booklet measured concept attainment. Without time limit, students viewed 38 instances and were asked to identify whether the instance was an example (by circling yes) or nonexample (by circling no) of an equilateral triangle. Results and conclusions. Means for the stratified groups reflected the initial levels with means for high > medium > low. As predicted, no significant difference in concept attainment was found between those who were presented with either a definition or a rational set. Contrary to the researchers’ prediction, there was also no significant difference from a combination of a definition and the single rational set. However, there was a significant difference between those presented with a definition and those who also received three rational sets. These findings are important as they suggest an advantage for presenting additional rational sets of examples and non-examples. Heuristics The results of these experiments suggest that designers should augment the presentation of the concept definition with multiple rational sets of examples and non-examples when teaching concepts. As seen in this experiment, providing learners with additional rational sets to consider may increase their attainment of the concept. Critique Page | 1 Submitted 20081008 IDT 873 Abstracts: Concepts Jennifer Maddrell The results of this study are important as they provide support for the hypothesis that presenting learners with more examples and non-examples is better. However, if three sets of examples and non-examples are better than one, is more than three even better? A criticism of this study is the short intervention and the focus on a single math related concept. Would these results be replicated over a longer period of time with other types of concepts and with different age groups of learners? Tennyson, R. D., & Rothen, W. (1977). Pretask and on-task adaptive design strategies for selecting number of instances in concept acquisition. Journal of Educational Psychology, 69(5), 586-592. Research Purpose and focus. Tennyson and Rothen (1977) sought to expand the previously reviewed work of Klausmeier and Feldman (1975) by evaluating the effect on concept attainment of adapting the number of examples and nonexamples based on individual need. They predicted that an adaptive design strategy that varied the presentation of examples and nonexamples based on student need would improve concept attainment over a nonadaptive strategy. Methodology. 67 undergraduate students participated in the study. The students were randomly assigned to one of three treatment groups, including 1) full adaptive, 2) partial adaptive, and 3) nonadaptive. The adaptive designs were modified using a computer-based Bayesian adaptive strategy which altered the number of examples learners viewed based on a) pretreatment measures of aptitude, b), pretests of prior achievement, and c) task performance. A pretest, treatment lesson, and posttest were administered individually via computer. The untimed lesson focused on two legal concepts, including best evidence rule and hearsay. For all groups, the learning task defined the concept based on the critical attributes of the concepts. The number of instances presented to students varied based on their assigned treatment group. The nonadaptive group received the same number of instances. The number of instances in the partial adaptive model was based on pretest data while the number presented within the full adaptive model was modified based on both pretest data and on-task responses. The study also evaluated the time on task which did not include pre- or post-test time. Results and conclusions. While no significant mean differences were found in pretest measures, significant mean differences were reported regarding time on task and posttest score measures. As predicted by the researchers, the results suggest that full adaptive strategies were more effective than partial adaptive strategies and that the two adaptive strategies were more effective than nonadaptive conditions. In addition, the full adaptive group finished the program significantly faster than the partial group which in turn finished faster than the nonadaptive groups. In attempting to explain the results, the researchers suggest that learning tasks where instance presentation is not modified based adaptive strategies may not keep learners’ interest in the task. Heuristics The results of these experiments suggest modifying instructional concept presentation based on learner mastery. Based on the findings of this study, presentation of examples and nonexamples after the learner has achieved mastery may result in learners losing interest in the learning task. Critique Page | 2 Submitted 20081008 IDT 873 Abstracts: Concepts Jennifer Maddrell The results of this study are important as they suggest that optimal presentation varies based on the each individual learner’s level of mastery. In this controlled experiment, using a computer based model, the researchers were able to alter the individual presentation based on each learner’s level of mastery which resulted in more effective instruction. However, altering presentation to an individual learner in real-world instructional settings is difficult, especially in group face to face settings. Therefore, while the results suggest an important finding with regard to tailoring instruction to meet the individual learner, such modifications may not be feasible in practice. Page | 3 Submitted 20081008

IDT873 Maddrell Fact Abstract 5 - Upload a Document to Scribd

IDT 873 Abstracts: Facts Jennifer Maddrell Woloshyn, V. E., Willoughby, T., Wood, E., & Pressley, M. (1990). Elaborative interrogation facilitates adult learning of factual paragraphs. Journal of Educational Psychology, 82(3), 513-524. Research Purpose and focus. Woloshyn, Willoughby,Wood, and Pressley (1990) evaluate elaborative interrogation as a means of teaching facts. Two separate experiments were conducted. While the second was performed after the results from the first were known, the purpose was the same, namely to evaluate whether encouraging learners to activate and elaborate upon relevant prior knowledge facilitates fact acquisition. Methodology. In the first experiment, 80 college students participated in a single session study at a Canadian university. The students were randomly assigned to one of four experimental conditions, including 1) elaborative interrogation, 2) imagery, 3) self-reference, and 4) reading control groups. Those in the elaborative interrogation condition received accompanying “why” prompt questions on the slide and were asked to respond to the question. Those in the imagery condition were asked to “create an image” of the fact and associate it with the university. Selfreference subjects were asked to consider whether the shown fact would influence a decision to attend the school. Reading-control subjects were prompted to “read the sentence out loud at a rate that allows you to understand the fact.” The initial study consisted of four phases including 1) the instructions, 2) presentation of the study material, 3) tests which included both fact recall for each school and associative matching of the facts to the school, and 4) a post test interview which addressed the students’ attention to the instructions, the difficulty of the task, and prior knowledge about the school. The 20 students in each group viewed 43 slides which contained facts about the Canadian universities not well known to those not attending the schools. However, the instructions to each group differed based on the previously noted experimental condition. The second study included 240 college students from the same school, using the same materials, and incorporating the same four phases. However, the focus was on longer paragraphs beyond the single sentences in the first experiment. In addition, a free-recall memory test was included. Results and conclusions. While there were no significant differences between elaborative interrogation and imagery conditions, both groups performed significantly better than the reading control condition in the test measures. Further, in the first experiment, the self-reference subjects performed significantly better on the associative matching test than the reading control group, but the difference was not deemed significant on the fact recall. In post test interviews, subjects deemed elaborative interrogation and imagery as more “difficult” and requiring “extra efforts” than the reading control group. These findings are important as they suggest that elaborative interrogation yields superior results to the reading alone. Further, elaborative interrogation was as effective as the other elaborative procedures studied. Heuristics The results of these experiments suggest that designers should incorporate elaboration strategies when teaching facts. As seen in this experiment, prompting learners to elaborate beyond the presented fact (as in considering specific questions, creating mental images, or selfreflecting on the presented facts) provides greater support than simply presenting the fact alone. Page | 1 Submitted 20081001 IDT 873 Abstracts: Facts Jennifer Maddrell Critique The greatest strength of this research is the contribution of prescriptions for the teaching of facts. As noted, the results suggest instruction should encourage and foster elaborative strategies to support the learning of facts. In addition, the research sets the stage for further study on whether these strategies can be taught. Can learners be taught to use these strategies in a selfregulated manner? However, while the article presents a review of literature suggesting that elaborative interrogation strategies may involve more conscious and effortful memory processes, the articles falls short in explicitly evaluating the reported experimental results in the context of specific theory within either the discussion or conclusions. In other words, how do the results support or refute a specific theory being tested? Kuo, M., & Hooper, S. (2004). The effects of visual and verbal coding mnemonics on learning Chinese characters in computer-based instruction. Educational Technology Research and Development, 52(3), 23-34. Research Purpose and focus. The purpose of the research conducted by Kuo and Hooper (2004) was to evaluate the effects of visual and verbal mnemonics on memorization, as well as any differences in outcomes between self-generated versus supplied mnemonics. The study focused on the learning and recall of Chinese characters. Kuo and Hooper noted that Chinese characters contain both visual and symbolic meanings, yet many traditional instructional methods tend to ignore the underlying meaning of the character and instead focus on repeated copying of the character to improve recall. In the reported study, Kuo and Hooper questioned whether or not visual and verbal mnemonic strategies could help the learner to generate meaning based on the visual and semantic information of the characters and if those strategies could improve recall. They examined whether verbal or visual mnemonics for learning Chinese characters would benefit both immediate and delayed recall, as well as the relative efficiency and efficacy of self-generated versus experimenter-supplied mnemonics. Methodology. 92 English speaking high school students with no previous Chinese language knowledge volunteered to participate in the study. The students were randomly assigned to one of five treatment groups, including 1) translation in which students were presented with the Chinese character and the English translation and told to memorize the character’s meaning, 2) verbal mnemonics (experimenter-supplied), 3) visual mnemonics (experimenter-supplied), 4) dual coding which included both verbal and visual experimentersupplied mnemonics, or 5) self-generated mnemonics in which students were presented with the Chinese character and the English translation and encouraged to create memory aid such as a picture, written sentence, or an associating story. The instruction to all of the groups was done through a self-paced computer-based tutorial conducted during a regularly scheduled single class period. The tutorial for all groups included the same 30 characters, divided between concrete words (representing physical objects) and abstract words (without referents). Two posttests were also administered via computer and included the same, but shuffled multiple-choice questions. The first test was administered immediately following the tutorial and the second one week later. The study also evaluated the time on task during the tutorial to learn Page | 2 Submitted 20081001 IDT 873 Abstracts: Facts Jennifer Maddrell the 30 characters. The self-generated mnemonics were also collected, sorted by character, and analyzed. In addition, a short survey was conducted from a subsample of students reflecting a representative cross-section of ability to examine the students’ learning strategies. Results and conclusions. While participants scored higher on the immediate versus the delayed posttest, students who self-generated their mnemonics performed significantly better in the recall posttests and spent more time on task than those in the translation, verbal, visual, and dual coding conditions. Further, the survey responses suggested that the learners used the presented or self-generated mnemonics techniques and called upon the mental images to link the character with the English word. The analysis of the self-generated mnemonics indicated that fewer and less descriptive mnemonics were created for abstract words than for concrete words. In addition, the descriptions reflected the students’ Western cultural backgrounds and experiences. These results suggest that the learners linked the to-be-leaned information with prior knowledge. Heuristics The researchers suggest that the findings support the use of generative learning strategies that extend beyond just efficient information presentation. Strategies should encourage learners to self-generate meaningful linkages between the content and prior knowledge. However, while the findings suggest that self-generated relationships are more effective, they may be less efficient than provided relationships. Critique This experiment and article describing the research were refreshingly straightforward with no doubts as to the researchers’ questions, methodology, and suggested findings. The study furthers prior theory and research and offers suggestions for future research. Nice package. Tight bow. Page | 3 Submitted 20081001

IDT873 Maddrell Cognite Load Abstract 4 - Upload a Document to Scribd

IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell van Merrienboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the Load Off a Learner's Mind: Instructional Design for Complex Learning. Educational Psychologist, 38(1), 513. Overview Citing decades of prior cognitive load theory and research, van Merrienboer, Kirschner, and Kester (2003) offer a theoretical framework and instruction design model for complex learning. Noting a recent emphasis on authentic learning tasks (such as project and problembased learning approaches) to support complex learning, they consider the implications on cognitive load and offer a model designed to manage both intrinsic and extraneous cognitive load. Theory While the theories underlying the use of authentic learning tasks may vary, a common assumption is that authentic tasks help learners to integrate the knowledge and skills necessary for complex task performance (van Merrienboer et al., 2003). However, given the novice learner’s weak problem-solving methods, they face high extraneous cognitive load when confronted with authentic tasks. In addition, the complexity inherent in the authentic task presents high intrinsic cognitive load. Therefore, based on cognitive load theory, engaging in highly complex authentic learning tasks may strain the novice learner’s limited working memory and subject the learner to excessive cognitive load. Proposal van Merrienboer et al. focus their attention on both the nature and the delivery timing of the presented information. They suggest that supportive information (knowledge necessary for problem solving and reasoning) is best presented before the learner engages in the learning task. Such supportive task specific information is inherently complex and needed in order to know how to approach the learning task. Presenting the supportive information first helps learners construct schemas to be used as they begin task performance. In contrast, van Merrienboer et al. suggest that procedural information (the how to instructions for rule application) is best presented when needed during task performance. They argue that such just-in-time presentation of procedural information reduces the potential for splitattention effects that may occur when the learner attempts to integrate procedural information learned previously with actions he or she is taking now. Heuristics From these suggested practices, van Merrienboer et al. offer an instructional design model (the 4C / ID model) for complex learning that focuses on four components: 1) learning tasks, 2) supportive information, 3) procedural information, and 4) part-task practice. The heuristics for designers within the 4C / ID model is to sequence from simple versions of the whole task beginning with a high level of support and ending with a complex version without support. In addition, as discussed above, supportive information is to be presented in advance of performance while procedural information required to perform the task is to be presented as the task is being performed. Finally, to encourage automaticity, additional repetitive practice should be incorporated for parts of the task. Critique The focus of the article is not an examination of the effects of authentic learning tasks on learning, but rather the implications of incorporating such tasks on the learner’s cognitive load. As such, the article offers a bridge across theory, research, and practice. A key strength of the article is the authors’ focus on the reality of limited working memory and the high cognitive load IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell imposed by authentic learning tasks. The 4C / ID model offers designers a way of incorporating authentic tasks while at the same time better managing cognitive load. However, as a theoretical article, it does not offer results from a study of the model in practice. Do the heuristics within the 4C / ID model help to manage cognitive load? Further, do authentic learning tasks designed within the framework of the 4C / ID model effectively and efficiently support learning? These questions are left to future research.

IDT873 Maddrell Cognitive TA Abstract 3 - Upload a Document to Scribd

Read this document on Scribd: IDT873 Maddrell Cognitive TA Abstract 3

IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Schaafstal, A., Schraagen, J. M., & van Berlo, M. (2000). Cognitive task analysis and innovation of training: The case of the structured troubleshooting. Human Factors, 42(1), 75–86. Research Overview. Following an instructional design evaluation of an existing Royal Netherlands Navy maintenance training course, Schaafstal, Schraagen, and van Berlo (2000) observed a gap between the instruction and the practice of troubleshooting the subject system. They observed that the existing instruction was based largely on the technical equipment documentation from engineers which focused exclusively on the system’s components. Following a comprehensive cognitive task analysis (CTA), Schaafstal et al. revised the instruction under the assumption that maintenance system troubleshooting is a complex cognitive task requiring not only knowledge about the system’s components, but also knowledge about how the system functions and how to consider possible causes and solutions to maintenance problems. The CTA consisted of several observational studies of troubleshooting with technicians of varying expertise levels. Based on information from the CTA, a modified course was prepared which focused on a functional understanding of the system versus the component orientation of the prior course. In addition, general troubleshooting strategies were incorporated which gave learners instruction on how to a) describe the problem, b) generate causes, c) test causes, d) repair, and e) evaluate solutions. Purpose. The purpose of the presented research was to evaluate the modified structured troubleshooting training course and to compare it with the exiting maintenance training course. Schaafstal et al. predicted superior outcomes from the revised course. Methodology. A series of experimental studies compared the learning outcomes of maintenance trainees taking the new structured troubleshooting training course with groups of maintenance trainees taking the existing training course. Outcome measures included malfunction identification, reasoning, and functional understanding of the system. Conclusions. The modifications in the course reduced the course duration by 33% (from six to four weeks). Even at the shortened length, those participating in the new course achieved statistically superior results as compared to those in the original course. Based on the results of the study, Schaafstal et al. suggest that novice technicians lack both a systematic approach to troubleshooting, as well as a functional understanding of the equipment. As seen in prior research, they observed that novices face information overload (lose the forest for the trees), lack hierarchically organized cognitive frameworks, lack functional understanding, possess inadequate mental models of underlying system, and lack the ability to isolate causes of the problem. Therefore, based on the results of their evaluation, they suggest that training in troubleshooting should focus on three areas: 1) system independent troubleshooting strategies to be used across systems, 2) system specific functional models, and 3) system specific domain knowledge. Heuristics Results of this research suggest the importance of moving away from a purely component oriented analysis to what the researchers term a functional decomposition when designing troubleshooting skills instruction. While analysis and instruction on the components is necessary, it is not sufficient. Analysis and instruction should also focus on the functional processes, including likely causes of potential problems and paths to solutions, in order for learners to know what to do when troubleshooting. Further, the results indicate that training in system independent troubleshooting skills can further augment the troubleshooting skills instruction. IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Critique The presented research is important for two reasons. The research suggests a positive influence of CTA on outcomes in troubleshooting training. By revising the instruction to focus on a functional understanding of the system from information gleaned in the CTA, the instruction appears to have been significantly improved. In addition, the findings suggest a positive impact from teaching system independent troubleshooting skills. Also, the paper is valuable for the information provided about the evolution of the authors’ CTA process. This information will be helpful to future designers and researchers. Unfortunately, the written presentation of this paper is horribly disjointed. It is doubtful that most readers will devote the time necessary to weave a coherent narrative out of the broken threads of theory, prior research, CTA processes, instructional design considerations, research methodologies, and conclusions. There is a wealth of information included in the paper, but unfortunately the reader must devote an unnecessary amount of effort to piece it all together.

IDT873 Maddrell Behavioral Abstract 2 - Upload a Document to Scribd

Read this document on Scribd: IDT873 Maddrell Behavioral Abstract 2

Behavioral Strategy Abstract Running head: BEHAVIORAL STRATEGY ABSTRACT 1 Behavioral Strategy Abstract: Self-Pacing Versus Instructor-Pacing Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Morrison September 8, 2008 Behavioral Strategy Abstract Self-Pacing Versus Instructor-Pacing 2 Overview Morris, Surber and Bijou (1978) report on research conducted to compare achievement, student satisfaction, and retention between self-paced and instructor-paced personalized systems of instruction (PSI). While noting that one of the key features of PSIs is the ability for learners to self-pace, the authors cite prior research that suggests students who are allowed to self-pace may be more likely to procrastinate or withdraw from the course entirely. These finding have led some to incorporate instructor-paced schedules into the PSI. However, what had been less clear in prior research is the impact of self-pacing on learner achievement (both short term and longer term following course completion) and learner satisfaction with the learning experience. Research Purpose. The purpose of the reported study is to compare progress rates, withdrawal rates, achievement, satisfaction, and longer term retention between learners completing selfpaced or instructor-paced PSI. The researchers set out to extend prior research by focusing on the effect of pacing on these measures. Methodology. All 149 students enrolled in an introductory child development class were randomly assigned to either self-paced (S-P) or instructor-paced (I-P) PSI. The syllabi, course materials, and assessments were identical for both groups. Within each of the 15 units of the PSI, all learners were required to either achieve 90% mastery within a 10-item short-answer essay quiz and oral examination at a testing center or take a make-up quiz until 90% mastery was achieved. Learners in the S-P condition were able to complete all 15 required units within the PSI at their own pace within the semester. Semester grades for the S-P group were based solely on the number of units mastered. In contrast, the I-P students were subject to a grading scheme that could result in a one letter grade drop if the student did not complete at least one unit of material each week. To evaluate and compare pacing, the semester was divided into five 15 day increments. For the purpose of measuring student achievement, a 53 item multiple-choice pre and post-test based on a few items from each unit was administered to all learners. In addition, nine months after the semester, students were asked to return (with compensation) for a follow-up test. They were all informed that the pre and post-tests would not impact final grades. A course evaluation questionnaire addressed student satisfaction with the course. Conclusions. As shown in prior research, the completion rates between the S-P and I-P groups were not the same. I-P learners progressed through the material at a more even rate throughout the semester, while S-P learners completed fewer units in the initial time periods as compared to the latter time periods. However, there were no statistically significant differences in course withdrawal rates, final grade distributions, course evaluations, or achievement measures between the two groups. Yet, there were statistically significant differences between the number of repeated quizzes during the semester and the follow up retention scores. S-P students repeated 4.1% of their quizzes, while I-P students repeated 7.2% of theirs. While the S-P learners’ delayed rate of complet