DID IT WORK? - ANALYSIS OF WAYS TO MEASURE THE IMPACT OF AN AFTERSCHOOL ROBOTICS OUTREACH PROGRAM

摘要

Nationwide, afterschool programs exist to offer children educational enrichment. Funding organizations, however, want to assure the money invested has the desired impact. This reasonable desire for results leads to the common need for effective and appropriate assessment tools. This paper explores the challenges faced in revising an assessment program for the Austin Children's Museum, which has an afterschool, outreach robotics program for third to fifth grade students from low income schools. This robotics program features LEGO MINDSTORMS NXT to facilitate the students' robotics education. The goals of the program include: enhanced academic skills such as computer programming and visual discrimination; increased interest and motivation for future experiences and positive attitudes towards mathematics, science, engineering and technology; and greater awareness of technology/teamwork processes such as brainstorming, planning, teamwork and troubleshooting. This paper presents the challenges and adaptive efforts to effectively assess the program's impact in achieving its goals. The assessment is brief due to the short program (eight sessions), the age of the students, and the voluntary nature of the program. Due to these young students' written comprehension level, traditional assessment techniques were not appropriate. These challenges led to a revision of the assessment at three stages. The resulting assessment program includes short pre-and postprogram written quizzes of the students' comprehension of content areas, pre-and post-program verbal interviews of the students, and in-session records of students' abilities to demonstrate understanding of the content discussed. The analysis in this paper focuses on revision of the assessment to adapt it to effectively measure changes in students' skills or attitudes and why the assessment was revised at three stages in the program. The analysis also discusses how each of the assessment techniques impacted the program and provides insight in this area that many programs find challenging.