Evaluation planning and learning assessment for STEM program

taf-logo-250px-not-transparent

 

Technology Access Foundation (TAF) is a nonprofit leader in science, technology, engineering and math (STEM) education. They address three problems that keep students of color away from STEM and essentially disqualify them from competing and thriving in today’s society:

 

  1. Low expectations
  2. A shortage of role models in STEM
  3. Lack of access to a quality, STEM- focused education

Evaluation Design Challenge:

TAF leaders wanted to take the evaluation of their TechStart program to the next level. The TechStart project-based curriculum is developed by STEM professionals and curriculum design specialists and is aligned with the state and local district academic standards. The key elements to all of TechStart curriculum units include opportunities for: an inquiry based approach to problem solving, integration of 21st Century Skills, critical thinking, hands-on learning, student voice within the exercise, peer collaboration, and presentation of the project. Where possible, projects address issues in the students’ communities.

Solution #1:

Data2insight provided expertise in evaluation focusing and planning.

Data2insight facilitated a one-day evaluation focus session for stakeholders. Attendees included TechStart staff, teachers, parents, and a student. Participants identified short, intermediate, and long-term program outcomes in an impact mapping activity. They formulated and prioritized questions that, when answered, provided evidence of changes TechStart catalyzed. Data2insight formalized the impact map and provided a detailed evaluation plan (including estimated cost) to answer top evaluation questions.

Results:

TechStart used the evaluation plan to inform decisions about what evaluation to conduct in the upcoming school year.

Learning Assessment Challenge:

Based on data2insight’s evaluation plan, TechStart program staff decided to conduct an evaluation to answer the question for the 2010-11 academic year:To what extent does TechStart impact student knowledge and performance in math, science, computer science, and engineering?

Solution #2:

Data2insight provided expertise in learning assessment development and administration.

Data2insight guided the TechStart staff and teachers in the design of pretest-posttest assessment blueprints for math, science, computer science, and engineering geared for grades 3-5 and 6-8. Based on the assessment blueprint, data2insight compiled customized and standardized test content for 8 assessments (1 assessment per topic per grade Additionally, data2insight provided technical assistance to the program manager, who built and administered the assessments in Moodle.

Results:

Preliminary findings from the pretest indicated that students who had previously participated in TechStart classes scored higher on the pretest in all content areas.

Data from these assessments provide TechStart staff  with insight into what students learn from the after-school program’s problem-based curriculum. Findings will establish a baseline for future assessment and evaluation.