A statewide global health alliance that works to facilitate economic development collaborations and to engage, educate, and inspire the general public about global health supported the design and development of an interdisciplinary chemistry, algebra, and United States history curriculum organized around global health diseases. Twelve public high school teachers designed and implemented the curriculum in four high schools (two rural and two urban) across the state from 2009 to 2011.
Data2insight designed and implemented an appropriate evaluation designed to answer the key evaluation questions.
The written evaluation report was used to inform program funders and program leaders future grant proposals to continue the development of interdisciplinary education. Lessons learned were applied to program design and development. Program leaders also are in the process of publishing the evaluation findings, which showed that program participants had statistically significant learning gains compared to their peers in all three subjects.
The assessments designed by teachers were used both to evaluate what students learned from the curriculum as a whole and individual lessons. Furthermore, teacher-designed assessments were embedded in the final version of the curriculum now being used state-wide by Math Engineering Science Achievement (MESA) teachers.
Western Institutional Review Board reviewed the evaluation methodologies for pretest-posttest and interim assessment secondary data analysis and approved an IRB review exemption request for that analysis. WIRB also reviewed and approved the process and impact evaluation study that included interviews with pilot school principals and counselors as well as a statewide global health alliance teacher focus group. For this study, consistent with the study protocol, all respondents voluntarily agreed to participate and their signed consent forms are on file. While results from interviews and focus groups are aggregated and study participants are not mentioned by name, confidentiality was not guaranteed due the small sample size (four principals, 10 counselors, and 10 teachers). Participants agreed to study results being published. Furthermore, summaries of the interview and focus group data have been shared with the participants for their review.
understanding following selected lessons in each of the three content areas administered during the academic year.
All statewide global health alliance teachers administered pen and paper interim assessments in their classrooms immediately following instruction of specified course lessons. One chemistry, math and history interim assessment was administered across all statewide global health alliance classrooms as a common interim assessment. These assessments were designed by teachers to measure student understanding of key learning objectives in the lessons being reviewed. Teachers scored these tests and included the scores as part of the student’s quiz grade.
Teachers mailed the completed assessments to a supervisory scientific agency where the tests were anonymized. A statewide global health alliance program staff member, masked to student identity, scored de-identified interim assessments using a scoring guide developed by statewide global health alliance teachers. The staff member then entered the test scores into a spreadsheet for analysis.
The statewide global health alliance staff administered the Site Exploration survey to students after the students completed of a half-day field trip including mini job shadows, conversations with the supervisory scientific agency scientists, tours of the agency’s research facilities and a hands-on lab investigation in the agency’s lab. This field trip took place at the supervisory agency’s lab in Seattle, WA. The pen and paper survey was designed to meet the following objectives:
a) Assess student perception of the relationship between the statewide global health alliance course work and the Site Exploration activities
b) Assess student understanding of the definition of global health
c) Assess student interest in global health careers
Descriptive statistics were used to summarize student sample (N=242) demographics and answers to questions about course preparation and student perceptions about math, science, history and global health.
Student responses to the three open-ended survey questions were analyzed qualitatively. These questions asked students to define global health, share what is attractive to them about global health careers, and share their thoughts about Site Exploration activities. A coding scheme was developed based on the project objectives. All narrative responses were coded, using the constant comparative method, and emergent themes were identified.
1) Teacher focus group and principal interviews
The purpose of the focus group was to gain retrospective feedback from statewide global health alliance teachers about the statewide global health alliance program. Focus group discussion centered broadly on the following topics:
a) Reflections about experiences during their 3 years of participation in the statewide global health alliance program.
b) Which aspects of the statewide global health alliance curriculum were most effective.
c) Which aspects of the statewide global health alliance program were least effective.
d) Recommendations for future curriculum, instructional and professional development strategies and future teacher training.
2) Counselor interviews
The purpose of the counselor interviews was to capture documented or perceived changes in statewide global health alliance student college readiness and/or global health awareness and interest. The counselor interviews were designed to provide the following data:
a) Metrics of college readiness for Year 2 statewide global health alliance students included the following:
b) Qualitative data about Year 2 and 3 statewide global health alliance students’ college readiness
c) Qualitative data about Year 2 and 3 statewide global health alliance students’ interest in global health topics and/or careers
Pilot school principal, counselor interviews and teacher focus group data analysis
Principal, counselor, and teacher responses to interview and focus group questions were analyzed qualitatively. A coding scheme was developed based on the project objectives. All narrative responses were coded, using the constant comparative method, and emergent themes were identified. Some responses fell into multiple theme categories, and were therefore counted in multiple categories. Common and related questions across groups were compared to determine convergence and divergence of observations and recommendations.
Data2insight provided learning assessment training for teachers
We guided teachers in the design of pretest-posttest, and interim lesson assessments for the curriculum. These assessments provided both assessmentfor learning (formative) and assessment of learning (summative).
In addition, data2insight guided teacher development of innovative formative assessments for each of the 41 lessons using the Understanding by Design curriculum framework.
Value of in-service day training for teachers
Teachers cited getting in-service training day schedules synchronized for the entire school year and receiving feedback on lessons from other participating teachers most frequently as the most valuable aspects of in-service day training, which focused on development of learning assessment knowledge and skills in Year 3. Also mentioned as valuable by teachers were writing curriculum and assessments, the formative assessment tools provided during training, lesson feedback, learning practical strategies for the classroom and taking a focused look at individual lessons and assessments.
Representative teacher comment on Year 3 teacher workshop:
It was really helpful to have a focus in this last year. Every time we talked about a lesson, we were talking about the assessment, which made us look at the lesson in a much more rigorous way, instead of just saying, “Oh, this is really fun,” we’re actually able to say, “Oh, well, this is actually getting them to learn it. This isn’t getting them to learn it.”