A Collaborative Data Driven Teaching Experiment

The below case study details how two teachers collaboratively implemented a series of common assessments and analyzed data to alter instruction with the stated goal of improving student achievement in a public middle school.

Assessment Methodologies

Teacher A and Teacher B decided to collaborate and analyze student achievement data based upon pre-test, formative and summative common assessments for one unit of study.  The teachers agreed to focus these common assessments towards two student performance standards:

1. Student understanding of US History content
2. Reading comprehension of primary and secondary source documents

Before any instruction was provided in the US History classes, both teachers gave students the same pre-test. The use of Einstruction's CPS clicker response pads expedited the grading time for these assessments.  This pre-test clicker data was disaggregated by two standards, those questions that were content-specific or questions assessing reading comprehension skill.

The ability to quickly use student achievement data was also improved by exporting CPS clicker data directly into an excel spreadsheet (CSV) format.  This data was then uploaded onto a google doc to enable real time collaboration and analysis between the teachers.

Pre-test results

Comparing the pre-test performance of students between teacher A and teacher B yields a few differences statistically however,  analyzing the combined pre-test performance demonstrates a larger sample size for later post-test analysis with the goal of determining effect size.  Not surprisingly, the teachers learned that students knew very little about the upcoming content of the unit with the combined average score on all content specific questions for all students was 44%.

The data also showed all students assessed using the pre-test had an average total score of 47%.  On average, students did marginally better on the reading skills questions, 57%.  One interesting statistic to note was the large combined standard deviation of 15 points from the average score for all students on the reading skill questions.

This is an interesting statistic to note as it indicates that we have a wide range of student reading ability in our classes.  See the pre-test student achievement data summary below from google docs.

Formative Assessment of Reading Standard Proficiency

Next both teachers provided a formative common assessment to all students to gauge change upon only the reading standard.  It is important to note that the formative achievement data is based upon a very small sample size of questions and this potentially skewed the results below.  

Teacher A: 87%

Teacher B: 88%

Summative Data Results

Following the formative assessment results teachers A and B decided to use different teaching  methodologies to address student reading comprehension.  Teacher A partnered students based upon proficiency levels proven from pre-test results and formative results.  Teacher B focused upon active reading and thematic reading skills and introduced and reviewed text reading strategies.

After altering instruction following the pre and formative assessment results a summative assessment was provided and an effect size was calculated based upon the formula below:

Teacher A:

Sample: 53

Mean: 88%

Std Dev: 11.77

Mean reading standard: 81%

Mean content standard: 91%

Effect Size = 1.681

Teacher B:

Sample: 38 students

Mean: 90.7%

Std Dev: 8.118

Mean reading standard: 83%

Mean content standard: 93%

Effect size = 1.513 

Analysis

Comparing the student achievement data throughout the assessment cycle of pre/formative/summative common assessments it is clear the student achievement increased.  Students gains were greatest on content standards largely due to a lack of previous knowledge.  Student achievement results on reading standard also increased but not at a similar rate.  Comparing the standard deviation on the assessment cycle both teachers significantly narrowed the variability of scores while increasing performance. This data indicates that both teachers were able to raise the achievement for all students and narrow the gap between proficiency and non-proficient students.  In addition, the effect size data clearly shows the teaching methods used, while divergent, were both highly effective instructional strategies.  It is difficult to isolate the causality in this case study and it is important to stress the extremely small sample size.