Effects of I Can Learn^® on Math Achievement in Gwinnett County Middle School

Kyle Scafide JRL Enterprises, Inc.

Abstract

Based on a random sample of 620 8th grade students from Lilburn Middle School in Gwinnett County, Georgia, the researcher compared student performance of I CAN Learn^® and traditionally-taught students on the Georgia Criterion-Referenced Competency Test (GCRCT), math scale. Students in the treatment group (I CAN Learn^® Mathematics) scored significantly higher on the math scale. The effect size was .16.

Intervention

The I Can Learn^® education system is a computerized mathematics software solution, specializing in Algebra and pre-Algebra. Aligned to state and national mathematics standards, it presents and assesses content in an interactive learning environment. The teacher in an I Can Learn^® classroom is the facilitator of instruction, providing individual, small-group, or whole-class instruction as needed. The "Classroom Explorer" is a class management/gradebook system that provides real-time feedback on student progress. Teachers use the Classroom Explorer to determine which students need additional assistance with any given concept.

The I Can Learn^® system is intended to be used as the primary system of instructional delivery-- not as a supplemental or resource tool. The I Can Learn^® system consists of 303 lessons from basic mathematics to advanced Algebra concepts. With the help of education service specialists, teachers choose the lessons that align to their local curriculum needs. Each lesson includes a pretest, presentation in a real-world context, guided practice, individual practice, and a posttest. Problem-solving skills also are strengthened with challenging "journal" problems that require writing in math while solving multi-step application problems. Complete information about the system is available at www.icanlearn.com.

Sample

Gwinnett County, Georgia has a population of about 673,000 and annual per capita income of $25,000. About 73 percent of county residents are White. (http://quickfacts.census.gov/qfd/states/13/13135.html). Lilburn Middle School in Gwinnett County serves about 2,100 students in grades 6 through 8. Approximately 70 percent of Lilburn students qualify for free or reduced-price school lunch and approximately 90 percent are non-white. In the 2002-2003 school year, 75 percent of Lilburn students met the state standard for mathematics on the Georgia Criterion-Referenced Competency Test (http://www.gwinnett.k12.ga.us/gcps-mainweb01.nsf/FileAttachments/784887D7D956485385256BCE004FEE8F/$file/LilburnMS.pdf).

In October 2002, I Can Learn^® courseware was introduced into 8th grade math classes at Lilburn. The 2003-2004 school year was the first full year of implementation. This evaluation is based on eighth grade students in the 2003-2004 school year who were assigned to either a traditionally-taught or I Can Learn^® classroom based on a random assignment process. The Pearson Digital Learning SASI mass scheduling software was used for student assignment, but according to the principal, special education students were separately assigned by random draw. This ensured that special education students were also represented in both the I CAN Learn^® and traditional classes.

The sample included in this study consists of all 620 8th grade students at Lilburn Middle School who took the GCRCT in May 2004. One teacher taught in I CAN Learn^® classes. Although natural student groups were used, this may also be considered an advantage in terms of external validity in that the setting was not contrived.

Tables 1, 2, and 3 describe the demographics of the treatment (I Can Learn^®) and control (traditionally-taught) groups.

Table 1
Student Gender by Class Type

Table 2
Ethnicity by Class Type

Table 3
Lunch Status by Class Type

The Chi squares for ethnicity and lunch status were both significant at the p <.05 level, indicating that these demographic variables differed in the two groups. The I Can Learn^® classes had significantly greater proportions of African American students and students qualifying for free or reduced-price lunch. Because these demographics differed by group, separate analyses are presented by ethnicity and lunch status.

Dependent Measure: Georgia Criterion Referenced Competency Test (GCRCT)

The GCRCT math test was implemented in Georgia in 2000 in grades 4, 6, and 8. Grades 1, 2, 3, 5, and 7 were added in 2002. The test was designed to assess how well students acquire the skills and knowledge described in the Georgia standards, the Quality Core Curriculum. The math test consists of 60 items in seven sub-parts-- Number Sense and Numeration, Geometry and Measurement, Patterns and Relationships/Algebra, Statistics and Probability, Computation and Estimation, and Problem Solving. The Georgia Department of Education offers claims of validity and reliability but does not offer specific technical information (http://www.doe.k12.ga.us/curriculum/testing/crct.asp). Scale scores for the math test range from 150 to 450. Scores that are at or above 350 indicate a level of performance that exceeds the standard for the state test; scores from 300 to 349 indicate a level of performance that meets the standard; and scores below 300 indicate a level of performance that does not meet the standard.

Implementation Fidelity

According to school staff and the I CAN Learn^® education representatives who visited the classroom weekly, implementation of the intervention was as intended. Teachers in the I CAN Learn^® classrooms received professional development in how to use the software and how to manage the instructional environment. They used the computer as the primary system of instructional delivery and relied on the Classroom Manager reporting system to help identify students who needed individual or small-group instruction on specific concepts.

Results

The GCRCT 2004 math scale score was compared for the I CAN Learn^® and traditionally-taught students. Means and standard deviations of the two groups were used to compute effect sizes (see Table 4).

Table 4
Descriptive Statistics and Effect Sizes

An independent samples t-test was used to determine if mean differences were statistically significant. The t-test results are presented in Table 5. The t for "equal variances not assumed" was used because the assumption of equality of variances was not met (see Levene’s test results in Table 5).

Table 5
t-tests of Mean Differences Between I CAN Learn^® and Traditional Classes on GCRCT Math Scale

As shown in Table 5, the mean difference was statistically significant, indicating that the I CAN Learn^® students outperformed traditionally-taught students.

Results by Student Sub-groups

Data were disaggregated by gender, ethnicity, and lunch status to determine the effects of the treatment on important student subgroups.

>Gender. Although males scored slightly higher than females in both groups, the ANOVA interaction between class type and gender was not statistically significant (F=1.192, p=.28). See Table 6 for means by class type.

Table 6
Mean Scale Scores by Gender and Class Type

Ethnicity. To determine whether student ethnicity and class type had an interactive impact on the GCRCT score, a one-way analysis of variance was computed. Ethnicity was coded as White or non-White only. The interaction term was not statistically significant (F=.12, p=.73) although the mean for non-white students in the I CAN Learn^® classes was higher than the mean for those students in traditional classes. See Table 7.

Table 7
Mean Scale Scores by Ethnicity and Class Type

Socioeconomic Status. Using free or reduced price school lunch as a proxy for socioeconomic status (SES), students' scores could be compared by SES subgroup. The interaction between lunch status and class type was not statistically significant (F=.88; p=.42).

Table 8
GCRCT Math Total Means by SES and Class Type

Conclusions

Given the findings based on this sample of 620 8th grade students at Lilburn Middle School, it must be concluded that the intervention, the I CAN Learn^® system, significantly improves student achievement in math as measured by the Georgia CRCT. Student gender, ethnicity, and socioeconomic status did not impact findings.

Although the effect size is small (.16), the difference is particularly meaningful when one considers that a passing score on the GCRCT is 300 or above. Of the I CAN Learn^® students, 92.9% scored above the criterion. Only 70.5% of traditionally-taught students scored above the criterion. A Chi-square test of differences is significant (Chi square = 18.6, p = .000). Thus, I CAN Learn^® students were significantly more likely to pass the math portion of the GCRCT.