I CAN Learn® Results in Dallas, Texas
9th Grade 2003-2004

Jeffrey Oescher, Ph.D. and Peggy C. Kirby, Ph.D.

The I CAN Learn® Education System

The Interactive Computer Aided Natural Learning system (I CAN Learn®) was created in 1995 as a complete education software system to deliver standards-based algebra and pre-algebra courses to middle and high school students. It is designed as a stand-alone system to be used as the primary mode of instruction, not solely as a remedial or enrichment tool. Students using the I CAN Learn® curriculum work at their own pace in a classroom with a one-to-one student to computer ratio. Teachers facilitate instruction by offering individual and small-group instruction as needed. The Classroom Explorer management tool enables teachers to track student progress in real time and identify areas of concern on an individual student basis.

Theoretical and empirical support for the I CAN Learn® system comes primarily from Madeline Hunter's work on direct instruction. Hunter (1993, 1995) concluded that effective teachers consistently use eight elements to present lessons. Her research showed these elements to be effective regardless of the teacher's style, student background, grade level, or subject. The eight elements of effective teaching, according to Hunter are:

1) Anticipatory Set - A short activity or prompt to focus the students' attention.
2) Purpose - Explicit statement of the reason for learning the day's lesson and how learning will be demonstrated.
3) Input - The skills and concepts to be taught.
4) Modeling - Teacher demonstration of what is to be learned.
5) Guided Practice - Teacher-led student practice of the skills and concepts.
6) Checking For Understanding - Mechanism for verifying that the student is acquiring skills and concepts.
7) Independent Practice - Student work without guidance during the process but with feedback after completion.
8) Closure - A review or wrap-up of the lesson.

Each interactive I CAN Learn® lesson uses the Direct Instruction method and includes a pretest, review, lesson presentation, guided practice and post-test. Also included are cumulative reviews, "real world" applications, and cumulative tests to facilitate and assess retention. Every I CAN Learn® lesson was developed and written by experienced educators and incorporates national and state performance standards.

Dallas Independent School District

During the 2003-2004 school year, the Dallas Independent School District (DISD) consisted of 218 schools with a total of 161, 261 students. The majority of students (60.9%) were Hispanic; 31.4% were African American; and 6.3% were White. The budget per student was $8,800 (http://www.dallasisd.org/inside_disd/facts _stats/0304factsheet.pdf).

One of the 28 high schools implemented the I CAN Learn® curriculum in 2003-2004. This school was considerably different than the average Dallas school in terms of ethnicity. Eighty-nine percent of the students were African American, 9.3% Hispanic, and 1.3% White. About 55% of the students were classified as low in socioeconomic status (SES) based on free or reduced-price lunch eligibility (http://www.dallasisd.org/ ClientDoc/madison%2Epdf).

The DISD research and evaluation department provided achievement and demographic data for all middle schools in the district without student names or other identifying information.

Implementation

The I CAN Learn® (ICL) math curriculum was implemented in one Dallas public high school in April 2003. Ninety-nine 9th graders used the system for Algebra I during the 2003-2004 school year. Teachers received training in how to use the system in a 2-day workshop in May 2003. On average, students completed 83% of the targeted number of Algebra I lessons.

Measure of Math Achievement

The Texas Assessment of Knowledge and Skills (TAKS) mathematics exam, grade 9, was the dependent measure used to assess student achievement in math. The Texas Student Assessment Program Technical Digest gives evidence of content, construct, and criterion validity of the TAKS and its sub-tests. There are ongoing validation processes to ensure that TAKS is a valid and reliable measure of the Texas state curriculum (Texas Education Agency, 2003). The math test consists of 10 objectives and 52 items. Raw scores and scale scores were reported.

Design

A quasi-experimental design using propensity score matching was used to determine the effects of the I CAN Learn® math curriculum on 9th grade student achievement on the math subtest of TAKS. Because 9th graders in the high school used the I CAN Learn® system as their primary mode of instruction, a comparison sample had to be found from other high schools in the system. Propensity score matching was used to identify 99 students who were most similar to the I CAN Learn® students on key indicators.

Sample

All 9th graders in the system not using the I CAN Learn® math curriculum were eligible to be included in the control group. From these 9,975 students, the 99 with the closest propensity scores were selected as the control sample.

Propensity score matching is a statistical technique that finds the best match for every treatment case from a pool of possible comparison cases. Comparison cases are removed and not reconsidered for subsequent matches. The program used in this study was developed by Levesque and Painter for use with SPSS 11.5 (Painter, 2004). The match variables were chosen by the researcher for their known correlation to the criterion variable, math achievement. The six matching variables were gender, ethnicity, socioeconomic status (as determined by free or reduced lunch classification), special education classification, English language proficiency, and reading/language arts achievement (as determined by the reading/language arts scale score on the Spring 2004 TAKS exam).

The procedure yielded a dependent sample with exact matches on ethnicity and limited English proficiency (LEP) code. The treatment group consisted of one more student receiving special education services and one more student receiving free lunch than the control group. Although the two groups were equivalent in numbers of males and females, two of the pairs were cross-matched on gender. Exact matches on reading/English language arts scores were found for 97 of 99 pairs. In one pair, the ICL student had a higher score; in the other, the control student had a higher score. Tables 1-6 present the descriptive statistics for both groups.

Table l
Gender by Group

Gender
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
Female
50
50.5
50
50.5
Male
49
49.5
49
49.5
Total
99
100.0
99
100.0

Table 2
Ethnicity by Group

Ethnicity
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
Asian
1
1.0
1
1.0
African American
88
88.9
88
88.9
Hispanic
9
9.1
9
9.1
White
1
1.0
1
1.0
Total
99
100.0
99
100.0

Table 3
Economic Status by Group

Economic Status
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
Not free/reduced meal eligible
26
26.3
27
27.3
Reduced-price meal eligible
4
4.0
4
4.0
Free meal eligible
69
69.7
68
68.7
Total
99
100.0
99
100.0

Table 4
English Language Proficiency by Group

LEP Code
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
No
95
96.0
95
96.0
Yes
4
4.0
4
4.0
Total
99
100.0
99
100.0

Table 5
Special Education Classification by Group

Special Education
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
Not participating
96
97.0
97
98.0
Participating
3
3.0
2
2.0
Total
99
100.0
99
100.0

Table 6
TAKS Reading/English Language Arts Means by Group

N
Minimum
Maximum
Mean
Std. Deviation
I CAN Learn®
99
1401
2355
2099.01
166.06
Control
99
1407
2355
2095.37
165.33

Data Analysis

Given that students in the two groups were nearly identical on all demographic and non-math achievement, groups could be compared on the dependent variable using a paired samples t-test. The probability level was set at .05 (2-tailed).

Results

The I CAN Learn® math students scored significantly higher (t=4.3, p<.001) than traditionally-taught students. The mean difference on the TAKS was 143.5 scale score points (see Tables 7 and 8 and Figure 1).

Table 7
Mean Differences in TAKS Math Scale Scores for ICL and Traditionally-Taught Students

N
Minimum
Maximum
Mean
Std. Deviation
I CAN Learn®
99
1401
2785
2031.21
216.26
Control
99
1407
2465
1887.73
337.99

click to view Figure 1

Table 8
Paired Samples t-Test

Paired Differences
t
df
Sig. (2-tailed)
Mean
Std. Deviation
Std. Error Mean
95% Confidence Interval of the Difference
143.48
332.41
33.41
77.18
209.79
4.294
98
.000

Based on these results, the effect size is estimated at 0.43 . Additional analyses compared the individual objective scores, the percentage of students meeting the pass standard on the TAKS, and gender differences in scores between groups.

ICL students scored significantly higher on all 10 TAKS math objectives (see Table 9).

Table 9
TAKS Objective Scores by Group
Paired Samples Statistics

Objective
 Group
Mean
N
Std. Deviation
Std. Error Mean

Paired

t
Sign

Objective 1: Functional Relationships
I CAN Learn
 2.80
99
1.18
.12
4.363
.000
Traditional
 2.07
99
1.48
.15

Objective 2: Properties and Attributes of Functions
I CAN Learn
 2.97
99
1.39
.14
4.027
.000
Traditional
 2.12
99
1.66
.17

Objective 3: Linear Functions
I CAN Learn
 2.62
99
1.40
.14

4.202
.000
Traditional
 1.94
99
1.33
.13

Objective 4: Linear Equations and Inequalities
I CAN Learn
 2.80
99
1.48
.15

1.856

.067
Traditional
 2.45
99
1.51
.15

Objective 5: Quadratic and Other Nonlinear Functions
I CAN Learn
 2.27
99
1.01
.10

3.053

.003
Traditional
 1.84
99
1.13
.11

Objective 6: Geometric Relationships and Spatial Reasoning
I CAN Learn
 2.24
99
1.06
.11

3.554

.001
Traditional
 1.70
99
1.15
.12

Objective 7: 2-D and 3-D Representations
I CAN Learn
 2.03
99
1.24
.12

3.255

.002
Traditional
 1.51
99
1.24
.12

Objective 8: Measurement
I CAN Learn
 2.57
99
1.26
.13

.954

.002
Traditional
 2.39
99
1.50
.15

Objective 9: Percents, Proportions, Probability, and Statistics
I CAN Learn
 2.88
99
1.39
.14

3.318

.001
Traditional
 2.27
99
1.53
.15

Objective 10: Mathematical Processes and Tools
I CAN Learn
 4.18
99
2.07
.21
1.607
.111
Traditional
 3.72
99
2.25
.23
Chi-square analysis shows that significantly more ICL students met the TAKS standard in 2004 (see Table 10).

Table 10
Proportion of Students Meeting TAKS Math Standard by Group

Met Standard in Math
I CAN Learn®
Control
Frequency
Percent
Frequency
Percent
No
54
54.5
70
70.7
Yes
45
45.5
29
29.3
Total
99
100.0
99
100.0
Chi square = 5.524; p < .05

No significant differences were found between TAKS math scores of males and females in either the ICL or traditional group. Comparisons by ethnic groups were not computed because the vast majority of students were African American.

Conclusions

Students in this Dallas high school who used the I CAN Learn® math curriculum performed better than their matched counterparts in other Dallas high schools who did not use the I CAN Learn® system on all 10 TAKS math objectives and on the math total scale. Significantly more I CAN Learn® math students met the TAKS standard for math in the spring of 2004. The I CAN Learn® math system was equally effective for males and females. Because the majority of students in the study were African American, further research should be conducted to determine whether these positive effects of I CAN Learn® math hold for students of other ethnic groups.

References

Dallas Independent School District. (2004). Web Site. [Retrieved November 6, 2004 from: http://www.dallasisd.org/inside_disd/facts_stats/0304factsheet.pdf]

Hunter, M. C. (1993). Enhancing teaching. Upper Saddle River, NJ: Prentice Hall

Hunter, M. C. (1995). Teach more - faster. Thousand Oaks, CA: Corwin Press.

Painter, J. S. SPSS Propensity Matching Program Description. [Available at http://sswnt7.sowo.unc.edu/VRC/Lectures/ ]

Texas Education Agency (2003). Texas Student Assessment Program Technical Digest 2002-2003. A Collaborative Effort of the Texas Education Agency, Pearson Educational Measurement, Harcourt Educational Measurement, and BETA, Inc. [Available from http://www.tea.state.tx.us/student.assessment/ taks/booklets/math/g9.pdf]