An analysis by Boston-based researchers show the state MCAS pass rate
reported by the Department of Education may be exaggerated by as much as 20
percent.
On March 3, the state Department of Education, or DOE, announced that 90
percent of the class of 2003 had passed the Massachusetts Comprehensive
Assessment System, or MCAS, the test required for high school graduation.
Problem is, the pass rate was calculated using only the current crop of
seniors. When all students who have been part of the Massachusetts class of
2003 are included in the count, the MCAS pass rate is 70 percent, or 20
percent lower than DOE claims.
According to researchers, this method of calculation does not account for
attrition, that is, students who have dropped out, been retained in a grade,
transferred out of state or into private school, or simply disappeared from
the rolls. Some students who repeat grade 9 may be included in the pass rates
reported for the grades following, but many will not.
Because the MCAS pass rate reported for the class of 2003 does not consider
all the students who were enrolled in the class prior to the 12th grade, the
DOE has discounted 16,991 students - 22 percent of the original class - from
its calculation of pass rates.
"The DOE's statewide pass rates are dramatically inflated," said Anne
Wheelock, researcher with the Progress Through the Educational Pipeline
Project at Boston College. "The problem with the DOE report, in general, is
that it gives only the proportion of current seniors who have passed. This
method of calculating pass rates has inflated MCAS pass rates for the class of
2003, statewide, by race. It also inflates pass rates in many districts."
According to the report, in Cambridge this translates to a pass rate of 73
percent instead of 76 percent but in other high-need districts the difference
is much higher. In Somerville, for example, the pass rate drops from 94
percent to 64 percent. In Boston it slips from a DOE figure of 70 percent to a
"on-time pass rate" of 48 percent.
The report also indicates that not a single one of 19 high-need districts
show real pass rates over 80 percent, and nine show real pass rates of 50
percent or less. Other DOE rates also mask wide variation in district pass
rates as in the case of charter schools and wealthy districts.
When rates are recalculated to account for all members of the class of
2003, results show the variability that exists, even among districts with the
same DOE rate. For example, among 15 districts showing a DOE pass rate of 93
percent, pass rates that account for all students range from 88 percent to 55
percent.
Researchers suggest that dividing the number of students passing MCAS by
the number of students enrolled in the class in 9th grade will yield a more
accurate pass rate - what they call the "on-time pass rate" - for each class.
Specifically, the "on-time pass rate" for the class of 2003 would be based on
those students who enrolled in ninth grade in October 1999, and passed MCAS in
time to graduate with their class in spring of 2003.
DOE spokespersons say that the report is baseless and does not count.
"We disagree with the whole premise of the report because it makes
assumptions that if those students stayed in class, they would fail," said
Heidi Perlman, media director, DOE. "It's a national phenomenon that every
student who starts in grade 9 does not necessarily graduate with that class."
In fact, the 1,300 students from the class of 2003 were moved ahead and
they graduated with the class of 2002, she added. The dropout rate in 2003 has
also gone down, she said.
As the debate continues as to which members of the senior class should be
counted, or whether students who left school but are in GED programs should be
included, the larger problem is that assessing pass rates based on 12th grade
enrollments is misleading from the start, Wheelock said.
Walt Haney and George Madaus from the National Board on Educational Testing
and Public Policy, Boston College who worked on this report believe that
district pass rates are not the only trouble; results by race are also
similarly distorted.
For instance, the DOE reports that 3,748 out of 4,984 African American
seniors have passed MCAS, resulting in a pass rate of 75 percent. But 7,003
African American students were enrolled in Massachusetts' original class of
2003 in 1999 and the pass rate based on grade nine enrollment is then 54
percent.
Similarly, the pass rate for Latino students drops from 70 to 40 percent
and for Asians from 90 to 79 percent.
"The failure to account for all students who were originally with the class
of 2003 results in MCAS pass rates that are distorted statewide, by race, and
for each district," said Haney.
Researchers believe that MCAS pass rates, both for the class of 2003 and
subsequent classes, should be restated based on ninth grade, or even eighth
grade, but not 12th grade, enrollment. Regardless of the reasons for student
attrition, all original members of the class of 2003 should be included in the
count.
"Calculating MCAS pass rates in a way that includes all students, rather
than 12th graders only, would be more consistent with other indicators
reported for each class, including the graduation rate required by federal No
Child Left Behind legislation and reported for each class," said Madaus. "In
addition, pass rates calculated to include all students would be more useful
to policy-makers."
According to Perlman their study "makes a meaningless point...We are not
changing anything. "
