|
COMMONWEALTH OF MASSACHUSETTS
SUFFOLK, ss.
SUPERIOR COURT DEPARTMENT
OF THE TRIAL COURT
CIVIL ACTION NO. 03-0071
STUDENT 1, A Minor by His Aunt and Next Friend, )
et al.,
)
Plaintiffs )
)
v. )
)
MASSACHUSETTS BOARD OF EDUCATION, )
et al.,
)
Defendants )
)
MEMORANDUM OF
AMICUS CURIAE IN SUPPORT
OF PLAINTIFF’S
MOTION FOR A PRELIMINARY INJUNCTION
CONCLUSTION …………………………………………………………….. 44
EXHIBIT A: Affidavit of Howard Fain in Support of
Amicus Memorandum
EXHIBIT B:
Dr. Eugene Gallagher, An Analysis of Problems on Two 10th
Grade MCAS Math Tests, Department of Environmental and Ocean Sciences, UMASS/Boston
(Sept. 29, 2001)
EXHIBIT C: Affidavit of Deborah Meier in Support of
Amicus Memorandum And Plaintiff’s Motion for a Preliminary Injunction.
INTRODUCTION
The Massachusetts
Coalition for Authentic Reform in Education (“MassCARE”), the statewide network
of more than 10,000 public school families, the National Center for Fair and
Open Testing (“FairTest”), the MassParents group, and the Center for
Collaborative Education, (“CCE”) which includes experts on student assessment
policies, submit this memorandum in support of the Plaintiffs’ Complaint and
Motion for Preliminary Injunction. Amici file this memorandum respectfully
refer the court to their statement of interest in their Motion for Leave to
Participate as Amicus Curiae filed March 18, 2003.
The imposition of the MCAS
tests by the Silber/Peyser Board of Education (“BOE”) represented a major
setback to education reform in Massachusetts. The use of a single standardized
pen-and-paper test as a graduation requirement is counter to modern research on
learning, teaching and authentic assessment of educational achievement. The
serious damage to students, classrooms and schools caused by the MCAS graduation
requirement was recognized early in the process by parents, teachers and
educators.
In response to
the seriousness of the problems, major Massachusetts organizations representing
individuals knowledgeable about the classroom environment publicly stated their
opposition and criticism of use of the standardized MCAS tests as a graduation
requirement in contradiction to the variety of assessments specified by the 1993
Education Reform Act (“Act”). These include the:
-Massachusetts Teachers Association, representing more
than 90,000 teachers;
-The Massachusetts Associations of School Committees, representing school
committees across the Commonwealth;
-MassCARE, representing more than 10,000 public school families across the
Commonwealth;
-The Alliance for High Standards not High Stakes, representing 50 education,
civil rights and other organizations;
-Citizens for Public Schools;
-New England Association for College Admission Counseling (NEACAC) representing
more than 3000 counselors;
-The Mass. State AFL- CIO, representing the interests of their working families.
-The Latino Parents Association.
Massachusetts led the
nation in the establishment of a publicly financed high school system to provide
a comprehensive education to middle class children. Insuring that access to
quality education was extended to children of lower income families was strongly
re-affirmed in the McDuffy decision. See McDuffy v. Sec’y of
the Executive Office of Educ., 415 Mass. 545 (1993). The 1993 Massachusetts
Education Reform Act (the “Act”) was the legislative mechanism for carrying out
the equitable access to education instruction of the McDuffy decision.
See G.L. c. 69 §§ 1 et seq. (1993).
Amici support the
plaintiffs’ argument that the 6.03 C.M.R. 30.03 regulation through which the
State Department of Education (“DOE”) claims to have implemented terms of the
Act is in violation of the Act, and is causing serious harm to students in
Massachusetts.
Unless DOE policies are
set aside or altered, more than 6,000 high school seniors, concentrated in lower
income communities of the state, will be denied their high school diplomas.
Department of Education, Class of 2003 Update (Mass.) (visited on Mar. 18, 2003)
<http://www.doe.mass.edu/mcas/results.html>.
This destructive result of the DOE testing policy on students represents a stark
violation of the McDuffy decision to increase access to quality education
among citizens of the Commonwealth.
Amici describe below the
significant harm to thousands of students, and their families, in the
Commonwealth as a result of the imposition of the high stakes MCAS tests,
including the grave harm to the plaintiffs, and other seniors, who will be
denied their high school diplomas. Amici document below multiple aspects of the
MCAS tests purchased from the Harcourt Corporation and imposed under 603 C.M.R.
30.03 that violate the provisions specified in Act and make it impossible that
such an assessment is fair and equitable as prescribed by McDuffy and in
the Act itself. In addition, Amici describe the failure of the DOE to provide
the necessary stable curriculum frameworks and frameworks implementation
necessary to meet the standard for instruction as called for in the Act.
I.
IRREPARABLE HARM TO PLAINTIFFS IF THEY ARE DENIED THEIR DIPLOMA DUE TO
THE 6.03 C.M.R. 30.03 MANDATED MCAS TESTS
Six
thousand Massachusetts high school seniors are about to be denied their high
school diplomas because they scored below the Needs Improvement cutoff of the
state sanctioned Harcourt MCAS Math or English Language Arts tests taken in 2001
or subsequently. See Department of Education, Reports (visited
March 2003) <http:www.doe.mass.edu>. These students are seniors who have
completed thirteen (13) years of schooling and have generally satisfied the
requirements of their school systems for high school graduation.
Many of these
students are bilingual, special education, minority, and vocational education
students. They include our children, our children’s friends and teammates, our
neighbor’s sons and daughters. They live in every region of Massachusetts and
represent the full diversity of our young adults. These students have stayed in
school and kept studying and working, despite the damage to their
self-confidence and self-esteem from being told that they have failed the MCAS
tests and retests.
In addition to
these
six thousand fifty-eight (6,058), sixteen thousand nine hundred ninety-one
(16,991)
other students who entered high school in the same freshman class
in 1999
will not be graduating with their class for various reasons. Auditi Guha,
Research says MCAS Pass Rate at 70, not 90 Percent, Cambridge Chronicle
(Mar. 12, 2003)
at 1, 4.
Among this very large group
(22% of the
original class)
are those who were discouraged by their
failing score
in the core 2001 MCAS tests, and the setting in place of what many considered an
arbitrary and discriminatory barrier to their graduation. See Department
of Education, Reports (visited March 2003) <http:www.doe.mass.edu>.
The prospect of
driving thousands of students a year out of public schools into the streets
represents a grave danger not only to the impacted youth, but also to
Massachusetts’ civil society and
its
economy.
The social and
human costs of this prospect are staggering. Denial of a high school diploma in
today’s economy is a death knell for a student’s future prospects. They will be
subject to this grave risk to their future regardless of courses taken,
classroom participation, grades, school citizenship, and special projects,
musical, dramatic or athletic contributions. Monty Neill, High Stakes, High
Risks: The Dangerous Consequences of High Stakes Testing, American School
Boards Journal, March 2003, at
18-21.
Absence of a diploma will
prevent these students from most forms of higher education. Denial of their
diploma will prevent them from receiving federal financial aid to schools they
could enter. Denial of their diploma will exclude them from many forms of
employment and even preclude enlistment in the Armed Services. For many
students, denial of their high school diploma will be the most damaging event in
their lives to date. The risk of failing to graduate high school is a source of
extreme anxiety to most of these students, who instead of planning for the next
step in their education or development must avoid stigmatization, unemployment,
and social marginalization. Dina Gerdeman, FEAR OF FAILURE: Students are
anxious, perplexed about the life-altering effects of flunking MCAS, The
Patriot Ledger (Feb. 22, 2003).
There can be little doubt
that in today’s highly technological and information-driven economy, absence of
a high school diploma deeply damages student’s prospects for future earnings and
opportunity. The "worth" of a high school diploma is clear. For example, U.S.
Census data show that those with a high school diploma, on average, earn $25,900
annually -- $7,000 more than high school dropouts. Laura Belsie, A Stronger
Link Between Degrees and Dollars, The Christian Science Monitor, July 18,
2002, at 1, 4.
A diploma also allows access to post-secondary education and the $45,400 average
annual salary that comes with college graduation. Id. Clearly, a
diploma is a ticket out of poverty, worth $1.2 million over a high school
graduate's lifetime. Id.
The denial of a diploma to these
students this year – and many thousands more next year, and the year after, will
be a social disaster in Massachusetts of great magnitude and impact not seen in
recent decades.
Amici argue below that
these students are being deprived of their diploma in violation of both the 1993
Education Reform Act, and contrary to the intent of the McDuffy
decision.
II.
THE MCAS TESTS DO NOT PROVIDE A
VALID ASSESSMENT OF STUDENT COMPETENCY AS REQUIRED BY THE 1993 EDUCATION REFORM
ACT
A.
THE SILBER/PEYSER BOE FAILED TO IMPLEMENT VALID AND AUTHENTIC STUDENT
ASSESSMENT AS MANDATED BY THE 1993 EDUCATION REFORM ACT
Following more than a
decade of litigation, the Massachusetts Supreme Judicial Court (SJC) ruled in
1993, in the landmark case of McDuffy v. Secretary of the Executive Office of
Education, that the system of financing public schooling in the Commonwealth
was unconstitutional and that the executive and legislative branches of
government were obligated to provide an education for all children of the
Commonwealth to “serve the interests of the children,” and “to prepare them to
participate as free citizens of a free State to meet the needs and interests of
a republican government.” Complaint, Student 1, et al. v. Driscoll et al.,
CA No. 02-3015-MAP, at 4 (Suffolk Superior Sept. 19, 2002); See 415 Mass.
545.
The Act specified that
Massachusetts’ public school students needed to acquire a common core of
knowledge as part of their education. G.L. c.69, § 1D. Chapter 69, Section 1D
of the Act required the development of “academic standards for the core subjects
of mathematics, science and technology, history and social science, English,
foreign language and the arts. The standards shall clearly set forth the
skills, competencies and knowledge expected to be possessed by all students at
the conclusion of individual grades or clusters of grades for grades
kindergarten through twelve.” G.L. c. 69, §1D.
Section 1D(i) goes on to
define the competency determination and refers to Section 1I for details of the
assessment instruments:
The competency
determination shall be based on the academic standards and curriculum frameworks
for tenth graders …and shall represent a determination that a particular student
has demonstrated mastery of a common core of skills, competencies and knowledge
in these areas, as measured by the assessment instruments described in section
one I. G.L. c. 69, § 1D(i).
The curriculum frameworks
referred to are described in Section 1E. G.L. c. 69, § 1E.
Chapter 69 Section 1I
describes the assessment process in some detail:
The system shall employ a
variety of assessment instruments on either a comprehensive or statistically
valid sampling basis. Such instruments shall be criterion referenced, assessing
whether students are meeting the academic standards described in this chapter.
As much as is practicable, especially in the case of students whose performance
is difficult to assess using conventional methods, such instruments shall
include consideration of work samples, projects and portfolios, and shall
facilitate authentic and direct gauges of student performance. G.L. c. 69, §
1I.
Furthermore,
The assessment instruments
shall be designed to avoid gender, cultural, ethnic or racial stereotypes and
shall recognize sensitivity to different learning styles and impediments to
learning. The system shall take into account on a nondiscriminatory basis the
cultural and language diversity of students in the Commonwealth and the
particular circumstances of students with special needs. Said system shall
comply with federal requirements for accommodating children with special needs.
All potential English proficient students from language groups in which programs
of transitional bilingual education are offered under chapter seventy-one A
shall also be allowed opportunities for assessment of their performance in the
language which best allows them to demonstrate educational achievement and
mastery. Id.
B.
MCAS USE AS SPECIFIED IN 603 C.M.R. 30.03 IS CONTRARY TO THE INTENT OF
THE EDUCATION REFORM ACT OF 1993; NO STANDARDIZED TESTS CAN PROVIDE A VALID AND
AUTHENTIC ASSESSMENT OF STUDENT LEARNING.
The legislators’ careful
description of the assessment requirement in the Act was derived from a large
and substantial body of experience and analysis of modes of assessing academic
skills and performance in children. H. Berlak, et al., Toward a New Science
of Educational Testing & Assessment, State University of New York Press
(1992); R.L. Linn, et al., Complex, Performance-based assessment:
Expectations and Validation Criteria, Educational Researcher 20 (1991) at
15-21; R. Mitchell, Testing for learning: How new approaches to evaluation
can improve American schools, New York: Free Press (1992); National Council
of Teachers of Mathematics (NCTM), Assessment standards for school
mathematics: Working Draft, (Oct. 1993); D. Wolf, et al., To use their
minds well: Investigating new forms of student assessment, G. Grant (Ed.)
(1991); American Educational Research Association, Review of Research in
Education, at 31-74.
A general conclusion that
has emerged is that no pen-and-pencil task can adequately assess the ability of
a student to perform complex intellectual skills and functions. Monty Neill,
The Dangers of Testing, Educational Leadership (Feb. 2003) at 43-46.
Multiple modes of assessment are required to assess skills in individuals whose
individual, cultural, family background differs. M.T. Nettles, & A.L. Nettles,
Equity and excellence in educational testing and assessment, Kluwer
Academic Publishers (1995). L. Valdez Pierce & J.M. O’Malley, Performance
and portfolio assessment for language minority students, DC: National
Clearinghouse for Bilingual Education (1992).
According to a board
member at the time, the BOE that participated in
the development of the Act had no intention of using a standardized test as the
determination of competency. MCAS Forum: Can We Afford MCAS in 2003?
Gardner Auditorium, (Feb. 11, 2003) (Statement of Frank Haydu, former BOE
member). The initial intent to follow the law was altered with the recomposition
of the state BOE and the ascendancy of John Silber to chairmanship of the BOE.
The Silber Board chose to ignore the letter of the law and previous work done by
the state DOE to create a system of multiple measures. The Silber/Peyser Board
reversed course, contracted out for a commercially developed standardized test,
which they termed the Massachusetts Comprehensive Assessment System (“MCAS”),
though such tests are neither comprehensive nor an “assessment system.”
Educators refer to such tests as a “high stakes” test, since student’s future
depends on the single score obtained.
At the time the
Massachusetts legislature passed the Act, a sound basis existed for using the
multiple measures required by the Act. Staff members of Amici FairTest
participated on the DOE’s advisory committee on testing, a group charged with
considering assessment options. While the state clearly intended to develop an
exam, it equally clearly did not intend to use that exam as the only form of
assessment for determining competency: and it did not intend an exam to be used
as a high-stakes hurdle that would deny students their diplomas. Rather,
multiple measures of different forms would all be used to provide evidence of
student progress, of school success, and for determining student proficiency or
competence. That is, the state intended to follow the letter and spirit of the
Act and to build on an existing body of knowledge to develop multiple measures.
(We also note that by the early 1990s, the state of Vermont was developing
portfolios, not just exams, to assess student learning.)
The definitive
national report on the use of high stakes testing was published in 1999 by the
National Research Council. National Research Council, High Stakes: Testing
For Tracking, Promotion, and Graduation, (Jay P. Heubert & Robert M. Hauser,
eds, National Academy Press, 1999). The report concluded that "no single test
score can be considered a definitive measure of a student's knowledge," at ES-2
and that "an educational decision that will have a major impact on a test taker
should not be made solely or automatically on the basis of a single test score."
at ES-2.
Rather than implementing
an authentic assessment system that could be used to raise standards, the Silber/Peyser
BOE on January 25, 2000, promulgated 603 C.M.R. 30.03. This imposed a “one size
fits all” testing system that creates barriers to educating public school
children, sharply discriminates against children from minority, special
education and vocational education, and is undermining public schools in
Massachusetts. The tests are not a determination of competency, but rather
arbitrary and capricious tests that often fail to assess the ability of students
to utilize their knowledge in society.
The Massachusetts Teachers
Association, The Boston Teacher’s Union, Citizens for Public Schools and more
than fifty (50) member organizations of the Alliance for High Standards, Not
High Stakes, all oppose the use of the MCAS tests as a graduation requirement.
For example, the resolution of The Massachusetts Association of School
Committees (“MASC”) states:
WHEREAS, a "high-stakes"
testing program will harm students by increasing high school drop-out rates,
discouraging other middle and high school students who perform at marginal
levels, and unnecessarily frustrating some younger children, especially those
with special needs, who are unable to succeed on the challenging MCAS tests, and
WHEREAS, there is
inadequate verification that MCAS results distinguish failing performance from
performance that needs improvement, and
WHEREAS, there is evidence
that states that have implemented education reform without high-stakes testing
are having better results in improving academic performance than states that
have adopted high-stakes testing,
NOW, THEREFORE, BE IT
RESOLVED, that MASC oppose the use of passing grade on the 10th grade Language
Arts and Math MCAS tests as a requirement for graduation from high school, and
be it
FURTHER RESOLVED that MASC
urge the Board of Education not to link MCAS results with the granting of high
school diplomas and MASC lobby for legislation that will prohibit the use of
MCAS results for that purpose. Massachusetts Association of School Committees,
Resolution, (Nov. 2000) (visited March 19, 2003)
http://www.masc.org/pos_frm.asp.
The DOE’s use of
standardized test results in isolation to bar a high school student from
promotion or graduation is contrary to professional standards regarding test
use. See Statement, American Education Research Association,
(visited Mar. 18, 2003) <http://www.aera.net/about/policy/stakes.htm>.
The American Educational Research Association (“AERA”) specifically states that
tests such as the MCAS should not be implemented:
In educational settings, a
decision or characterization that will have a major impact on a student should
not be made on the basis of a single test score. Other relevant information
should be taken into account if it will enhance the overall validity of the
decision. American Educational Research Association, American Psychological
Association, National Council on Measurement in Education. (1999) Washington,
DC: AERA at 146.
The AERA Position
Statement Concerning High-Stakes Testing in PreK-12 Education states:
“Decisions that affect individual students' life chances or educational
opportunities should not be made on the basis of test scores alone.” American
Educational Research Association, AERA Position Statement Concerning
High-Stakes Testing in PreK-12 Education,
(July 2000) <http://aera.net/about/policy/stakes.html.
The DOE denies
students the right to use spoken English language to express their English
Language Arts skills, thus assessing student competency in English Language arts
solely by a written exam. Amici document how this testing straitjacket violates
everything that educators have learned about assessing human language skills and
performance (Meyer and King affidavit). Amici assert that it is self-evident
that such an exam as the sole determinant cannot in any manner be considered a
valid, equitable or reliable method of assessment.
C.
THE MCAS MATH AND ENGLISH LANGUAGE ARTS (“ELA”) TESTS VIOLATE THE
SPECIFICATION FOR A VARIETY OF ASSESSMENTS AS CALLED FOR IN THE 1993 EDUCATION
REFORM ACT
The Act explicitly
requires the DOE to implement a “system of assessments” that would “employ a
variety of assessment instruments” including “consideration of work samples,
projects and portfolios.” G.L. c. 69, § 1D. Contrary to this requirement, the
state now seeks to prevent students from graduating from high school based on
whether or not their scores on just the Harcourt MCAS Math and ELA tests exceed
the arbitrary passing score of 220.
The DOE permits no other
evidence of student achievement, neither grades, awards, community service,
tutoring of other at-risk students, artistic talent, extracurricular
achievement, poetry written, articles published, or software programs written.
A student may be a leader in the community, a
three-letter athlete, the lead in the school play, mainstay of the jazz
ensemble, or an expert on Portuguese, Spanish, or Brazilian literature.
However, their lives will be derailed by the denial of their high school diploma
if they don’t score above the cutoffs on the MCAS math and ELA tests.
For example, Candido is a
Senior at the Burncoat High School in Worcester. See Affidavit of Howard
Fain, Exhibit 1. Candido has struggled to maintain good grades in high school,
bouncing back and forth between Special Education classes and regular classes.
Id. Candido successfully passed the ELA portion of the MCAS exam, but
has had difficulty with the math portion. Id. His guardians provide
extraordinary support, including taking advantage of the extra programs at the
school to prepare Candido for the MCAS math section as well as to working with
private tutors. Id. Unfortunately, Candido, even while achieving
tremendous improvement in his scores and excelling in his class work, has failed
to pass the exam after four attempts. Id.
Candido would like to give
back to his community someday by becoming a police officer. Id.
Tragically, Candido’s success story will be over before it begins because the
DOE insists on utilizing the MCAS exams to evaluate Candido’s knowledge and
skills in violation of the Act. A better solution would be to allow his
teachers, who observe and evaluate Candido’s knowledge and skills on a daily
basis, to assess his competency via a variety of assessment tools, including his
entire portfolio.
The Standards
for Educational and Psychological Testing, which have guided test developers and
users for decades, asserted in its 1999 edition, "[i]n educational settings,
a decision or a characterization that will have a major impact on a student
should not be made on the basis of a single test score." American Educational
Research Association, American Psychological Association, National Council on
Measurement in Education, Standards for Educational and Psychological Testing,
(1999) at 146. Even test publishers, including Harcourt Brace, CTB/McGraw-Hill,
Riverside and the Educational Testing Service, consistently warn against this
practice. Ulrich Boser, States Face Limited Choices In Assessment
Market, Education Week (Mar. 8, 2000). Relying
on a single standardized test to evaluate every student’s knowledge and skill
ignores individual ability and knowledge.
The claims of
the Defendants that the MCAS tests represent a “variety of assessments,” because
they have reading and writing questions in different formats, is specious. The
most profound aspect of human intelligence and the basis of human society is the
ability to communicate and interact through spoken language. The MCAS tests do
not allow students to use this most important and deepest expression of human
intelligence, cooperation, description, creativity and analysis.
Many
disciplines, including natural sciences, vocational education, music and arts,
have sophisticated assessment tools that test performance – actual mastery of
the skills and knowledge rather than just rote learning and test taking.
The National Forum on
Assessment produced Principles and Indicators for Student Assessment Systems,
which was signed by more than eighty (80)
national and regional education and civil rights organizations. FairTest,
Principles and Indicators for Student Assessment Systems, (1995). One of
the indicators reads, “Assessment systems allow students multiple ways to
demonstrate their learning.” Id. at 7. Another states, “Teachers,
schools, districts or states make reports on and decisions about individuals on
the basis of cumulative evidence of learning, using a variety of assessment
information, not on the basis of any single assessment.” Id. at 9.
D.
NEIGHBOR STATES USE A VARIETY OF DIRECT AND AUTHENTIC ASSESSMENTS RATHER
THAN JUST STANDARDIZED TESTS
The only New
England State that determines high school graduation using a test as a high
stakes hurdle is Massachusetts. V. Edwards, Quality Counts 2003,
Education Week (2003) at 88. Maine and Rhode Island explicitly use a variety of
assessment tools, as called for in the Act. Regulations of the Board of Regents
for Elementary and Secondary Education Regarding Public High Schools and
Ensuring Literacy for Students Entering High School, Final Version, (Jan.
2, 2003) at 1-11; FairTest, FairTest Examiner, (Spring 2002) at 12-13,
16. Tests are included, but they are only one component in the
assessment process. FairTest, supra at 13.
Decades of research on college admissions testing show that it is far
more sound (more valid and with smaller adverse impact on minorities and
females) to make decisions flexibly by using test scores, grades and other
information than to make decisions based on test scores alone. C. Rooney,
Test Scores do not Equal Merit, FairTest (1998). The recent exposure of
widespread errors in test scoring and reporting in the testing industry
indicates how unwise it is for the DOE to make important decisions mechanically
based on test scores in isolation. Henriques & Steinberg, None of the Above:
Right Answer, Wrong Score: Test Flaws Take Toll, N.Y. Times (May 20, 2001)
at A1; Steinberg & Henriques, None of the Above: Right Answer, Wrong Score:
Test Flaws Take Toll, N.Y. Times (May 21, 2001) at A1.
In response,
in the legislative session of 2001-2002, Senator Cynthia
Creem introduced legislation, consonant with the position of the AERA and
the National Forum on Assessment, to replace the current MCAS with a system that
would include a state exam and local assessments.
Senator Cynthia Creem, An Act to Require a Comprehensive Assessment System
for Students, Schools and Districts of 2002,
Sen. 257, (Mass. 2002).
Both Maine and Nebraska
have already developed state assessment systems that include local assessments,
which in turn typically include assessments used by teachers in their
classrooms. FairTest, supra at 13. Maine explicitly used the National
Forum’s Principles as a starting point for development of its state
system, which includes both an exam in three grades and local assessments.
Id. Maine’s system focuses on developing tasks and projects that teachers
can incorporate into their curriculum and score and report. Id. Items
from state “banks” of such tasks combined with ongoing classroom work can be
combined into portfolios that are also scored in light of state standards.
Id. See Also <http://www.state.me.us/education/lsalt/MAP.htm>.
Nebraska focuses more on
encouraging local districts to develop and use assessments that chart student
progress. FairTest Examiner, Nebraska and Maine Assessment Models,
FairTest (Spring 2002) 12-13, 16; See Also
http://www.nde.state.ne.us. It requires
districts to show that their assessments meet state standards for assessments;
their standards are highly similar to Maine’s and thus to the Principles.
Id. These states’ approaches complement a voluminous array of material
on using multiple forms of assessment. Id.
E.
THE
MCAS TEST VIOLATES THE 1993 EDUCATION REFORM ACT BY FAILING TO ALLOW
CONSIDERATION OF STUDENTS’ “SAMPLES, PROJECTS, OR PORTFOLIOS”
Under 603
C.M.R., students are not permitted to substitute any of the named categories
“samples, projects, or portfolios” in the 1993 Ed Reform Act as evidence of
their competencies. Articles published, computers built, experiments carried
out, songs written, employment as a cashier--none of these activities, which
provide a performance-based authentic assessment of the ability to use
knowledge, are permitted by the state BOE.
See 603 C.M.R. et seq.
This is particularly
egregious for students in vocational and technical education programs whose
responsibility is to develop practical skills that can be applied in the world.
Authentic assessment
of these students’ mastery requires giving them opportunities to actively
demonstrate their skills.
The teaching and learning
that leads to an educated young person has many dimensions and aspects. Thus a
young person capable of writing the equation relating voltage, current and
resistance may be incapable of wiring a circuit to perform accordingly. On the
other hand, young people skilled and knowledgeable in the wiring and design of
electrical circuits may not shine in a description limited to algebraic
equations. Similarly students very competent in the construction of a stage set
or a house may be unable to prove Euclid’s theory for the geometry they were
implicitly making use of. On the other hand, many students who can write out
Euclid’s proofs are incompetent to use them to build stage sets or buildings.
In fact, this reality led to the development of the vocational technical high
schools that function throughout Massachusetts. The general citizenry needs
individuals with both dimensions of these skills. However, no single individual
need be the master of both to fully participate in and contribute to society.
The proper use of multiple
measures requires the use of different methods of assessment. Many states and
school systems use such assessment procedures, which are variously referred to
as performance, authentic and direct assessment.
The use of multiple
measures is frequently conceived as including evidence of student learning based
on work the student does as part of her or his regular schoolwork. This approach
is being adopted in a number of states, including Nebraska, Maine, New
Hampshire, Rhode Island and Vermont. The latter four have formed a consortium to
develop assessments for charting student progress to meet the requirements of
the federal Elementary and Secondary Education Act (also now known and the No
Child Left Behind Act). These assessments will develop assessments that teachers
can weave into their ongoing curriculum. Linda Borg, Educators Find Strength
in Numbers for Tests, Providence Journal (Feb. 24, 2003) <http://signon.projo.com/reg_signin.jsp?fw=mhtml:mid://00000541/>.
The evidence
from these other states – as well as from other nations – shows that a single
test used as a graduation requirement is not necessary and that it is possible
to evaluate students in light of governmental standards without resorting to a
single exam. Office of Technology Assessment, Testing in American Schools;
Asking the Right Questions, U.S. Congress OTA-SET-519 (Feb. 1992) (See
especially Chapter 5, “How Other Countries Test” at 135-164).
F.
THE 1993 EDUCATION REFORM ACT CALLS FOR CRITERION-REFERENCED TESTS; MCAS
TESTS IMPOSED BY 603 C.M.R. USE NORM-REFERENCED TEST CONSTRUCTION PROCEDURES
A test used to assess
mastery of a defined body of material is termed criterion referenced. Examples
would include many tests required for certification in professions or trades. A
very different mode of scoring is used in “norm referenced” tests. In this
case, scores are scaled or distributed according to the scores of other students
taking the test; they do not compare the student’s performance to a standard or
criterion, in this case, the body of knowledge and skills expected to be
mastered. Often, in using norm-referenced tests, the tester makes a policy
decision as to the distribution of grades to be associated with the raw scores.
For example, they may decide that the top 20% of scores will be given an A, that
the next 40% will be called B, the next 30% graded C, and 10% D, and then
normalizing the grade distribution so that the results conform.
As enacted in 1993, the
Act specifically calls for a “system of assessments” that would be “criterion
referenced.” G.L. c. 69, § 1E. However, as is apparent from the technical
reports on the MCAS, the MCAS tests have been developed using norm-referenced
test construction procedures. Walter Haney, Ensuring failure: How a
State’s Achievement Test May be Designed to do Just That, Education Week,
21:42 (July 19, 2002), at 56, 58. As explained below, this failure to comply
with the legal mandate of the Act guarantees that some proportion of students
will always fail the MCAS. Such intentional failing of the lower scoring
students – generally the most disadvantaged – independent of all other evidence
of school achievement, belies the plain language and intent of the Act.
From the
beginning, the MCAS test was intended by the DOE to be extremely difficult for
Massachusetts students, rather than to truly assess mastery of the frameworks
material. Anne Wheelock, et al., What Can Student Drawings Tell Us About
High-Stakes Testing in Massachusetts? Teachers College Record (Nov. 2,
2000) (visited Feb. 18, 2003)
http://www.tcrecord.org, ID No. 10634. From the beginning, policy-makers
intended MCAS results to be scored at a more demanding level than nationally
normed standardized tests. John Silber, then-Chairman of the Massachusetts BOE
in 1997, stated:
If on one of the
nationally normed tests that is given in grades 4, 8 and 10,
students should turn out with a B, and on the Advanced Systems test they came
out with a C, we might conclude that Advanced Systems has pegged it just right
with a more demanding standard, a standard that would approach international
standards. On the other hand, if the situation were reversed, where a nationally
normed test shows that the students were performing at about a C level
and Advanced Systems had a B level, then we would know that the standards in
that exam were perhaps not rigorous enough. (emphasis added)
Id.
The MCAS
imposed under 603 C.M.R. by the BOE raises the barrier so high that failure is
guaranteed for many, in clear contradiction to McDuffy and the intent of the
1993 Act.
For the MCAS, these
categories are Advanced, Proficient, Needs Improvement and Failing. These
categories are not referenced to the Standards or the Frameworks, but have been
chosen according to norm-referenced procedures. As is clear from Chairman
Silber’s statement and many DOE reports, the tests are designed so that on any
given administration, some students will always fail. Thus, in the development
of recent MCAS tests, if more than eighty percent (80%) of Massachusetts’
students answered a question correctly, it is excluded from a test.
Lakewoebeguaranteed: Misuse of Test Scores in Massachusetts, Part 1,
Education Policy Analysis Archives (May 3, 2002) <http://epaa.asu.edu/epaa/v10n24/>
visited Sept. 9, 2002. Criterion-referenced
tests, which the Act requires, do not require that a certain fraction of
students fail. Walter Haney, supra at 17.
Amici find it difficult to
believe that intentionally failing the lower scoring students – generally the
most disadvantaged – independently of all other evidence of school achievement,
was the intent of the Act.
African-American and
Latino students, those with special needs, and English language learners tend on
average to score low on MCAS, as they do on most standardized tests. The
norm-referenced construction of MCAS means that these groups are set up to have
failure rates. This represents both the innately discriminatory character of the
MCAS test and the failure of the DOE to insure that all children have
access to the education needed, as required by McDuffy. See 415
Mass. 545.
The above results violate
both the plain language and intent of the Act, which is to ensure children a
comprehensive and quality public education. See G.L. c. 69, et seq. The
legislative intent of the Act was not to provide a mechanism to deprive tens of
thousands of their deserved high school diploma. See Id. Amici
note that there are well established standards within all Massachusetts school
systems – classes attended, grades attained, skills demonstrated, that need to
be met to insure a high school diploma. There is no lack of historic, cultural
and economic barriers to a high school diploma for children from immigrant,
lower income or minority families.
The opportunity to take
the tests repeatedly helps some students get over the barrier. However, all the
failings, flaws and inequities of the initial test are repeated in the retests.
None of the critiques below are ameliorated by taking the MCAS test on multiple
occasions. In fact, the time taken from authentic teaching and learning further
erodes students’ access to quality education, and the testing experience erodes
their self-confidence.
Consider Vocational
student Ashley Shea:
When Shea took the MCAS
for the first time in her sophomore year, she passed the English section, but
failed the math section by two points. The second time she took it, her math
score dropped an additional two points. On her third attempt, her score dropped
by another two points.
“I kept going down, and I
started thinking, Is the test getting harder, or am I getting stupider?''' she
said. “I was feeling so much stress.''
The more she failed, the
more anxious she became, which seemed to put a damper on her performance.
“I'm just not a good
test-taker,'' she said. “I tend to freeze up.''’
Shea's math tutor, Linda
Deady, could see that the test was eroding Shea's confidence.
“For her, math isn't the
strongest it could be to begin with, but what this test has done is made her
feel even less confident about her math,'' she said.
Dina Gerdeman, MAKING
THE GRADE: SO CLOSE:...College-Material Vocational Students Often Have a Tougher
Time Meeting MCAS Requirements, Quincy Patriot Ledger (Feb. 25, 2003).
G.
THE SILBER/PEYSER BOARD OF EDUCATION CHANGED THE CURRICULUM FRAMEWORKS IN
MIDSTREAM
After the passage of the Act, the sitting BOE established a
broad-based program to develop Curriculum Frameworks that represented the
understanding and values of citizens in the Commonwealth. The first of these
were adopted in 1995. However, the Silber/Peyser Board subsequently dismantled
those frameworks and replaced them with altered frameworks. The Math Frameworks
were changed dramatically in 2000 during the period in which current plaintiffs
were supposed to master the material for the tests. This change was carried out
over the opposition of all the leading groups of math teachers and educators in
the Commonwealth.
Clive McFarlane, Math framework a tough shift; Old math about-face for city
schools, Worcester Telegram and Gazette (August 3, 2000). One high-level
educator warned:
‘I believe this document
will doom Massachusetts' children to mathematical illiteracy, as it perpetuates
the status quo and a vision of education that reduces mathematics to a set of
computational skills,’ said James E. Hamos, director of science education at the
University of Massachusetts Medical School and chairman of the mathematics and
science advisory council to the state board.
…… nearly 170
principals and math teachers throughout the state signed a letter urging the
panel to reject the proposal.’ Id.
In an extraordinary public
display of no confidence, The Math Advisory Committee to the DOE resigned in
protest. Ed Hayward, Math Curriculum Panel Quits to Protest Meddling,
Boston Herald (Feb. 11, 2000). In February 2000 the panel wrote in their
resignation letter, "We no longer have confidence in the quality or integrity of
the final version of the framework being considered for adoption." Id.
New frameworks were
adopted over the opposition of leading math teachers and educators throughout
the Commonwealth, and promulgated for implementation by school systems.
Unfortunately, retraining teachers and introducing new curricula take years to
implement. As a result, the plaintiffs could not possibly be expected to have
mastered the material being tested for in 2001 and subsequently. Clive
McFarland, Math framework update widens education rift, Worcester
Telegram and Gazette (July 26, 2000).
III.
THE DEFECTS AND FAULTS IN THE CONSTRUCTION AND SCORING OF THE MCAS MATH
AND ELA TESTS INVALIDATE THEIR USE AS A GRADUATION REQUIREMENT
A.
DEFECTS IN THE TENTH (10TH) GRADE MCAS
ELA TESTS
To appreciate how limited
standardized tests are for assessing student achievement the Court would benefit
from some examples. These reveal both the capricious and arbitrary of many of
the question as well as their lack of alignment with DOE curriculum frameworks.
To illustrate, below are
two questions from the MCAS tenth (10th) grade English tests. The
first appeared on the MCAS test in 1998, and then reappeared on the DOE web site
as an exemplary question. Students who failed the 10th grade English test in
2001 were urged to study it in preparation for their retest.
The item concerns a long
passage from Shakespeare’s Anthony and Cleopatra (Act II, Scene 2, lines
196-229), describing Cleopatra’s arrival on a river barge. The passage is
followed by six multiple-choice questions and one “open response” question.
For one of the multiple-choice questions, students are directed to
reread the following lines:
Enobarbus:
Her gentlewomen, like the
Nereides,
So many mermaids, tended
her i’ the eyes,
And made their bends
adorning;
The question is:
...In a contemporary
American production of Anthony and Cleopatra, these lines could be
rewritten as:
Her personal attendants,
resembling the sea nymphs Nereides,
Responded to her every
look as if they were so many mermaids,
And made their graceful
bowings beautiful.
These rewritten lines are
an example of
A. standard English.
B. non-standard English.
C. slang.
D. dialect.
How is a tenth (10th)
grader to respond? The rewritten lines resemble Standard English, but they look
unlike anything a thinking adult would ever write or say. Unfortunately,
“awkward English” isn’t one of the choices. In any case, a student who has
studied such distinctions knows that Standard English is a dialect, and so
“dialect” is perhaps the safest answer, but it’s the wrong answer.
A logical question is:
what does this question assess? The DOE claims the question is aligned with
English Language Arts Learning Standard 6:
Students will describe and
analyze how oral dialects differ from each other in English, how they differ
from written standard English, and what role standard American English plays in
informal and formal communication. See Dept. Of Education, <http:www.doe.mass.edu>.
Worthy goals—but does the
question get at the standard? For this question and many others like it: you
can’t know from a student’s response, right or wrong, whether the student has
achieved mastery of the standard. Therefore, the question is not aligned
with the standard. For the DOE, however, alignment is more like a word search:
if the word dialect appears both in the standard and in the question,
then the two are declared to be “aligned.”
Another question, from the
tenth (10th) grade English test, 2000: Students are asked to read a
passage about Pandora. One of the questions asks for the “best” meaning of a
word:
Olympian Zeus had filled
the jar to the brim with evils for man, thousands of sorrows and sicknesses that
now hovered, some to attack by day, and others to steal in by night.
36. What is the best
meaning for hovered as used in paragraph 8 of “Pandora”?
A.
lingered
B. threatened
C. wavered
D.
challenged
The related Learning
Standard is number eight (#8): “Students will decode accurately and understand
new words encountered in their reading materials, drawing on a variety of
strategies as needed...” [emphasis added] See Dept. Of Education
http://www.doe.mass.edu.
The context, semantic and
syntactical, suggests that “threatened” (B) may be the best choice. However,
Webster’s Unabridged Dictionary defines hover
alternately as “to linger” (definition #2) and “to waver” (definition #3).
Which response is correct? If we are checking for the use of strategies to
understand new words, then the choice of “threatened” demonstrates the use of
contextual cues. If we’re checking for prior knowledge, then “lingered” or
“wavered” are both plausible. When the context points one way and the
dictionary indicates another, the question can’t possibly assess mastery of the
learning standard. It’s not just a tricky question—it’s a trick question.
Each of the questions
above fails to test for mastery of its related standard. However, the fault
lies, not in these particular items, but in the very nature of such tests, which
are not suited for the assessment of rigorous, complex standards. Even the
written essay questions, relatively innocuous in themselves, are best assessed,
not by anonymous raters grading for an hourly stipend, but by experienced
teachers in the context of an ongoing classroom conversation.
Depriving a student of a
high school diploma on the basis of a “wrong” answer to one of these items, or
any of dozens like them, deeply violates the spirit and content of the 1993
Massachusetts Education Reform Act.
B.
DEFECTIVE QUESTIONS AND TEST CONSTRUCTION OF THE TENTH (10TH)
GRADE MCAS MATH TESTS INVALIDATE THEIR USE TO DENY HIGH SHOOL DIPLOMAS TO
SENIORS
The MCAS Tests administered in 2001 were rife with erroneous questions such that
test scores are not valid measures of a student’s knowledge or understanding.
The 2001 tenth (10th) grade MCAS math test contained so many
defective questions that it cannot be used as a graduation requirement,
independently of whether it conforms to the Act.
Analysis of the questions
on the 2001 tenth (10th) grade math test revealed six (6) of the
forty-one (41) 2001 MCAS math problems were flawed or erroneous, and should have
been excluded from the test. Eugene D. Gallagher, An Analysis of Problems on
Two 10th Grade MCAS Math Tests, Dept. of Environmental, Coastal &
Ocean Sciences, Univ. of Mass. at Boston (Sept. 29, 2001) at 3.
Problems 9, 39 and 40 should have dropped from the scoring. Id. Problem
24 had more than one correct answer. Id. Problems 9 and 30 were based
on concepts introduced for the first time in the grade 11-12 learning
standards. Id. The tests should have been rescored, or students whose
scores could be affected should be given the right to appeal their total score
if they listed a correct answer to any of these questions. Id.
Gallagher’s study went on
to say:
Question 31 used the wrong
formula for loan payments. Problem 39 showed a boxplot, but not a boxplot that
matches any to be found in statistics reference books. Students were expected
to know that MCAS uses one feature of the standard boxplot, the median, but not
the other features of Tukey’s standard boxplot. Id.
The 2000 and 2001
MCAS math exams was problematic in the area of probability and statistics.
Id. For example, due to poor phrasing, the only correct answer to one
probability question (Problem 9) was not one of the acceptable choices. Id.
at 7. Moreover, the question was based on a concept (simulating probabilities
using random numbers) from the twelfth (12th) grade curriculum
framework. Id. Other questions were also based primarily on eleventh
(11th) and twelfth (12th) grade frameworks. Id. at
4.
Another
‘open-response’ question (Item 40 on the Spring 2001 Exam), involving a survey
of students, was so flawed as to be unanswerable. Id. at 21. The only
part that wasn’t flawed required statistics calculated in a few keystrokes from
$100 graphing calculators, but not the four-function calculators that DOE says
are adequate for the MCAS exam. Id. at 4, 19. The criteria apparently
used to award points for this flawed question (Item 40) were based on grammar as
much as mathematics. Id.
Another question (Problem
39) asked students whether the centerline in a boxplot represented the mean or
the median. Id. at 16. Only fifty percent (50%) of Massachusetts tenth
(10th) graders answered the question correctly. Id. In
fact, most scientists trained before the 1980s wouldn’t be able to answer the
question. Id. at 16-18. The boxplot was introduced by Tukey in the late
1970s and is used by statisticians to identify outliers in data. Id. at
16. Unfortunately, the MCAS boxplot is not a valid Tukey boxplot, the type of
boxplot used by statisticians and taught in college statistics courses. Id.
The only DOE reference to their boxplot, which doesn’t identify outliers, is a
link to the Harcourt Education web site. Id.
Below are two examples of erroneous
questions from the 2001 MCAS math test:
Problem 24: two applications of the
distributive property
Problem 24 from the Spring 2001 10th
grade
math test. Both C and D apply the distributive property.
Why question 24 is
flawed
While 24C is a correct answer, so too is 24D. The
first step used in the proof of the equality of 24D uses the distributive
property:
-5xy +
5xy + 3xz = xy (-5 + 5) + 3xz [Distributive
property]
= xy (0)
+ 3xz
=
3xz
(1)
Clearly, the solution shows an application of the distributive
property. The correct answer is either 24C or 24D.
Question 30: A Euclidean proof
involving congruence and similarity
The problem
Problem 30 requires a deductive proof in Euclidean geometry.
Problem 30 is based on an 11th
and 12th
grade
learning standard G.G.5 (Box below). The concept of congruence and
similar triangles was listed as a 9th
and 10th
grade
learning standards, but the question asks students which statement can be used
to prove
similarity. The learning standard requiring students to know about
proofs involving congruence and similar triangles is G.G.2, shown in Figure 7,
which appears for the first time as an 11th
& 12th
grade
learning standard. This question should not have been on the 10th
grade
math test.
Problem 30 involves a deductive proof in Euclidean geometry. This
learning concept is introduced in G.G.2, from the 11th-12th grade
learning standards (MA Math Curriculum Frameworks, 2000, p. 95)
Problem 31: How not to calculate your
loan payment
The problem
One of the problems that virtually every adult member of the Commonwealth will
experience is making payments on a loan. This is a fundamental concept that is
rightly included in the K-10 math curriculum frameworks. The subject of using
exponential equations and compound interest is covered in 10.P.7, shown in the
box. Every student taking the MCAS will have their lives affected by the
calculation of compound interest and amortization. It is overly ambitious to
require students to know the equations used in an amortization, they should know
that it is illegal to charge on principal that has been paid off, and that the
monthly payment is largely devoted to the payment of interest during the early
stages of a payment schedule.
Figure 8. Grades 9-10 learning standards from page 73 of the November 2000
Massachusetts Mathematics curriculum framework. Learning standard 10.P.7
asks that students know how to solve everyday problems using exponential
functions, including compound interest.
Figure 9. Question 31 from the Spring 2001 10th
grade
math test. The formula for a loan payment is incorrect.
The problem with question 31
A student can calculate the loan payment in question 31, producing
the answer B. $145. If a student has been exposed to the concept of
compound interest in a bank account or the value of paying down the principal on
a loan, then the student might think, “But, this equation doesn’t reduce the
interest payments as the original loan amount is paid off; the payment MUST be
less than $145! I’ll choose the only answer less than D.” Indeed. The proper
payment on a $3000 loan at 8% for two years is $135.68, roughly half way between
choices B and D.
A student with a TI-83 calculator might be perplexed by this
question. By hitting the financial calculation key, the student would merely
have to enter the principal, interest rate, and loan duration, and, like a
spreadsheet, the TI-83 calculator would instantly calculate the proper loan
payment of $135.68.
MCAS introduced needless complexity to the question by using the
term principal, a term not mentioned in the Massachusetts Mathematics Curriculum
Framework. Many 10th
graders wouldn’t know that the
principal is the amount of the loan. This confusion could have been alleviated
by replacing “principal” with “loan amount” or by stating that the loan had a
principal of $3000. Gallagher, at 10-11, 13-15; See Exhibit B.
C.
THE MCAS CATEGORIES “FAILING” AND “NEEDS IMPROVEMENT” DO NOT CORRESPOND
TO THE CRITERION-REFERENCED REQUIREMENT OF THE 1993 EDUCATION REFORM ACT, BUT
ARE ARBITRARY CUTOFFS
Categorical designations
based on MCAS results of students as advanced, proficient, needing improvement,
or failing have no reasonable scientific or educational basis. Walter M. Haney,
Address entitled The MCAS Miasma, at the MCAS Forum Can We Afford MCAS
in 2003? Gardner Auditorium (Feb. 11, 2003). Students supposedly “failing”
MCAS have been shown to score well on national tests, such as the Stanford-9,
Explore and the Preliminary Scholastic Aptitude Test. Id. The absence
of correlation between MCAS scores and more thoroughly validated tests is
striking.
At its January 2000
meeting, the BOE established the official requirement that students in the
graduating class of 2003 meet or exceed the threshold scaled score of 220 on
English and mathematics MCAS grade ten (10) tests, in order to satisfy
requirements of the "Competency Determination" for a high school diploma.
Minutes, Competency Determination: Adoption of Regulation on Grade 10 Passing
Standard for Class of 2003, Board of Education, Mass. (Jan. 25, 2000) at 1,
<http://www.does.mass.edu/boe/bib/bib00/12800.html>. No compelling evidence or
rationale was presented by the BOE matching these cut scores to the content of
the curriculum frameworks. Since the test scores are not criterion referenced,
as noted above, the cut scores appear to represent a policy decision as to what
constitutes an acceptable level of failures.
The lay public
generally assumes that the 220 number represents an objective count of the
number of items answered correctly. In fact this is not the case. The “220”
cut score is a scaled value that does not represent a direct comparison with the
frameworks or core curriculum, but an arbitrary decision made in private by
individuals engaged by the DOE. A recent investigation of the setting of the
cut score in the New York State Regent’s Physics exam is revealing. Michael
Winerip, Passing Grade Defies Laws of Physics, New York Times, (March 12,
2003) at A22. The state convened a panel of
teachers to decide the cut score of their standardized test. Id. When
the panel brought in their decision, the Commissioner of Education decided it
was too low, and convened another panel to raise the cut score higher. Id.
According to a member of the first panel “They asked our opinion and when they
didn’t get what they wanted, they disregarded it and came up with a completely
different number.” Id. According to another experienced panel member “It
was pure voodoo, the way the state picked that passing score”. Id. We
believe the Massachusetts’ process was driven by similar political criteria as
was documented in New York. Thus, the cut scores cannot be taken as a reliable
measure of student academic achievement.
D.
SCORING ERRORS RAISE SERIOUS QUESTIONS AS TO THE RELIABILITY OF MCAS
SCORES
MCAS tests are not scored
by Massachusetts’s teachers, but by temporary workers hired by Harcourt.
Cameron Fortner, Who’s Scoring Those High Stakes Test? Poorly Trained Temps.,
Christian Science Monitor (Sept. 18, 2001) at 19 <http//wwwcsmonitor.com/2001/0918/p19s1-lekt.html.
Harcourt has already been found liable for serious errors in the scoring of its
tests. Ulrich Boser, States Face Limited Choices In Assessment Market,
Education Week (Mar. 8, 2000).
The California Board of
Education in August 2001 voted to withhold One Million One Hundred Thousand
Dollars ($1,100,000) in contract money from Harcourt because of errors in the
grading of more than 400,000 students' tests. Id. California
legislators subsequently ordered an audit of the state's two-year-old
Standardized Testing and Reporting program. Id. Harcourt was one of
several testing firms found guilty of grading errors. Id. Subsequently
the state of Vermont found that Harcourt had seriously erred in the scoring of
tests used in that state (luckily for Vermont students, they were not high
stakes tests). Id.
With respect to the MCAS, more than 600 students who
thought they failed the Grade 8 MCAS history exam earned passing scores only
after John J. Gibbons Jr., a sharp-eyed teacher at Clinton Middle School,
spotted a multiple choice question with two right answers.
Associated Press, Teacher Finds MCAS Mistake,
Daily Hampshire Gazette (Sept. 27, 2002). State
education officials acknowledged that Gibbons was right, and the DOE found that
14,000 of about 75,000 eighth-graders were docked points for choosing one of the
correct responses. Id.
As a result, 666 borderline students had their scored
bumped from failing to passing. Id.
This is the second year that the DOE had publicly acknowledged erroneous
Massachusetts Comprehensive Assessment System questions.
These errors were not
found by Harcourt or the DOE. Given the growing track record of errors on
Harcourt tests, and given the impact of the test scores on the plaintiffs’
future prospects, the DOE needed to provide a mechanism to allow plaintiffs to
check for grading errors. Unfortunately the DOE does not provide such a
mechanism, such as forwarding the scored tests to families or to schools.
IV.
DEVASTING EFFECTS ON LATINO, AFRICAN-AMERICAN, ENGLISH AS A SECOND
LANGUAGE AND OTHER CHILDREN FROM LOW-INCOME FAMILIES
Any progress that may have
been made in the decade since McDuffy is about to be abrogated by the
enforcement of 603 C.M.R. 30.03, with 25% of Black and 30% of all Hispanic
seniors, but only 6% of white seniors, about to be prevented from graduating
from High School. D. I. Vigue, Race Gap Endures on MCAS Results: Minority
Leaders, State Plan Talks, Boston Globe (May 19, 2000) at B1; Department of
Education, Progress Report on Students Attaining the Competency Determination
Statewide and by District; Classes of 2003 and 2004, (Mass.) (2003). When
all students who entered high school in the class of 2003 are included in the
calculation, only 54% of Black and 40% of Hispanic students originally enrolled
in the class will graduate compared to 76% of White students. Auditi Guha,
Researcher says MCAS pass rate at 70, not 90 percent, Cambridge Chronicle
(Mar. 12, 2003) at 1, 4.
Students in the poorest
districts will also be denied a diploma at high rates. According to the
Massachusetts DOE 30% of Boston, 46% of Chelsea, 24% of Holyoke, 31% of
Springfield, and 44% of Lawrence twelfth (12th) graders will be
denied a diploma in 2003. See Department of Education, supra p. 33.
The above failure rates
underestimate the negative impact of the tests. Across Massachusetts, thousands
of students have dropped out of high school, many of them deeply discouraged by
their experience on the MCAS tests. The numbers the DOE reports in public
pronouncements of pass rates include only the proportion of current seniors who
have passed, disregarding 16,991 students who started in ninth grade with the
Class of 2003 but have dropped out, been retained in grade, transferred out of
state or into private school, or simply disappeared from the rolls. Auditi Guha,
supra.
These numbers establish
that either the tests are intrinsically discriminatory, or that the DOE has
failed to provide the education to these students as required by McDuffy.
See 415 Mass. 545. Amici believe that both are true, that the MCAS tests
are intrinsically discriminatory and that the DOE has failed to provide the
proper level of education to these students.
The most relevant variable
for explaining the MCAS results is the income level of the community of the
participating students. See Craig Bolon, Significance of Test-Based
Ratings for Metropolitan Boston Schools, Education Policy Analysis Archives
9:42 (2001); See Also, Laurel J. Sweet, Study Links MCAS Scores,
Quality of School Libraries, The Boston Herald, Oct. 26, 2000. This result
is most simply explained by the fact that the wealthier the school system, the
better able it is to educate and prepare its students. Though efforts have
certainly been made, the Commonwealth still has a ways to go to establish
educational equity. Under such circumstances, use of a single test is
fundamentally inequitable.
The fundamentally
inequitable character of the MCAS tests and the DOE’s preparation of students is
evident from the disproportionate number of Latino, African-American, and
Bilingual Students who were scored as failing or needs Improvement. As first
reported by the Mauricio Gaston Institute of the University of Massachusetts
(Boston), in the fifteen Massachusetts cities with the largest Latino
populations, the student failure rate was between seventy percent (70%) and
ninety-five percent (95%) of the MCAS tests. Miren Uriarte & Lisa Chavez,
Latino Students and the Massachusetts Public Schools, Mauricio Gaston
Institute of the University of Massachusetts (Boston) (April 2000) at 11.
African-American students faired similarly.
These students come
from the lowest income communities, whose schools have historically lacked the
resources for a quality modern education. As late as last year, nine (9) years
after the Act, Boston schools lacked basic textbooks, their laboratory classes
lacked supplies, and their class sizes were among the largest in the state.
Joan Vennochi, The Book Gap in Hub Schools, The Boston Globe, May 29,
2001, at 15
When high stakes tests are
used to evaluate students, there is little question about which students will be
disproportionately denied diplomas as a result of low scores, or which will
simply give up and drop out in anticipation of failing. If we allow the BOE to
insist that a student’s fate depends on a single test, the result over the next
few years will be the termination and the reversal of whatever progress was made
in the last decade in giving access to all our young people to a quality
education.
A.
THE MCAS TESTS AS SPECIFIED BY 603 C.M.R. DISCRIMINATE AGAINST SPECIAL
EDUCATION STUDENTS
The Act
implicitly
recognized that public schools are obligated to serve students who may not be
able to effectively demonstrate their knowledge and skills on standardized
assessments. For this and other reasons, Section 1I of the statute mandates that
the system shall "recognize sensitivity to different learning styles and
impediments to learning" and that it shall take into account "the particular
circumstances of students with special needs." G.L. c. 69, § 1I.
The use of
MCAS as a graduation requirement violates the letter and spirit of the Act,
particularly regarding special needs students, and conflicts with these
students' right to an individualized education and opportunity to demonstrate
competency in order to earn a high school diploma. See G.L. c. 69, §§
1D, 1I, 415 Mass. 545. Because the MCAS is of questionable validity for
students with special needs (that is, it has not been shown to reliably predict
the success or failure of these students after high school), it should not,
by itself, stand in the way of their future education and/or employment.
The current
graduation requirement to pass the MCAS test means that
many special needs students’ education
has been narrowed to intensive remediation and test preparation for two
academic subjects, English and math. As a result, they
are at risk of
being
denied a range of educational opportunities that may be essential to their
future success.
The DOE is
ostensibly complying with the legal obligation to offer alternative
assessments for special
needs students who cannot demonstrate their knowledge and skills on a
pencil-and-paper test. However, the implementation of the MCAS alternate
assessment raises questions about whether students who are judged eligible for
the portfolio-based alternate MCAS are, by virtue of their eligibility,
preordained for failure. The DOE offers a portfolio-based alternate assessment
MCAS for students with the "most significant disabilities." The DOE web site
asserts that, "Alternate assessments are intended for a very small number of
students who are unable to take MCAS tests, even with accommodations, due to the
nature and complexity of their disability (ies)...Participation in alternate
assessment has remained steady at about 1% of all students scheduled for
assessments across grades 3-10." Department of Education, Report
(visited March 18, 2003) <http://www.doe.mass.edu/mcas/alt//0102ssum/report.html>.
Special education
teachers, who spent many hours compiling alternate assessment portfolios that
they believed clearly demonstrated their students' progress and proficiency,
were devastated and mystified, along with their students, when virtually all of
these alternate assessment portfolios--699 out of 700--were given failing
grades. (Source: "Only One Passes MCAS Alternative," Boston Globe, 11/06/01,
Page A7). Some special education teachers said they were not provided adequate
instruction in compiling students portfolios and that their students were
penalized with failing grades for errors made by the teachers in compiling the
portfolios. [there's no published source for this claim. This is something that
was told to me by parents and teachers]
Even more troubling than
this failure rate was the response of DOE Associate Commissioner for Testing
Jeffrey Nelhaus, who was quoted in the Boston Globe saying that this "does not
represent a problem with the system." Id. In the same article, however,
Julia Landau of the Massachusetts Advocacy Center noted that "disabled students
working at grade level are allowed to take the alternate assessment, making it
unlikely that only one would pass." The article notes that Landau suggested "a
scoring problem might account for the dismal results."
We submit that
if the goal is to make every child a winner, this failure rate has to
indicate either a problem with the system of assessment or the quality of
educational access and instruction. Amici believe that while both factors are
operating, it is clear that an alternative assessment that could have been
implemented to enable some special needs students to be judged based on their
ability, not their disability, has been implemented in such a way that it is a
theoretical, not a practical, alternative for students who wish to demonstrate
their competency.
According to
the Disability Rights Advocates' Do No Harm, "There is no educational or
legal basis for limiting the availability of alternate assessments. An alternate
assessment should be available to all students for whom all or part of the
general assessment is not appropriate, and should have no pre-set artificial
caps limiting the alternate assessments to a certain percentage of the
population. Instead, the governing principle should be to design assessments
which ensure that students with learning disabilities are evaluated in a manner
that is fair."
Disability Rights Advocates,
Do No Harm -- High Stakes Testing and Students with Learning Disabilities,
(2001) at 6.
"The reality
is that many of these tests are not properly developed or implemented, leading
to high failure rates, increased numbers of students dropping out of school, and
loss of self-esteem and educational advancement," according to "Do No Harm."
"These tests thus are often of very questionable validity for students with
disabilities, and most states have not given sufficient consideration to either
appropriate accommodations or alternative assessments for disabled students. In
addition, because implementation of the high-stakes tests is frequently rushed,
students are not given enough time to prepare." Id. at 1.
It does not
have to be that way. The National Center for Fair and Open Testing, in
collaboration with the Massachusetts Teachers Association, together with
independent education assessment professionals, have developed a carefully
constructed portfolio assessment system that could answer the statute's
requirement for multiple forms of assessment that do not discriminate against
minority groups. See
<http://www.fairtest.org/care/accountability-sum.html>
B.
ADDITIONAL HARM DONE TO SPECIAL EDUCATION STUDENTS
The
high-stakes use of the MCAS has had a disproportionately negative impact on
students with special needs and/or learning disabilities: Nearly one (1) in
three (3) special needs students in the Class of 2003 have failed the math and
ELA sections of the MCAS even after four opportunities for retesting. <http://boston.com/education/news/04/030403_mcas_pass.html>
These numbers
do not capture the pain experienced by special education students and families,
who have worked enormously hard to continue the children’s education and now
feel that the stigma of disability has been magnified, not minimized, by the
MCAS graduation requirement.
While those
responsible for implementing the use of a single standardized test as a
competency determination often cite the need for high stakes to motivate
students and teachers to work hard and perform to high academic levels, they
peculiarly and perversely ignore the testimony of experts in special education
and learning disabilities regarding the negative psychological consequences of
high-stakes testing on these populations of students.
A recent
examination of studies concerning high-stakes tests and student motivation
offers insight into how, despite lofty rhetoric about students' working harder,
high-stakes MCAS may undermine motivation and contribute to growing achievement
gaps. Researcher Wynne Harlen, formerly Director of Education Research for the
United Kingdom, reports that studies point to "a serious and negative effect" on
motivation and a downward spiral in engagement that results in lower achievement
for already low-achieving students."
Wynne Harlen, et al., A Systematic Review of the
Impact of Summative Assessment and Tests on Students’ Motivation for Learning,
Evidence for Policy and Practice Information and Coordinating Centre (June 2002)
(review) (visited March 2003)
http://eppi.ioe.ac.uk/EPPIWeb/home.aspx?page=/reel/review_groups/assessment/review_one_abstract.htm.
Lower-achieving students are doubly disadvantaged by summative assessment
[defined as assessment used to summarize the learning that had taken place in
order to grade, certificate or record progress]. Id. Being labeled as
failures has an impact not just on current feelings about their ability to
learn, but lowers further their already low self-esteem and reduces the chance
of future effort and success." Id.
In "Do No
Harm: High Stakes Testing and Students with Learning Disabilities," the authors
note that, in the imposition of high-stakes tests for learning disabled
students, there has been "no recognition of the potential negative consequences
of high-stakes standardized testing, nor was there a discussion of the
safeguards that must be in place in order to ensure that standardized testing is
carried out in a fair and nondiscriminatory manner.”
Disability Rights Advocates,
at 1.
Individual special needs students testify to the
psychological impact of the tests. An article entitled, For special ed
students, the MCAS can be a nightmare, by Dina Gerdeman, begins:
Every night,
Jonathan Galina lies awake in bed with the same thought: ‘I'm not going to
school tomorrow. I quit.' Galina, a Randolph High School senior with a learning
disability, is worried that he might not see graduation day in June because he
can't pass the MCAS.
And if he doesn't pass the test and can't get a high school diploma, Galina
wonders if he will ever finish college, get a decent job, leave home. He spends
a good part of each day worrying about the test, stressing about his future.
‘It scares me. I
feel very stressed out,' Galina said, staring at his
hands. ‘I feel that I'm a failure.’
Dina Gerdeman, For
special ed students, the MCAS can be a nightmare, The Patriot Ledger, (Feb.
24, 2003).
Josh Kaufman, a Brookline
High School junior, described the misery of worrying about this test blocking
his future while getting solicitations from Ivy League colleges because of his
excellent verbal abilities (he has a math disability). Address of Josh Kaufman,
Can We Afford MCAS in 2003?, Gardner Auditorium (Feb. 11, 2003).
Just the other
day, I received a letter, which read:
’Each year, Brown University works with the College Board and other testing
organizations to identify exceptional high school students from all over the
world. Congratulations on your academic achievement.’
As a student who has failed the MCAS, this letter only deepened my frustration.
My successes, because they come in the shadow of MCAS failure, are hard to feel
good about. I did well on the PSATs (hence the letter), get good grades, am team
photographer for the basketball team, and work hard in school. But I also have a
learning disability, which means that no matter how hard I work, or how much I
study, I may not pass
the math retest. Id.
C.
THE MCAS TESTS AS SPECIFIED BY 603 C.M.R. DISCRIMINATE AGAINST VOCATIONAL
STUDENTS, IN VIOLATION OF THE 1993 EDUCATION REFORM ACT
An important component of
the Massachusetts public education system is its Vocational Technical high
schools. Students spend half the time on Basic English, math and social
studies, as do students in conventional high schools. During the remaining time
they develop advanced skills in electronics, carpentry, automotive diagnosis and
repair, culinary arts, drafting and other skills essential to the Massachusetts
economy. All of these students master skills and are assessed through actual
performance tests that are considerably advanced over their conventional high
school counterparts. Yet, no component of the 603 C.M.R. 30.03 mandated MCAS
tests assesses those invaluable and significant areas of knowledge and skill.
As a result, students in vocational technical schools are scored as failing at a
far higher rate than their counterparts in conventional high schools.
Trustees of
the vocational technical schools have been clear in their opposition to the
MCAS. Thus the Board of the Smith School in Northampton, Massachusetts,
unanimously endorsed the MASC resolution (above).
‘Four years ago, I said to
this board that the MCAS is the greatest threat to vocational education that I
can imagine,’ said trustee David Bourbeau.
‘The amount of resources
the school has put into (preparing for the test) has changed the nature of our
program,’ Bourbeau said. Unknown, Smith School Board raps State Test,
Daily Hampshire Gazette (Oct. 16, 2002) <http://www.gazettenet.com/10162002/schools/904.htm>
As of this fall, and the
third retest, twenty-four (24) of Smith Voke's 1997 seniors had not passed the
MCAS exam, despite being solid students in their programs. Id.
Though the Defendants
claim that the MCAS tests improve educational outcomes, all of the evidence is
to the contrary. MCAS is long, but not comprehensive. It is difficult, but not
rigorous. It alludes to Massachusetts’s standards, but it is not truly aligned
with those standards. The impact of the tests and teaching to the test has been
to lower educational standard in the Commonwealth; replace deeper education such
as critical thinking and application of classroom instruction to societal
situations, with test preparation. Examination of typical MCAS questions in the
sections below reveal how divorced the test content are from core learning.
Though the MCAS is rife
with errors and faults, the tests cannot be “fixed.” Rather, the experiences of
the MCAS tests supports the more general argument made by the National Academy
of Sciences, professional assessment organizations, and the test companies
themselves: that high stakes decisions about who passes, who fails, who is
retained, and who graduates are best made by those who know the student well,
using a variety of assessments. American Educational Research Association,
American Psychological Association, National Council on Measurement in
Education. (1999) Washington, DC: AERA at 146.
The intense focus on
preparing students for the MCAS has transformed many schools into test prep
centers, chilling the school environment and driving many students out of
school. Amici believe these factors contribute to schools’ weakening holding
power on students. The table below illustrates how serious the attrition problem
has become.
MASSACHUSETTS CLASS
OF 2003
GRADE
|
|
9TH
|
10TH |
11TH |
12TH |
|
|
77,733 |
69,447 |
64,105 |
60,781 |
|
|
NA |
8,286 |
13,628 |
16,952 |
|
Percentage of Students Lost From 9th Grade |
NA |
10.7% |
17.5% |
21.8% |
See Department
of Education, Reports (visited March 2003) <http:www.doe.mass.edu>.
Compared with earlier
classes, a higher percentage of students from the Class of 2003 have failed to
reach twelfth (12th) grade with their cohort. The percentage of the
class failing to reach 12th grade is now 22%, compared to 20% for earlier
classes.
The percentage of students
failing to reach the 11th grade with their cohort jumped dramatically from 13.4%
for the Class of 2002, to 17.5% the Class of 2003; this percentage grows to
19.3% for the Class of 2004.
A jump of 2.5
points in the percentage of ninth (9th) grade students failing to
reach tenth (10th) grade (from 8.2% for the Class of 2002 to 10.7%
for the Class of 2003) artificially boosted first-round MCAS scores for the
Class of 2003. Continued high attrition rates between grades nine (9) and ten
(10) will continue to prop up MCAS scores for future classes.
The high percentage of
students from the Class of 2004 failing to reach the eleventh (11th)
grade with their cohort (at 19.3%, the highest in six years) along with higher
enrollment in 9th grade for the Class of 2005, raises concerns that dropout
rates will rise for future classes. All of these numbers are much higher for
minority, special education, and English impaired students.
After careful study,
teachers, educators, academic experts in test assessment and school committee
members, have all concluded that the MCAS tests do not maintain standards as
claimed. See Affidavit of Deborah Meier(Exhibit C) (As a leading
educator and theorist, Ms. Meier asserts that the MCAS exams, for numerous
reasons detailed in the affidavit, negatively impact the education system and
society). They do the opposite, lowering the quality of education, narrowing
curriculum, and undermining inquiry-based education.
CONCLUSION
After thirteen
(13) years in school, no student’s knowledge, skills or potential can be summed
up by a single standardized exam. When any single standardized test is applied
indiscriminately across the entire range of economic, social, linguistic,
ethnic, and learning style diversity that exist in the Commonwealth of
Massachusetts, the results will most reliably reflect the socioeconomic
circumstance of the student, not his or her ability to make a contribution to
society.
The provisions
of the 1993 Education Reform Act that specify the competency determination
clearly articulated the need for a variety of assessments. These provisions
have been violated by the Silber/Peyser BOE, and the DOE by replacing the
variety of assessments specified in the Act with the standardized MCAS tests.
The harm to
more than 6,000 Massachusetts high school students is immediate and
substantial. Their current lives have been thrown into disarray and turmoil,
their future possibilities in doubt. The harm to the education system is longer
term. The quality of education in classrooms has been undermined. In some
cases, rigorous inquiry-based curricula is replaced by drill-and-kill test prep
sessions. In other cases, students have witnessed a transition from schools
starved of resources and support to intensive test preparation, without ever
experiencing the rich and diverse curricula many suburban students take for
granted. The faith of many parents, teachers and students in their education
system has been replaced with nagging doubts and disbelief. The attempt by the
Silber/Peyser BOE to control schools and curricula is lowering the educational
level of our future work force, weakening our schools, and putting future
student generations at risk.
The MCAS
graduation requirement should be set aside and replaced by authentic student and
school based assessments already in use in many districts and in neighboring
states. This action would be a major step toward raising the quality of public
schools in Massachusetts and bring State education policies back into alignment
with the Act and the McDuffy decision.
WHEREFORE,
Amici respectfully request this Honorable Court grant the Plaintiffs’ Motion for
Preliminary Injunction.
Massachusetts Coalition for
Authentic Reform in Education,
National Center for Fair and
Open Testing,
Cambridge Massparents for
Education not MCAS, and
Center for Collaborative
Education
By their Attorney,
Mark Alan Perkins, Esq.
Carrigan And Erickson
101 Tremont Street, Suite 412
Boston, MA 02108
(617) 338-9555
BBO# 648203
CERTIFICATE OF SERVICE
I Mark Alan Perkins, Esq., hereby certify that I served the above
Memorandum and supporting Exhibits to all parties in the matter via first class
mail this 20th day of March, 2003.
Mark Alan Perkins, Esq.
|
