|
COMMONWEALTH OF MASSACHUSETTS
SUFFOLK, ss.
SUPERIOR COURT DEPARTMENT
OF THE TRIAL COURT
CIVIL ACTION NO. 03-0071
STUDENT 1, A Minor by His Aunt and Next Friend, )
et al.,
)
Plaintiffs )
)
v. )
)
MASSACHUSETTS BOARD OF EDUCATION, )
et al.,
)
Defendants )
)
MEMORANDUM OF
AMICUS CURIAE IN SUPPORT
OF PLAINTIFF’S
MOTION FOR A PRELIMINARY INJUNCTION
CONCLUSTION …………………………………………………………….. 44
EXHIBIT A: Affidavit of Howard Fain in Support of
Amicus Memorandum
EXHIBIT B:
Dr. Eugene Gallagher, An Analysis of Problems on Two 10th
Grade MCAS Math Tests, Department of Environmental and Ocean Sciences, UMASS/Boston
(Sept. 29, 2001)
EXHIBIT C: Affidavit of Deborah Meier in Support of
Amicus Memorandum And Plaintiff’s Motion for a Preliminary Injunction.
INTRODUCTION
The Massachusetts
Coalition for Authentic Reform in Education (“MassCARE”), the statewide network
of more than 10,000 public school families, the National Center for Fair and
Open Testing (“FairTest”), the MassParents group, and the Center for
Collaborative Education, (“CCE”) which includes experts on student assessment
policies, submit this memorandum in support of the Plaintiffs’ Complaint and
Motion for Preliminary Injunction. Amici file this memorandum respectfully
refer the court to their statement of interest in their Motion for Leave to
Participate as Amicus Curiae filed March 18, 2003.
The imposition of the MCAS
tests by the Silber/Peyser Board of Education (“BOE”) represented a major
setback to education reform in Massachusetts. The use of a single standardized
pen-and-paper test as a graduation requirement is counter to modern research on
learning, teaching and authentic assessment of educational achievement. The
serious damage to students, classrooms and schools caused by the MCAS graduation
requirement was recognized early in the process by parents, teachers and
educators.
In response to
the seriousness of the problems, major Massachusetts organizations representing
individuals knowledgeable about the classroom environment publicly stated their
opposition and criticism of use of the standardized MCAS tests as a graduation
requirement in contradiction to the variety of assessments specified by the 1993
Education Reform Act (“Act”). These include the:
-Massachusetts Teachers Association, representing more
than 90,000 teachers;
-The Massachusetts Associations of School Committees, representing school
committees across the Commonwealth;
-MassCARE, representing more than 10,000 public school families across the
Commonwealth;
-The Alliance for High Standards not High Stakes, representing 50 education,
civil rights and other organizations;
-Citizens for Public Schools;
-New England Association for College Admission Counseling (NEACAC) representing
more than 3000 counselors;
-The Mass. State AFL- CIO, representing the interests of their working families.
-The Latino Parents Association.
Massachusetts led the
nation in the establishment of a publicly financed high school system to provide
a comprehensive education to middle class children. Insuring that access to
quality education was extended to children of lower income families was strongly
re-affirmed in the McDuffy decision. See McDuffy v. Sec’y of
the Executive Office of Educ., 415 Mass. 545 (1993). The 1993 Massachusetts
Education Reform Act (the “Act”) was the legislative mechanism for carrying out
the equitable access to education instruction of the McDuffy decision.
See G.L. c. 69 §§ 1 et seq. (1993).
Amici support the
plaintiffs’ argument that the 6.03 C.M.R. 30.03 regulation through which the
State Department of Education (“DOE”) claims to have implemented terms of the
Act is in violation of the Act, and is causing serious harm to students in
Massachusetts.
Unless DOE policies are
set aside or altered, more than 6,000 high school seniors, concentrated in lower
income communities of the state, will be denied their high school diplomas.
Department of Education, Class of 2003 Update (Mass.) (visited on Mar. 18, 2003)
<http://www.doe.mass.edu/mcas/results.html>.
This destructive result of the DOE testing policy on students represents a stark
violation of the McDuffy decision to increase access to quality education
among citizens of the Commonwealth.
Amici describe below the
significant harm to thousands of students, and their families, in the
Commonwealth as a result of the imposition of the high stakes MCAS tests,
including the grave harm to the plaintiffs, and other seniors, who will be
denied their high school diplomas. Amici document below multiple aspects of the
MCAS tests purchased from the Harcourt Corporation and imposed under 603 C.M.R.
30.03 that violate the provisions specified in Act and make it impossible that
such an assessment is fair and equitable as prescribed by McDuffy and in
the Act itself. In addition, Amici describe the failure of the DOE to provide
the necessary stable curriculum frameworks and frameworks implementation
necessary to meet the standard for instruction as called for in the Act.
I.
IRREPARABLE HARM TO PLAINTIFFS IF THEY ARE DENIED THEIR DIPLOMA DUE TO
THE 6.03 C.M.R. 30.03 MANDATED MCAS TESTS
Six
thousand Massachusetts high school seniors are about to be denied their high
school diplomas because they scored below the Needs Improvement cutoff of the
state sanctioned Harcourt MCAS Math or English Language Arts tests taken in 2001
or subsequently. See Department of Education, Reports (visited
March 2003) <http:www.doe.mass.edu>. These students are seniors who have
completed thirteen (13) years of schooling and have generally satisfied the
requirements of their school systems for high school graduation.
Many of these
students are bilingual, special education, minority, and vocational education
students. They include our children, our children’s friends and teammates, our
neighbor’s sons and daughters. They live in every region of Massachusetts and
represent the full diversity of our young adults. These students have stayed in
school and kept studying and working, despite the damage to their
self-confidence and self-esteem from being told that they have failed the MCAS
tests and retests.
In addition to
these
six thousand fifty-eight (6,058), sixteen thousand nine hundred ninety-one
(16,991)
other students who entered high school in the same freshman class
in 1999
will not be graduating with their class for various reasons. Auditi Guha,
Research says MCAS Pass Rate at 70, not 90 Percent, Cambridge Chronicle
(Mar. 12, 2003)
at 1, 4.
Among this very large group
(22% of the
original class)
are those who were discouraged by their
failing score
in the core 2001 MCAS tests, and the setting in place of what many considered an
arbitrary and discriminatory barrier to their graduation. See Department
of Education, Reports (visited March 2003) <http:www.doe.mass.edu>.
The prospect of
driving thousands of students a year out of public schools into the streets
represents a grave danger not only to the impacted youth, but also to
Massachusetts’ civil society and
its
economy.
The social and
human costs of this prospect are staggering. Denial of a high school diploma in
today’s economy is a death knell for a student’s future prospects. They will be
subject to this grave risk to their future regardless of courses taken,
classroom participation, grades, school citizenship, and special projects,
musical, dramatic or athletic contributions. Monty Neill, High Stakes, High
Risks: The Dangerous Consequences of High Stakes Testing, American School
Boards Journal, March 2003, at
18-21.
Absence of a diploma will
prevent these students from most forms of higher education. Denial of their
diploma will prevent them from receiving federal financial aid to schools they
could enter. Denial of their diploma will exclude them from many forms of
employment and even preclude enlistment in the Armed Services. For many
students, denial of their high school diploma will be the most damaging event in
their lives to date. The risk of failing to graduate high school is a source of
extreme anxiety to most of these students, who instead of planning for the next
step in their education or development must avoid stigmatization, unemployment,
and social marginalization. Dina Gerdeman, FEAR OF FAILURE: Students are
anxious, perplexed about the life-altering effects of flunking MCAS, The
Patriot Ledger (Feb. 22, 2003).
There can be little doubt
that in today’s highly technological and information-driven economy, absence of
a high school diploma deeply damages student’s prospects for future earnings and
opportunity. The "worth" of a high school diploma is clear. For example, U.S.
Census data show that those with a high school diploma, on average, earn $25,900
annually -- $7,000 more than high school dropouts. Laura Belsie, A Stronger
Link Between Degrees and Dollars, The Christian Science Monitor, July 18,
2002, at 1, 4.
A diploma also allows access to post-secondary education and the $45,400 average
annual salary that comes with college graduation. Id. Clearly, a
diploma is a ticket out of poverty, worth $1.2 million over a high school
graduate's lifetime. Id.
The denial of a diploma to these
students this year – and many thousands more next year, and the year after, will
be a social disaster in Massachusetts of great magnitude and impact not seen in
recent decades.
Amici argue below that
these students are being deprived of their diploma in violation of both the 1993
Education Reform Act, and contrary to the intent of the McDuffy
decision.
II.
THE MCAS TESTS DO NOT PROVIDE A
VALID ASSESSMENT OF STUDENT COMPETENCY AS REQUIRED BY THE 1993 EDUCATION REFORM
ACT
A.
THE SILBER/PEYSER BOE FAILED TO IMPLEMENT VALID AND AUTHENTIC STUDENT
ASSESSMENT AS MANDATED BY THE 1993 EDUCATION REFORM ACT
Following more than a
decade of litigation, the Massachusetts Supreme Judicial Court (SJC) ruled in
1993, in the landmark case of McDuffy v. Secretary of the Executive Office of
Education, that the system of financing public schooling in the Commonwealth
was unconstitutional and that the executive and legislative branches of
government were obligated to provide an education for all children of the
Commonwealth to “serve the interests of the children,” and “to prepare them to
participate as free citizens of a free State to meet the needs and interests of
a republican government.” Complaint, Student 1, et al. v. Driscoll et al.,
CA No. 02-3015-MAP, at 4 (Suffolk Superior Sept. 19, 2002); See 415 Mass.
545.
The Act specified that
Massachusetts’ public school students needed to acquire a common core of
knowledge as part of their education. G.L. c.69, § 1D. Chapter 69, Section 1D
of the Act required the development of “academic standards for the core subjects
of mathematics, science and technology, history and social science, English,
foreign language and the arts. The standards shall clearly set forth the
skills, competencies and knowledge expected to be possessed by all students at
the conclusion of individual grades or clusters of grades for grades
kindergarten through twelve.” G.L. c. 69, §1D.
Section 1D(i) goes on to
define the competency determination and refers to Section 1I for details of the
assessment instruments:
The competency
determination shall be based on the academic standards and curriculum frameworks
for tenth graders …and shall represent a determination that a particular student
has demonstrated mastery of a common core of skills, competencies and knowledge
in these areas, as measured by the assessment instruments described in section
one I. G.L. c. 69, § 1D(i).
The curriculum frameworks
referred to are described in Section 1E. G.L. c. 69, § 1E.
Chapter 69 Section 1I
describes the assessment process in some detail:
The system shall employ a
variety of assessment instruments on either a comprehensive or statistically
valid sampling basis. Such instruments shall be criterion referenced, assessing
whether students are meeting the academic standards described in this chapter.
As much as is practicable, especially in the case of students whose performance
is difficult to assess using conventional methods, such instruments shall
include consideration of work samples, projects and portfolios, and shall
facilitate authentic and direct gauges of student performance. G.L. c. 69, §
1I.
Furthermore,
The assessment instruments
shall be designed to avoid gender, cultural, ethnic or racial stereotypes and
shall recognize sensitivity to different learning styles and impediments to
learning. The system shall take into account on a nondiscriminatory basis the
cultural and language diversity of students in the Commonwealth and the
particular circumstances of students with special needs. Said system shall
comply with federal requirements for accommodating children with special needs.
All potential English proficient students from language groups in which programs
of transitional bilingual education are offered under chapter seventy-one A
shall also be allowed opportunities for assessment of their performance in the
language which best allows them to demonstrate educational achievement and
mastery. Id.
B.
MCAS USE AS SPECIFIED IN 603 C.M.R. 30.03 IS CONTRARY TO THE INTENT OF
THE EDUCATION REFORM ACT OF 1993; NO STANDARDIZED TESTS CAN PROVIDE A VALID AND
AUTHENTIC ASSESSMENT OF STUDENT LEARNING.
The legislators’ careful
description of the assessment requirement in the Act was derived from a large
and substantial body of experience and analysis of modes of assessing academic
skills and performance in children. H. Berlak, et al., Toward a New Science
of Educational Testing & Assessment, State University of New York Press
(1992); R.L. Linn, et al., Complex, Performance-based assessment:
Expectations and Validation Criteria, Educational Researcher 20 (1991) at
15-21; R. Mitchell, Testing for learning: How new approaches to evaluation
can improve American schools, New York: Free Press (1992); National Council
of Teachers of Mathematics (NCTM), Assessment standards for school
mathematics: Working Draft, (Oct. 1993); D. Wolf, et al., To use their
minds well: Investigating new forms of student assessment, G. Grant (Ed.)
(1991); American Educational Research Association, Review of Research in
Education, at 31-74.
A general conclusion that
has emerged is that no pen-and-pencil task can adequately assess the ability of
a student to perform complex intellectual skills and functions. Monty Neill,
The Dangers of Testing, Educational Leadership (Feb. 2003) at 43-46.
Multiple modes of assessment are required to assess skills in individuals whose
individual, cultural, family background differs. M.T. Nettles, & A.L. Nettles,
Equity and excellence in educational testing and assessment, Kluwer
Academic Publishers (1995). L. Valdez Pierce & J.M. O’Malley, Performance
and portfolio assessment for language minority students, DC: National
Clearinghouse for Bilingual Education (1992).
According to a board
member at the time, the BOE that participated in
the development of the Act had no intention of using a standardized test as the
determination of competency. MCAS Forum: Can We Afford MCAS in 2003?
Gardner Auditorium, (Feb. 11, 2003) (Statement of Frank Haydu, former BOE
member). The initial intent to follow the law was altered with the recomposition
of the state BOE and the ascendancy of John Silber to chairmanship of the BOE.
The Silber Board chose to ignore the letter of the law and previous work done by
the state DOE to create a system of multiple measures. The Silber/Peyser Board
reversed course, contracted out for a commercially developed standardized test,
which they termed the Massachusetts Comprehensive Assessment System (“MCAS”),
though such tests are neither comprehensive nor an “assessment system.”
Educators refer to such tests as a “high stakes” test, since student’s future
depends on the single score obtained.
At the time the
Massachusetts legislature passed the Act, a sound basis existed for using the
multiple measures required by the Act. Staff members of Amici FairTest
participated on the DOE’s advisory committee on testing, a group charged with
considering assessment options. While the state clearly intended to develop an
exam, it equally clearly did not intend to use that exam as the only form of
assessment for determining competency: and it did not intend an exam to be used
as a high-stakes hurdle that would deny students their diplomas. Rather,
multiple measures of different forms would all be used to provide evidence of
student progress, of school success, and for determining student proficiency or
competence. That is, the state intended to follow the letter and spirit of the
Act and to build on an existing body of knowledge to develop multiple measures.
(We also note that by the early 1990s, the state of Vermont was developing
portfolios, not just exams, to assess student learning.)
The definitive
national report on the use of high stakes testing was published in 1999 by the
National Research Council. National Research Council, High Stakes: Testing
For Tracking, Promotion, and Graduation, (Jay P. Heubert & Robert M. Hauser,
eds, National Academy Press, 1999). The report concluded that "no single test
score can be considered a definitive measure of a student's knowledge," at ES-2
and that "an educational decision that will have a major impact on a test taker
should not be made solely or automatically on the basis of a single test score."
at ES-2.
Rather than implementing
an authentic assessment system that could be used to raise standards, the Silber/Peyser
BOE on January 25, 2000, promulgated 603 C.M.R. 30.03. This imposed a “one size
fits all” testing system that creates barriers to educating public school
children, sharply discriminates against children from minority, special
education and vocational education, and is undermining public schools in
Massachusetts. The tests are not a determination of competency, but rather
arbitrary and capricious tests that often fail to assess the ability of students
to utilize their knowledge in society.
The Massachusetts Teachers
Association, The Boston Teacher’s Union, Citizens for Public Schools and more
than fifty (50) member organizations of the Alliance for High Standards, Not
High Stakes, all oppose the use of the MCAS tests as a graduation requirement.
For example, the resolution of The Massachusetts Association of School
Committees (“MASC”) states:
WHEREAS, a "high-stakes"
testing program will harm students by increasing high school drop-out rates,
discouraging other middle and high school students who perform at marginal
levels, and unnecessarily frustrating some younger children, especially those
with special needs, who are unable to succeed on the challenging MCAS tests, and
WHEREAS, there is
inadequate verification that MCAS results distinguish failing performance from
performance that needs improvement, and
WHEREAS, there is evidence
that states that have implemented education reform without high-stakes testing
are having better results in improving academic performance than states that
have adopted high-stakes testing,
NOW, THEREFORE, BE IT
RESOLVED, that MASC oppose the use of passing grade on the 10th grade Language
Arts and Math MCAS tests as a requirement for graduation from high school, and
be it
FURTHER RESOLVED that MASC
urge the Board of Education not to link MCAS results with the granting of high
school diplomas and MASC lobby for legislation that will prohibit the use of
MCAS results for that purpose. Massachusetts Association of School Committees,
Resolution, (Nov. 2000) (visited March 19, 2003)
http://www.masc.org/pos_frm.asp.
The DOE’s use of
standardized test results in isolation to bar a high school student from
promotion or graduation is contrary to professional standards regarding test
use. See Statement, American Education Research Association,
(visited Mar. 18, 2003) <http://www.aera.net/about/policy/stakes.htm>.
The American Educational Research Association (“AERA”) specifically states that
tests such as the MCAS should not be implemented:
In educational settings, a
decision or characterization that will have a major impact on a student should
not be made on the basis of a single test score. Other relevant information
should be taken into account if it will enhance the overall validity of the
decision. American Educational Research Association, American Psychological
Association, National Council on Measurement in Education. (1999) Washington,
DC: AERA at 146.
The AERA Position
Statement Concerning High-Stakes Testing in PreK-12 Education states:
“Decisions that affect individual students' life chances or educational
opportunities should not be made on the basis of test scores alone.” American
Educational Research Association, AERA Position Statement Concerning
High-Stakes Testing in PreK-12 Education,
(July 2000) <http://aera.net/about/policy/stakes.html.
The DOE denies
students the right to use spoken English language to express their English
Language Arts skills, thus assessing student competency in English Language arts
solely by a written exam. Amici document how this testing straitjacket violates
everything that educators have learned about assessing human language skills and
performance (Meyer and King affidavit). Amici assert that it is self-evident
that such an exam as the sole determinant cannot in any manner be considered a
valid, equitable or reliable method of assessment.
C.
THE MCAS MATH AND ENGLISH LANGUAGE ARTS (“ELA”) TESTS VIOLATE THE
SPECIFICATION FOR A VARIETY OF ASSESSMENTS AS CALLED FOR IN THE 1993 EDUCATION
REFORM ACT
The Act explicitly
requires the DOE to implement a “system of assessments” that would “employ a
variety of assessment instruments” including “consideration of work samples,
projects and portfolios.” G.L. c. 69, § 1D. Contrary to this requirement, the
state now seeks to prevent students from graduating from high school based on
whether or not their scores on just the Harcourt MCAS Math and ELA tests exceed
the arbitrary passing score of 220.
The DOE permits no other
evidence of student achievement, neither grades, awards, community service,
tutoring of other at-risk students, artistic talent, extracurricular
achievement, poetry written, articles published, or software programs written.
A student may be a leader in the community, a
three-letter athlete, the lead in the school play, mainstay of the jazz
ensemble, or an expert on Portuguese, Spanish, or Brazilian literature.
However, their lives will be derailed by the denial of their high school diploma
if they don’t score above the cutoffs on the MCAS math and ELA tests.
For example, Candido is a
Senior at the Burncoat High School in Worcester. See Affidavit of Howard
Fain, Exhibit 1. Candido has struggled to maintain good grades in high school,
bouncing back and forth between Special Education classes and regular classes.
Id. Candido successfully passed the ELA portion of the MCAS exam, but
has had difficulty with the math portion. Id. His guardians provide
extraordinary support, including taking advantage of the extra programs at the
school to prepare Candido for the MCAS math section as well as to working with
private tutors. Id. Unfortunately, Candido, even while achieving
tremendous improvement in his scores and excelling in his class work, has failed
to pass the exam after four attempts. Id.
Candido would like to give
back to his community someday by becoming a police officer. Id.
Tragically, Candido’s success story will be over before it begins because the
DOE insists on utilizing the MCAS exams to evaluate Candido’s knowledge and
skills in violation of the Act. A better solution would be to allow his
teachers, who observe and evaluate Candido’s knowledge and skills on a daily
basis, to assess his competency via a variety of assessment tools, including his
entire portfolio.
The Standards
for Educational and Psychological Testing, which have guided test developers and
users for decades, asserted in its 1999 edition, "[i]n educational settings,
a decision or a characterization that will have a major impact on a student
should not be made on the basis of a single test score." American Educational
Research Association, American Psychological Association, National Council on
Measurement in Education, Standards for Educational and Psychological Testing,
(1999) at 146. Even test publishers, including Harcourt Brace, CTB/McGraw-Hill,
Riverside and the Educational Testing Service, consistently warn against this
practice. Ulrich Boser, States Face Limited Choices In Assessment
Market, Education Week (Mar. 8, 2000). Relying
on a single standardized test to evaluate every student’s knowledge and skill
ignores individual ability and knowledge.
The claims of
the Defendants that the MCAS tests represent a “variety of assessments,” because
they have reading and writing questions in different formats, is specious. The
most profound aspect of human intelligence and the basis of human society is the
ability to communicate and interact through spoken language. The MCAS tests do
not allow students to use this most important and deepest expression of human
intelligence, cooperation, description, creativity and analysis.
Many
disciplines, including natural sciences, vocational education, music and arts,
have sophisticated assessment tools that test performance – actual mastery of
the skills and knowledge rather than just rote learning and test taking.
The National Forum on
Assessment produced Principles and Indicators for Student Assessment Systems,
which was signed by more than eighty (80)
national and regional education and civil rights organizations. FairTest,
Principles and Indicators for Student Assessment Systems, (1995). One of
the indicators reads, “Assessment systems allow students multiple ways to
demonstrate their learning.” Id. at 7. Another states, “Teachers,
schools, districts or states make reports on and decisions about individuals on
the basis of cumulative evidence of learning, using a variety of assessment
information, not on the basis of any single assessment.” Id. at 9.
D.
NEIGHBOR STATES USE A VARIETY OF DIRECT AND AUTHENTIC ASSESSMENTS RATHER
THAN JUST STANDARDIZED TESTS
The only New
England State that determines high school graduation using a test as a high
stakes hurdle is Massachusetts. V. Edwards, Quality Counts 2003,
Education Week (2003) at 88. Maine and Rhode Island explicitly use a variety of
assessment tools, as called for in the Act. Regulations of the Board of Regents
for Elementary and Secondary Education Regarding Public High Schools and
Ensuring Literacy for Students Entering High School, Final Version, (Jan.
2, 2003) at 1-11; FairTest, FairTest Examiner, (Spring 2002) at 12-13,
16. Tests are included, but they are only one component in the
assessment process. FairTest, supra at 13.
Decades of research on college admissions testing show that it is far
more sound (more valid and with smaller adverse impact on minorities and
females) to make decisions flexibly by using test scores, grades and other
information than to make decisions based on test scores alone. C. Rooney,
Test Scores do not Equal Merit, FairTest (1998). The recent exposure of
widespread errors in test scoring and reporting in the testing industry
indicates how unwise it is for the DOE to make important decisions mechanically
based on test scores in isolation. Henriques & Steinberg, None of the Above:
Right Answer, Wrong Score: Test Flaws Take Toll, N.Y. Times (May 20, 2001)
at A1; Steinberg & Henriques, None of the Above: Right Answer, Wrong Score:
Test Flaws Take Toll, N.Y. Times (May 21, 2001) at A1.
In response,
in the legislative session of 2001-2002, Senator Cynthia
Creem introduced legislation, consonant with the position of the AERA and
the National Forum on Assessment, to replace the current MCAS with a system that
would include a state exam and local assessments.
Senator Cynthia Creem, An Act to Require a Comprehensive Assessment System
for Students, Schools and Districts of 2002,
Sen. 257, (Mass. 2002).
Both Maine and Nebraska
have already developed state assessment systems that include local assessments,
which in turn typically include assessments used by teachers in their
classrooms. FairTest, supra at 13. Maine explicitly used the National
Forum’s Principles as a starting point for development of its state
system, which includes both an exam in three grades and local assessments.
Id. Maine’s system focuses on developing tasks and projects that teachers
can incorporate into their curriculum and score and report. Id. Items
from state “banks” of such tasks combined with ongoing classroom work can be
combined into portfolios that are also scored in light of state standards.
Id. See Also <http://www.state.me.us/education/lsalt/MAP.htm>.
Nebraska focuses more on
encouraging local districts to develop and use assessments that chart student
progress. FairTest Examiner, Nebraska and Maine Assessment Models,
FairTest (Spring 2002) 12-13, 16; See Also
http://www.nde.state.ne.us. It requires
districts to show that their assessments meet state standards for assessments;
their standards are highly similar to Maine’s and thus to the Principles.
Id. These states’ approaches complement a voluminous array of material
on using multiple forms of assessment. Id.
E.
THE
MCAS TEST VIOLATES THE 1993 EDUCATION REFORM ACT BY FAILING TO ALLOW
CONSIDERATION OF STUDENTS’ “SAMPLES, PROJECTS, OR PORTFOLIOS”
Under 603
C.M.R., students are not permitted to substitute any of the named categories
“samples, projects, or portfolios” in the 1993 Ed Reform Act as evidence of
their competencies. Articles published, computers built, experiments carried
out, songs written, employment as a cashier--none of these activities, which
provide a performance-based authentic assessment of the ability to use
knowledge, are permitted by the state BOE.
See 603 C.M.R. et seq.
This is particularly
egregious for students in vocational and technical education programs whose
responsibility is to develop practical skills that can be applied in the world.
Authentic assessment
of these students’ mastery requires giving them opportunities to actively
demonstrate their skills.
The teaching and learning
that leads to an educated young person has many dimensions and aspects. Thus a
young person capable of writing the equation relating voltage, current and
resistance may be incapable of wiring a circuit to perform accordingly. On the
other hand, young people skilled and knowledgeable in the wiring and design of
electrical circuits may not shine in a description limited to algebraic
equations. Similarly students very competent in the construction of a stage set
or a house may be unable to prove Euclid’s theory for the geometry they were
implicitly making use of. On the other hand, many students who can write out
Euclid’s proofs are incompetent to use them to build stage sets or buildings.
In fact, this reality led to the development of the vocational technical high
schools that function throughout Massachusetts. The general citizenry needs
individuals with both dimensions of these skills. However, no single individual
need be the master of both to fully participate in and contribute to society.
The proper use of multiple
measures requires the use of different methods of assessment. Many states and
school systems use such assessment procedures, which are variously referred to
as performance, authentic and direct assessment.
The use of multiple
measures is frequently conceived as including evidence of student learning based
on work the student does as part of her or his regular schoolwork. This approach
is being adopted in a number of states, including Nebraska, Maine, New
Hampshire, Rhode Island and Vermont. The latter four have formed a consortium to
develop assessments for charting student progress to meet the requirements of
the federal Elementary and Secondary Education Act (also now known and the No
Child Left Behind Act). These assessments will develop assessments that teachers
can weave into their ongoing curriculum. Linda Borg, Educators Find Strength
in Numbers for Tests, Providence Journal (Feb. 24, 2003) <http://signon.projo.com/reg_signin.jsp?fw=mhtml:mid://00000541/>.
The evidence
from these other states – as well as from other nations – shows that a single
test used as a graduation requirement is not necessary and that it is possible
to evaluate students in light of governmental standards without resorting to a
single exam. Office of Technology Assessment, Testing in American Schools;
Asking the Right Questions, U.S. Congress OTA-SET-519 (Feb. 1992) (See
especially Chapter 5, “How Other Countries Test” at 135-164).
F.
THE 1993 EDUCATION REFORM ACT CALLS FOR CRITERION-REFERENCED TESTS; MCAS
TESTS IMPOSED BY 603 C.M.R. USE NORM-REFERENCED TEST CONSTRUCTION PROCEDURES
A test used to assess
mastery of a defined body of material is termed criterion referenced. Examples
would include many tests required for certification in professions or trades. A
very different mode of scoring is used in “norm referenced” tests. In this
case, scores are scaled or distributed according to the scores of other students
taking the test; they do not compare the student’s performance to a standard or
criterion, in this case, the body of knowledge and skills expected to be
mastered. Often, in using norm-referenced tests, the tester makes a policy
decision as to the distribution of grades to be associated with the raw scores.
For example, they may decide that the top 20% of scores will be given an A, that
the next 40% will be called B, the next 30% graded C, and 10% D, and then
normalizing the grade distribution so that the results conform.
As enacted in 1993, the
Act specifically calls for a “system of assessments” that would be “criterion
referenced.” G.L. c. 69, § 1E. However, as is apparent from the technical
reports on the MCAS, the MCAS tests have been developed using norm-referenced
test construction procedures. Walter Haney, Ensuring failure: How a
State’s Achievement Test May be Designed to do Just That, Education Week,
21:42 (July 19, 2002), at 56, 58. As explained below, this failure to comply
with the legal mandate of the Act guarantees that some proportion of students
will always fail the MCAS. Such intentional failing of the lower scoring
students – generally the most disadvantaged – independent of all other evidence
of school achievement, belies the plain language and intent of the Act.
From the
beginning, the MCAS test was intended by the DOE to be extremely difficult for
Massachusetts students, rather than to truly assess mastery of the frameworks
material. Anne Wheelock, et al., What Can Student Drawings Tell Us About
High-Stakes Testing in Massachusetts? Teachers College Record (Nov. 2,
2000) (visited Feb. 18, 2003)
http://www.tcrecord.org, ID No. 10634. From the beginning, policy-makers
intended MCAS results to be scored at a more demanding level than nationally
normed standardized tests. John Silber, then-Chairman of the Massachusetts BOE
in 1997, stated:
If on one of the
nationally normed tests that is given in grades 4, 8 and 10,
students should turn out with a B, and on the Advanced Systems test they came
out with a C, we might conclude that Advanced Systems has pegged it just right
with a more demanding standard, a standard that would approach international
standards. On the other hand, if the situation were reversed, where a nationally
normed test shows that the students were performing at about a C level
and Advanced Systems had a B level, then we would know that the standards in
that exam were perhaps not rigorous enough. (emphasis added)
Id.
The MCAS
imposed under 603 C.M.R. by the BOE raises the barrier so high that failure is
guaranteed for many, in clear contradiction to McDuffy and the intent of the
1993 Act.
For the MCAS, these
categories are Advanced, Proficient, Needs Improvement and Failing. These
categories are not referenced to the Standards or the Frameworks, but have been
chosen according to norm-referenced procedures. As is clear from Chairman
Silber’s statement and many DOE reports, the tests are designed so that on any
given administration, some students will always fail. Thus, in the development
of recent MCAS tests, if more than eighty percent (80%) of Massachusetts’
students answered a question correctly, it is excluded from a test.
Lakewoebeguaranteed: Misuse of Test Scores in Massachusetts, Part 1,
Education Policy Analysis Archives (May 3, 2002) <http://epaa.asu.edu/epaa/v10n24/>
visited Sept. 9, 2002. Criterion-referenced
tests, which the Act requires, do not require that a certain fraction of
students fail. Walter Haney, supra at 17.
Amici find it difficult to
believe that intentionally failing the lower scoring students – generally the
most disadvantaged – independently of all other evidence of school achievement,
was the intent of the Act.
African-American and
Latino students, those with special needs, and English language learners tend on
average to score low on MCAS, as they do on most standardized tests. The
norm-referenced construction of MCAS means that these groups are set up to have
failure rates. This represents both the innately discriminatory character of the
MCAS test and the failure of the DOE to insure that all children have
access to the education needed, as required by McDuffy. See 415
Mass. 545.
The above results violate
both the plain language and intent of the Act, which is to ensure children a
comprehensive and quality public education. See G.L. c. 69, et seq. The
legislative intent of the Act was not to provide a mechanism to deprive tens of
thousands of their deserved high school diploma. See Id. Amici
note that there are well established standards within all Massachusetts school
systems – classes attended, grades attained, skills demonstrated, that need to
be met to insure a high school diploma. There is no lack of historic, cultural
and economic barriers to a high school diploma for children from immigrant,
lower income or minority families.
The opportunity to take
the tests repeatedly helps some students get over the barrier. However, all the
failings, flaws and inequities of the initial test are repeated in the retests.
None of the critiques below are ameliorated by taking the MCAS test on multiple
occasions. In fact, the time taken from authentic teaching and learning further
erodes students’ access to quality education, and the testing experience erodes
their self-confidence.
Consider Vocational
student Ashley Shea:
When Shea took the MCAS
for the first time in her sophomore year, she passed the English section, but
failed the math section by two points. The second time she took it, her math
score dropped an additional two points. On her third attempt, her score dropped
by another two points.
“I kept going down, and I
started thinking, Is the test getting harder, or am I getting stupider?''' she
said. “I was feeling so much stress.''
The more she failed, the
more anxious she became, which seemed to put a damper on her performance.
“I'm just not a good
test-taker,'' she said. “I tend to freeze up.''’
Shea's math tutor, Linda
Deady, could see that the test was eroding Shea's confidence.
“For her, math isn't the
strongest it could be to begin with, but what this test has done is made her
feel even less confident about her math,'' she said.
Dina Gerdeman, MAKING
THE GRADE: SO CLOSE:...College-Material Vocational Students Often Have a Tougher
Time Meeting MCAS Requirements, Quincy Patriot Ledger (Feb. 25, 2003).
G.
THE SILBER/PEYSER BOARD OF EDUCATION CHANGED THE CURRICULUM FRAMEWORKS IN
MIDSTREAM
After the passage of the Act, the sitting BOE established a
broad-based program to develop Curriculum Frameworks that represented the
understanding and values of citizens in the Commonwealth. The first of these
were adopted in 1995. However, the Silber/Peyser Board subsequently dismantled
those frameworks and replaced them with altered frameworks. The Math Frameworks
were changed dramatically in 2000 during the period in which current plaintiffs
were supposed to master the material for the tests. This change was carried out
over the opposition of all the leading groups of math teachers and educators in
the Commonwealth.
Clive McFarlane, Math framework a tough shift; Old math about-face for city
schools, Worcester Telegram and Gazette (August 3, 2000). One high-level
educator warned:
‘I believe this document
will doom Massachusetts' children to mathematical illiteracy, as it perpetuates
the status quo and a vision of education that reduces mathematics to a set of
computational skills,’ said James E. Hamos, director of science education at the
University of Massachusetts Medical School and chairman of the mathematics and
science advisory council to the state board.
…… nearly 170
principals and math teachers throughout the state signed a letter urging the
panel to reject the proposal.’ Id.
In an extraordinary public
display of no confidence, The Math Advisory Committee to the DOE resigned in
protest. Ed Hayward, Math Curriculum Panel Quits to Protest Meddling,
Boston Herald (Feb. 11, 2000). In February 2000 the panel wrote in their
resignation letter, "We no longer have confidence in the quality or integrity of
the final version of the framework being considered for adoption." Id.
New frameworks were
adopted over the opposition of leading math teachers and educators throughout
the Commonwealth, and promulgated for implementation by school systems.
Unfortunately, retraining teachers and introducing new curricula take years to
implement. As a result, the plaintiffs could not possibly be expected to have
mastered the material being tested for in 2001 and subsequently. Clive
McFarland, Math framework update widens education rift, Worcester
Telegram and Gazette (July 26, 2000).
III.
THE DEFECTS AND FAULTS IN THE CONSTRUCTION AND SCORING OF THE MCAS MATH
AND ELA TESTS INVALIDATE THEIR USE AS A GRADUATION REQUIREMENT
A.
DEFECTS IN THE TENTH (10TH) GRADE MCAS
ELA TESTS
To appreciate how limited
standardized tests are for assessing student achievement the Court would benefit
from some examples. These reveal both the capricious and arbitrary of many of
the question as well as their lack of alignment with DOE curriculum frameworks.
To illustrate, below are
two questions from the MCAS tenth (10th) grade English tests. The
first appeared on the MCAS test in 1998, and then reappeared on the DOE web site
as an exemplary question. Students who failed the 10th grade English test in
2001 were urged to study it in preparation for their retest.
The item concerns a long
passage from Shakespeare’s Anthony and Cleopatra (Act II, Scene 2, lines
196-229), describing Cleopatra’s arrival on a river barge. The passage is
followed by six multiple-choice questions and one “open response” question.
For one of the multiple-choice questions, students are directed to
reread the following lines:
Enobarbus:
Her gentlewomen, like the
Nereides,
So many mermaids, tended
her i’ the eyes,
And made their bends
adorning;
The question is:
...In a contemporary
American production of Anthony and Cleopatra, these lines could be
rewritten as:
Her personal attendants,
resembling the sea nymphs Nereides,
Responded to her every
look as if they were so many mermaids,
And made their graceful
bowings beautiful.
These rewritten lines are
an example of
A. standard English.
B. non-standard English.
C. slang.
D. dialect.
How is a tenth (10th)
grader to respond? The rewritten lines resemble Standard English, but they look
unlike anything a thinking adult would ever write or say. Unfortunately,
“awkward English” isn’t one of the choices. In any case, a student who has
studied such distinctions knows that Standard English is a dialect, and so
“dialect” is perhaps the safest answer, but it’s the wrong answer.
A logical question is:
what does this question assess? The DOE claims the question is aligned with
English Language Arts Learning Standard 6:
Students will describe and
analyze how oral dialects differ from each other in English, how they differ
from written standard English, and what role standard American English plays in
informal and formal communication. See Dept. Of Education, <http:www.doe.mass.edu>.
Worthy goals—but does the
question get at the standard? For this question and many others like it: you
can’t know from a student’s response, right or wrong, whether the student has
achieved mastery of the standard. Therefore, the question is not aligned
with the standard. For the DOE, however, alignment is more like a word search:
if the word dialect appears both in the standard and in the question,
then the two are declared to be “aligned.”
Another question, from the
tenth (10th) grade English test, 2000: Students are asked to read a
passage about Pandora. One of the questions asks for the “best” meaning of a
word:
Olympian Zeus had filled
the jar to the brim with evils for man, thousands of sorrows and sicknesses that
now hovered, some to attack by day, and others to steal in by night.
36. What is the best
meaning for hovered as used in paragraph 8 of “Pandora”?
A.
lingered
B. threatened
C. wavered
D.
challenged
The related Learning
Standard is number eight (#8): “Students will decode accurately and understand
new words encountered in their reading materials, drawing on a variety of
strategies as needed...” [emphasis added] See Dept. Of Education
http://www.doe.mass.edu.
The context, semantic and
syntactical, suggests that “threatened” (B) may be the best choice. However,
Webster’s Unabridged Dictionary defines hover
alternately as “to linger” (definition #2) and “to waver” (definition #3).
Which response is correct? If we are checking for the use of strategies to
understand new words, then the choice of “threatened” demonstrates the use of
contextual cues. If we’re checking for prior knowledge, then “lingered” or
“wavered” are both plausible. When the context points one way and the
dictionary indicates another, the question can’t possibly assess mastery of the
learning standard. It’s not just a tricky question—it’s a trick question.
Each of the questions
above fails to test for mastery of its related standard. However, the fault
lies, not in these particular items, but in the very nature of such tests, which
are not suited for the assessment of rigorous, complex standards. Even the
written essay questions, relatively innocuous in themselves, are best assessed,
not by anonymous raters grading for an hourly stipend, but by experienced
teachers in the context of an ongoing classroom conversation.
Depriving a student of a
high school diploma on the basis of a “wrong” answer to one of these items, or
any of dozens like them, deeply violates the spirit and content of the 1993
Massachusetts Education Reform Act.
B.
DEFECTIVE QUESTIONS AND TEST CONSTRUCTION OF THE TENTH (10TH)
GRADE MCAS MATH TESTS INVALIDATE THEIR USE TO DENY HIGH SHOOL DIPLOMAS TO
SENIORS
The MCAS Tests administered in 2001 were rife with erroneous questions such that
test scores are not valid measures of a student’s knowledge or understanding.
The 2001 tenth (10th) grade MCAS math test contained so many
defective questions that it cannot be used as a graduation requirement,
independently of whether it conforms to the Act.
Analysis of the questions
on the 2001 tenth (10th) grade math test revealed six (6) of the
forty-one (41) 2001 MCAS math problems were flawed or erroneous, and should have
been excluded from the test. Eugene D. Gallagher, An Analysis of Problems on
Two 10th Grade MCAS Math Tests, Dept. of Environmental, Coastal &
Ocean Sciences, Univ. of Mass. at Boston (Sept. 29, 2001) at 3.
Problems 9, 39 and 40 should have dropped from the scoring. Id. Problem
24 had more than one correct answer. Id. Problems 9 and 30 were based
on concepts introduced for the first time in the grade 11-12 learning
standards. Id. The tests should have been rescored, or students whose
scores could be affected should be given the right to appeal their total score
if they listed a correct answer to any of these questions. Id.
Gallagher’s study went on
to say:
Question 31 used the wrong
formula for loan payments. Problem 39 showed a boxplot, but not a boxplot that
matches any to be found in statistics reference books. Students were expected
to know that MCAS uses one feature of the standard boxplot, the median, but not
the other features of Tukey’s standard boxplot. Id.
The 2000 and 2001
MCAS math exams was problematic in the area of probability and statistics.
Id. For example, due to poor phrasing, the only correct answer to one
probability question (Problem 9) was not one of the acceptable choices. Id.
at 7. Moreover, the question was based on a concept (simulating probabilities
using random numbers) from the twelfth (12th) grade curriculum
framework. Id. Other questions were also based primarily on eleventh
(11th) and twelfth (12th) grade frameworks. Id. at
4.
Another
‘open-response’ question (Item 40 on the Spring 2001 Exam), involving a survey
of students, was so flawed as to be unanswerable. Id. at 21. The only
part that wasn’t flawed required statistics calculated in a few keystrokes from
$100 graphing calculators, but not the four-function calculators that DOE says
are adequate for the MCAS exam. Id. at 4, 19. The criteria apparently
used to award points for this flawed question (Item 40) were based on grammar as
much as mathematics. Id.
Another question (Problem
39) asked students whether the centerline in a boxplot represented the mean or
the median. Id. at 16. Only fifty percent (50%) of Massachusetts tenth
(10th) graders answered the question correctly. Id. In
fact, most scientists trained before the 1980s wouldn’t be able to answer the
question. Id. at 16-18. The boxplot was introduced by Tukey in the late
1970s and is used by statisticians to identify outliers in data. Id. at
16. Unfortunately, the MCAS boxplot is not a valid Tukey boxplot, the type of
boxplot used by statisticians and taught in college statistics courses. Id.
The only DOE reference to their boxplot, which doesn’t identify outliers, is a
link to the Harcourt Education web site. Id.
Below are two examples of erroneous
questions from the 2001 MCAS math test:
Problem 24: two applications of the
distributive property
Problem 24 from the Spring 2001 10th
grade
math test. Both C and D apply the distributive property.
Why question 24 is
flawed
While 24C is a correct answer, so too is 24D. The
first step used in the proof of the equality of 24D uses the distributive
property:
-5xy +
5xy + 3xz = xy (-5 + 5) + 3xz [Distributive
property]
= xy (0)
+ 3xz
=
3xz
(1)
Clearly, the solution shows an application of the distributive
property. The correct answer is either 24C or 24D.
Question 30: A Euclidean proof
involving congruence and similarity
The problem
Problem 30 requires a deductive proof in Euclidean geometry.
Problem 30 is based on an 11th
and 12th
grade
learning standard G.G.5 (Box below). The concept of congruence and
similar triangles was listed as a 9th
and 10th
grade
learning standards, but the question asks students which statement can be used
to prove
similarity. The learning standard requiring students to know about
proofs involving congruence and similar triangles is G.G.2, shown in Figure 7,
which appears for the first time as an 11th
& 12th
grade
learning standard. This question should not have been on the 10th
grade
math test.
Problem 30 involves a deductive proof in Euclidean geometry. This
learning concept is introduced in G.G.2, from the 11th-12th grade
learning standards (MA Math Curriculum Frameworks, 2000, p. 95)
Problem 31: How not to calculate your
loan payment
The problem
One of the problems that virtually every adult member of the Commonwealth will
experience is making payments on a loan. This is a fundamental concept that is
rightly included in the K-10 math curriculum frameworks. The subject of using
exponential equations and compound interest is covered in 10.P.7, shown in the
box. Every student taking the MCAS will have their lives affected by the
calculation of compound interest and amortization. It is overly ambitious to
require students to know the equations used in an amortization, they should know
that it is illegal to charge on principal that has been paid off, and that the
monthly payment is largely devoted to the payment of interest during the early
stages of a payment schedule.
Figure 8. Grades 9-10 learning standards from page 73 of the November 2000
Massachusetts Mathematics curriculum framework. Learning standard 10.P.7
asks that students know how to solve everyday problems using exponential
functions, including compound interest.
Figure 9. Question 31 from the Spring 2001 10th
grade
math test. The formula for a loan payment is incorrect.
The problem with question 31
A student can calculate the loan payment in question 31, producing
the answer B. $145. If a student has been exposed to the concept of
compound interest in a bank account or the value of paying down the principal on
a loan, then the student might think, “But, this equation doesn’t reduce the
interest payments as the original loan amount is paid off; the payment MUST be
less than $145! I’ll choose the only answer less than D.” Indeed. The proper
payment on a $3000 loan at 8% f | 
