New York State’s Regents Exams have been in the news recently. Last month, the New York State Education Department (NYSED) released a statement saying that all students would receive credit for question #24 on the June 2017 Geometry Regents Exam. It was the third question on this exam that had flaws. How can Regents exams be improved? As someone who has served on three NYSED committees, I have an insider’s perspective on the process.
In fact, I am currently participating in a four-day meeting in Albany that is sponsored by NYSED as part of its Regents exam development process. The process has six major steps (explained in more detail at http://www.p12.nysed.gov/assessment/teacher/test-development-process.pdf):
- Writing: Test questions and scoring guidelines for each question are written and reviewed.
- Field-Testing: Suggested test questions are given to a sample of students around the state.
- Rangefinding: Field-tested questions are graded using their scoring guidelines. If necessary, scoring guidelines (and sometimes the questions themselves) are modified. Student responses are chosen for the model response set.
- Preparation: Questions, scoring guidelines, and the model response set are reviewed, edited, and compiled into an exam.
- Administration: Students take the exams, which are then graded by teachers.
- Standard Setting: Based on statistical analysis and teacher input, a cut score is created for each exam, a conversion chart is generated, and raw scores are converted to scaled scores.
Right now, I’m in Albany for the rangefinding committee meetings. The process has been incredibly intense. We’ve spent eight hours a day over several days grading and discussing field-tested student responses to future Regents exam questions. This meeting has surprisingly turned out to be far more intense because we are actually selecting the model responses that are used to guide the grading statewide. Participating in these discussions has made me rethink how I will incorporate Regents questions into my instruction, but that is a subject for another article!
Having seen different parts of NYSED’s work, I’ve noticed four noteworthy things about the test development process:
First, personalities matter. Much of NYSED’s work is done through committees of educators. This requires good supervision and guidance to reach a true consensus. A strong personality can dominate a discussion and skew the results, while less vocal people won’t have their ideas implemented. Fortunately, all of the NYSED committees that I’ve worked on have had fair leaders and respectful members who respect each other’s opinions, allowing the committees to get proper input from everyone. Nevertheless, it’s always tricky to find the right balance of people to get work done.
Second, educators have significant input. Before serving on these committees, I imagined tests, curricula, and standards were written by a cadre of bureaucrats toiling in Albany. I was surprised to learn that in fact most of the work in New York State is done by actual educators – a combination of teachers, administrators, and college professors. (Unlike Grade 3-8 exams, which are written by testing companies, high school Regents exams have been written by educators for over a century.) NYSED selects educators from around the state using an open application process. From what I’ve seen, an individual teacher can actually make a difference.
Third, educators don’t control everything. From what I’ve seen, the NYSED staff genuinely value teacher input and want to allow teachers to make decisions whenever possible. Unfortunately, sometimes committees reach a consensus that is often very different from the final product that NYSED releases. The current Board of Regents, which has several former educators on it, genuinely values teacher input, but I suspect political concerns may also affect what happens in the end. In the long run, I think that teachers will always have limited input into the overall decision-making process.
Fourth, educators and NYSED need better support. NYSED’s test development process certainly isn’t perfect. Despite the many opportunities for teacher input, many errors still occur in Regents exams. Overall, I believe that the process of soliciting educator input is both valuable and generally sound. The NYSED staff and the educators that are involved are trying to do the best they can, but they have limited time and resources.
I believe that we can do more to improve testing in New York. More educators, particularly experienced ones, should apply online to participate in NYSED’s educator participation opportunities at http://www.p12.nysed.gov/assessment/teacher/home.html. Educators should continue to examine and comment on Regents exam questions. Unlike the publishers of other standardized tests like the SAT or PARCC exams, NYSED releases all Regents exam questions to the public. Most importantly, we need more supporting material on standards, assessments, and curricula so that all teachers and their students can benefit. This is critical since classroom teachers are ultimately the ones who implement standards and grade student assessments. Since NYSED lacks the funding to provide all of the supporting material that educators could use, NYSED needs more resources, something that only the state Legislature could remedy. Until then, others – perhaps experienced educators or organizations – need to fill in the gap by writing unofficial supporting material for standards, curricula, and assessments. Together, we can help make education in New York better for our students.