Finals! Huh! Yeah! What are they good for?

We just finished finals at Grand Valley. Coincidentally, this week of exams overlapped with the release of the second Hobbit movie; this might explain why these lines (modified from this poem) were going through my head Thursday as I walked around campus:

I am not trying to suggest that like the One Ring, final exams are evil and must be destroyed. However, I do wonder about the wisdom and unintended consequences from placing so much emphasis on a single assessment event.  Rarely do final exam scores make that much of a difference on final grades. Yet, there is so much stress associated with final exams, for both the students and the teachers, that it got me to wondering if these tests are worth it.

During the MTH 323 final event (we did not have a written exam), I asked the teachers to consider whether or not our school ought to have final exams and why. This is the course for preservice teachers that I designed to reflect what it looks like to be a math teacher in a K-8 school. So we created a "school" and went about developing a curriculum. Therefore, it seemed reasonable to discuss the final exam policy we would have at our school.

Grade-level groups talked about the question and then shared their thoughts and rationale. I added my two cents, but mostly these points represent the teachers' perspective.

• We need to have some sort of culminating experience as a way to identify what a student has mastered and possible weaknesses in the curriculum.
• The experience needs to be different than the traditional exams that represent a cram and purge approach that does not result in lasting learning.
• Like it or not, exams are a part of our culture and we would be doing our students a disservice if we fail to provide some experience with these types of assessments.
• Perhaps we could give the exams a couple of weeks before the end of the semester so those who need to can make revisions and the rest can extend their learning.
• Whatever we have them do needs to be meaningful.

It was the last point that resonated with me. How do we ensure that the culminating experience is meaningful - that it fosters the upper levels of the Engagement Continuum and not just compliance?

Developed by MTH 323 Teachers to monitor their engagement
Based on Engagement Taxonomy of Morgan and Saxton

At breakfast this morning, I asked a colleague what he thought final exams were good for. He said, "To see if students have synthesized the information addressed in the course." After a pause, he continued, "Of course, those are the questions they often struggle on most, which is discouraging."

"Why do we give all the other questions, then? The ones that focus on recall. Why not focus on the synthesis questions?" I asked.

"Probably because it's what we've always done," he responded. Neither of us were satisfied with this answer, but finals week was over (it was, after all, final) so there was nothing we could do about it now except think about what we might do different the next time around.

Does Test-Prep Make Sense?

If you drop by a Michigan middle school mathematics classroom this September, chances are you will find yourself in the middle of test-prep instruction. October brings the Michigan Educational Assessment Program (MEAP), our high-stakes standardized test, and schools want to ensure that their pupils are adequately prepared. The test-prep instruction usually entails one part content, one part test-taking strategies, and one part cheerleading.

This is not the relationship between instruction and assessment conceptualized in the Teaching-Learning Cycle. The role of assessment is to gather data about learners’ progress toward predetermined educational goals. Current educational policy stresses standardized tests, however, which stresses out administrators and teachers. Consequently, the MEAP becomes master of instruction rather than partner.

I rallied against this test-prep situation (and other standardized testing practices) for years, to no avail. The stakes were too high for anyone to consider an alternative. It was at a Michigan Reading Association Conference that I found a possible answer. (I learn a lot at this conference.) Patrick Allen led a session on developing deep, long thinkers in the time of standardized testing based on a book he had co-authored, Put Thinking to the Test. I thought that these ideas could reframe test-prep.

Put Thinking to the Test applies the research on proficient readers to taking tests. In other words, it treats standardized tests as a specific genre which readers can make sense of using well-documented comprehension strategies. Each strategy is addressed in a chapter demonstrating how it can be applied to a standardized test. From the table of contents:

• Ask Questions

• Create Mental Images

• Draw Inferences

• Synthesize New Learning and Ideas

• Activate, Utilize, and Build Background Knowledge (Schema)

• Determine the Most Important Ideas and Themes

• Monitor for Meaning and Problem-Solve When Meaning Breaks Down

Even in test-prep, teachers can focus on fostering thinking rather than memorizing facts or practicing skills.

As a result of reading this book, I now have concrete approaches that teachers can use to return instruction to its rightful place in the Teaching-Learning Cycle. I share these approaches in the classes I teach for preservice and inservice teachers and in professional development workshops. I would love to share it with you, but first I need to know what questions and concerns you have about what I have written thus far. After all, asking questions is one of the essential comprehension strategies propelling our thinking forward.

Are multiple-choice tests capable of evaluating learning?

"The writer's [teacher's] job is not to judge, but to understand."

Ernest Hemingway

Last week, I returned to writing about the Teaching-Learning Cycle (T-LC) by considering whether or not we can reclaim assessment from high-stakes tests. Recall that evaluation follows assessment in the T-LC: What can learners do? What are they trying to do? What comes next? Staying with the theme of standardized tests, I want to explore the idea of using multiple-choice items to evaluate learning.

My favorite example of this comes from the 1983 NAEP math survey. I had recently started teaching middle school math when the results for the item shown to the right came out. Only 24% of the national sample of thirteen-year-olds answered this item correctly. Nearly as many of the learners answered (c) 31.33. This is the first time I remember examining the other possibilities and wondering, “What were the kids that selected these other responses thinking?” [This question continues to fascinate me and resulted in this article that I wrote with Dr. Pam Wells.]

Using the evaluation framework, I assume that most of those answering 12 can find the remainder of the division problem and are trying to use it to answer the question. The kids answering 31 might be finding the correct number of buses by estimating the quotient or rounding it to the nearest whole number. And the 31.33 answers suggest that these thirteen-year-olds can do the division correctly but forgot to keep the context in mind. In fact, this focus on making sense of the situation seems to be what comes next for all of those who chose (a), (b), or (c).

You will notice that I did a fair amount of equivocating in my evaluation of the different responses in the previous paragraph. This is the problem with multiple-choice items – we cannot be sure why the test-taker selected one response over the other. Was it based on understanding? Or was it a guess? Or was it an intentional effort to undermine the assessment process? Only by looking at a group of responses can we begin to reduce such errors and be more confident in our evaluation of learners’ thinking. That is why I do not like using multiple-choice items as summative assessments but use them to track the progress of classes as a whole on content we are exploring.

Still, I believe I can improve the question so that it is easier to more accurately evaluate where learners are fluent and where they are approximating. One possible way is to replace 12 (which I consider the most unreasonable answer) with “I’m not sure so any choice would be a guess.” This does not add to my evaluation burden and because the item is formative the learner is more likely to be honest if they know I will use the information to inform instruction and improve their likelihood of later success.

For even more data, do as Karen Bailey suggests (in this PowerPoint) and simply add “Explain” to the end of the item. Some possible rationales are shown to the left. These responses are fascinating and certainly can improve our ability to evaluate their thinking and tailor future instruction to support their continued growth.

It will require more evaluation effort to look through and make sense of open-ended responses and teachers need to decide if it is justified. I once heard Grant Wiggins ask of assessments, “Is the juice worth the squeeze.” But this question should apply to the test-takers as well as the evaluators. Why should we subject kids to an assessment if the task does not truly measure what was intended?

This is one of the first posts that truly did get at my reason for starting this blog. I began this post thinking I would defend multiple-choice items as a reasonable way to evaluate groups of learners from a formative perspective. While I still believe evaluation can be done using a selected response format, I’m not sure identifying what learners can do and are trying to do is as easy as I thought. What do you think?

Can we reclaim assessment?

Each month, a group of faculty members from the two colleges involved in teacher education at GVSU meet to discuss issues related to teacher preparation and professional development. Because of my joint appointment I am a member of this group known as the Professional Teacher Education Advisory Council (PTEAC). Two months ago, a colleague from the English Department brought up the results from the recent PISA (Program for International Student Assessment) for us to discuss.

My two homes in the College of Liberal Arts & Sciences and the College of Education

Many of us were familiar with the results that suggest the United States is lagging behind other countries in education. Going beyond the results, we spent some time putting the data into context. (e.g. U.S. is much more diverse than top countries like Finland and Singapore.) I may come back to this point and what it means in another post, but for now, let’s just say that I see such comparisons as a distraction. Looking at sample PISA math questions, I can say that I want U.S. pupils to be able to answer them correctly. Whether they get more right than pupils in other countries is irrelevant.

A few weeks after this discussion, I attended a higher education conference put on by the GVSU Mathematics Department. One of the keynote speakers, Dr. Zalman Usiskin, opened the conference by addressing some of the same issues we had raised in PTEAC. While it was affirming to hear Dr. Usiskin’s presentation, I was inspired most by his words over lunch. He asked where I taught and I said at the university. He said, “You’re lucky then, because you have more freedom than K12 teachers. I don’t know why anyone would want to be a public school teacher under current conditions.” His words echoed in my head the rest of the conference.

During a panel session at the end of the conference, I listened as a parade of K12 mathematics teachers expressed their frustration with an educational system driven by standardized tests and their concerns about what the Common Core State Standards would add. Being a former middle school teacher I could empathize with their plight but I was somewhat disconnected now being in higher educations. It was then that Dr. Usiskin’s words hit me full force – I am lucky because of the freedom I have. What if I (and other teacher educators) used that freedom to provide cover for our K12 colleagues?

What I am proposing is that we reclaim the narrative related to assessment. Let the politicians and the newspapers have their standardized tests. We can reply, “Yes, those are the results and this is what they REALLY mean but they do not tell a complete story.” Here is where the collaboration between teacher educators and public schools comes into play. Teacher educators have the freedom and the resources available to help design reliable and valid assessments that assess the things local communities decide are important. Things like problem solving and critical thinking that cannot be easily assessed with a multiple-choice, standardized test.

What if schools came to colleges of education with a list of essential skills and processes they wanted to assess? What if the colleges of education used their experiences and resources to develop, implement, and analyzed the assessments? Could this provide the political cover schools need to reclaim the assessment narrative and return them to their proper role as preparers of democratic citizens (instead of test-prep factories)? Is this even possible? If so, then what comes next?