What is the purpose of pre-assessments?

In our course, Probability and Statistics for K-8 Teachers, we are trying to apply Design Thinking to a project involving teaching in a local 6th-grade math class. We will be focusing on 6.SP from the Common Core State Standards. Before we begin planning our lessons, we want to know what students can already do in order to build on their strengths. Therefore, we decided to design a pre-assessment.

We recently went through the Design Thinking process and tested out our pre-assessment ideas with the 6th-grade teacher. Design Thinking is an iterative process, and the feedback we received from the teacher reinforced this idea. It was clear that we had not spent enough time defining the project, which resulted in a lot of disconnected pre-assessment ideas. So tomorrow we will return to the Define step using the Project Priority Puzzle shown below.

Instructions:

• Select a phrase from each row (what, why, how, and when) in the table below that you feel ought to define our 6.SP pre-assessment. If you think a phrase is missing, write it in one of the blank spaces provided; 
• Use scissors to cut out each of your selections, along with the top phrase; and 
• Combine the phrases in order from top to bottom using tape to create your “Define” artifact.

Project Priority Puzzle

You could help us out by providing your definition of pre-assessment. Use the puzzle pieces above or create your own. Please add your definitions to the comments. Thank you in advance for your support of our future teachers.

How can we assess 3.MD.B.3?

Draw a scaled picture graph and a scaled bar graph to represent a data set with several categories. Solve one- and two-step "how many more" and "how many less" problems using information presented in scaled bar graphs. For example, draw a bar graph in which each square in the bar graph might represent 5 pets. [3.MD.B.3]

Teaching-Learning Cycle

Teachers use #MTBoS as a way to find interesting and effective math lessons. Recently, some of us have noticed that assessments are often lacking as this community shares in the work of teaching. So I am proposing the #MTBoSAP (Math Teacher Blog-o-Sphere Assessment Project) as a way to pass along our wisdom and experience assessing students in our mathematics classes. I figured I could start by sharing some work I am doing in one of my courses for preservice elementary teachers.

The teachers are currently researching assessment items related to the Measurement & Data Domain (focusing on Data) from the Common Core State Standards (CCSS) in Mathematics. One of the third-grade standards in that domain is written at the beginning of this post. Because we are partnering with a school district that uses EngageNY, we looked for assessment items that already existed within that curriculum. This is an exit ticket that we found in the Grade 3, Module 6, Lesson 4. 

We thought that this item did a fair job of assessing the last part of 3.MB.B.3 but completely missed the first part, "Draw a scaled picture graph and a scaled bar graph..." So we looked through the rest of the lesson and thought this item from the problem set looked promising.

This item has students "draw a scaled bar graph" (still no scaled picture graph) and asks that they solve "two-step 'how many more' and 'how many less' problems." As we thought about it further, however, we were concerned that it was not clear whether students would use the chart or the graph to answer the questions.

Therefore, I decided to try to modify the original exit ticket in order to assess more of the standard. I got rid of two of the bars and wrote questions intended to have them "draw (a part of) a scaled bar graph" and answer multi-step questions that require some "information presented in scaled bar graphs."

Please complete the bar graph using the following information. 

• The number of books checked out on Thursday was 10 more than the number of books checked out on Tuesday. Draw the Thursday bar.
• 1,480 books were checked out Monday through Friday. Draw the Friday bar.

What are your thoughts about using this modified assessment item to gather data on students' mathematical understanding related to 3.MD.B.3? Do you have a good item for this standard that you'd be willing to share? If so, please share the item or a link to the item in the comments. Also, If you have any other effective CCSS assessment items, please share them on Twitter using #MTBoSAP. Thank you in advance for all you do to advance the profession.

How will you make 100?

I heard a math professor share this assessment approach today and wanted to pass it along. The assessment is a take-home test made up of 60 or more questions. Some of the questions are worth five points - fairly simple straightforward items. Then there are the fifty pointers that are open-ended monsters. And all sorts of problems in-between worth points corresponding to their difficulty. 

100 Sculpture

Students have two weeks to complete the test. They can pick to submit any combination of problems that add up to 100 points. One might think that the students would select the easier problems, but the professor has found that given the choice (and the time) the students pick challenging problems. The mere choosing of items provides assessment information to the professor.

I love this idea and want to try it in my classes next semester - maybe  in concert with standards-based grading. Alternative assessment approaches are always of interest to me. And throw in the element of choice - I'm sold.

Has anyone else heard of or tried this assessment approach. I would love to know your thoughts or experiences. Thank you in advance for your participation.

Not so easy, is it?

Deborah Ball, Dean of the University of Michigan's School of Education, chairs the Michigan Council for Educator Effectiveness (MCEE) which was tasked to make recommendations to the Legislature about teacher evaluation. As part of her testimony before a joint meeting of the House and Senate Education Committees, she took some time to make the point that the work of teaching is often misunderstood and harder than most non-educators imagine.

The example she gives comes from fourth grade mathematics - multiplying two two-digit numbers (4.NBT.B.5). First, the teacher must be competent in the content. For example, what is the product of 49 and 25?

From Dr. Ball:

Obviously one wouldn't want anyone teaching third or fourth grade who couldn't do that. But, in fact, what skilled teaching involves is responding when students don't understand the material...

Knowing the mathematics is not enough. Teaching requires more than simply marking students' answers right or wrong. To be effective, teachers must be able to diagnose students' misconceptions in order to provide the support necessary for students to develop mathematically. Dr. Ball provides three examples of possible errors teachers might encounter from students multiplying 49 and 25:

When she asks the legislators how the students might have arrived at these answers, ... watch for yourself (it starts at about 2:30).

Dr. Ball does an effective job of pointing out that teaching is more than knowing and sharing content. However, this example focuses almost entirely on the "upper-half" of the Teaching-Learning Cycle: Assessment and Evaluation. Developing a plan for addressing the student-misconceptions and implementing that plan with a class of fourth graders raises the level of difficulty even higher. I hope these lawmakers now understand that there is more to teaching than what they experienced from the student-side of the desk.

Finals! Huh! Yeah! What are they good for?

We just finished finals at Grand Valley. Coincidentally, this week of exams overlapped with the release of the second Hobbit movie; this might explain why these lines (modified from this poem) were going through my head Thursday as I walked around campus:

I am not trying to suggest that like the One Ring, final exams are evil and must be destroyed. However, I do wonder about the wisdom and unintended consequences from placing so much emphasis on a single assessment event.  Rarely do final exam scores make that much of a difference on final grades. Yet, there is so much stress associated with final exams, for both the students and the teachers, that it got me to wondering if these tests are worth it.

During the MTH 323 final event (we did not have a written exam), I asked the teachers to consider whether or not our school ought to have final exams and why. This is the course for preservice teachers that I designed to reflect what it looks like to be a math teacher in a K-8 school. So we created a "school" and went about developing a curriculum. Therefore, it seemed reasonable to discuss the final exam policy we would have at our school.

Grade-level groups talked about the question and then shared their thoughts and rationale. I added my two cents, but mostly these points represent the teachers' perspective.

• We need to have some sort of culminating experience as a way to identify what a student has mastered and possible weaknesses in the curriculum.
• The experience needs to be different than the traditional exams that represent a cram and purge approach that does not result in lasting learning.
• Like it or not, exams are a part of our culture and we would be doing our students a disservice if we fail to provide some experience with these types of assessments.
• Perhaps we could give the exams a couple of weeks before the end of the semester so those who need to can make revisions and the rest can extend their learning.
• Whatever we have them do needs to be meaningful.

It was the last point that resonated with me. How do we ensure that the culminating experience is meaningful - that it fosters the upper levels of the Engagement Continuum and not just compliance?

Developed by MTH 323 Teachers to monitor their engagement
Based on Engagement Taxonomy of Morgan and Saxton

At breakfast this morning, I asked a colleague what he thought final exams were good for. He said, "To see if students have synthesized the information addressed in the course." After a pause, he continued, "Of course, those are the questions they often struggle on most, which is discouraging."

"Why do we give all the other questions, then? The ones that focus on recall. Why not focus on the synthesis questions?" I asked.

"Probably because it's what we've always done," he responded. Neither of us were satisfied with this answer, but finals week was over (it was, after all, final) so there was nothing we could do about it now except think about what we might do different the next time around.

Will this be on the test?

Sure, I can be smug when it comes to assessment. In my defense, I have been considering different ways to assess learning for over twenty years. Much of this work has involved using alternative assessments, but I am not above using more traditional approaches, like tests, especially when they involve some twist. 

Because of the negative associations many students have with math tests, I try to make it clear that my tests are used to support learning. Each test is merely a benchmark that monitors our progress toward established learning goals. We use the results to determine what we can do and what needs more practice. Yes, at some point a grade gets put in the books, but there is no need to worry until that time comes.

That is what I tell them, but often this is what they hear:

In case you are unfamiliar with Game of Thrones, some context.

So, it's no surprise when I start hearing this (of course I add the last bit in my head):

I understand that this is a conditioned response. At some point it worked; it made it easier to decide what to memorize in order to get a good grade. However, I want learners to decide for themselves what's worth remembering and to consider ways to make that learning last beyond the test.

So here's how I want to respond from now on (turns out I'm smug and snarky - who knew?)

Does it make you more curious about the material?

Are you likely to try to learn more about the topic on your own?

Would you consider putting the ideas in a concept map in order to make connections?

Can you see yourself discussing the subject with others in order to talk your way to understanding?

Might you examine your thinking in light of the information and consider refining it?

In all seriousness, I recognize test-anxiety as a real issue among many of my students - an unintended consequence of high-stakes testing in the past. I hope my efforts at humor do not come across as flippant. My goal is to reframe "Will this be on the test?" as an opportunity to examine alternatives to the binge-and-purge approach to studying. Because if I am satisfied with learning that leaves instead of learning that lasts, then why even bother?

Did you watch TEACH?

Friday night I watched the documentary TEACH with two other educators. Dad taught high school math and science for over three decades. My wife was an elementary teacher for 15 years. I taught middle school math and computers for seven-and-a-half years, making me the least experienced of the bunch, but we could all empathize with the stories being told.

And that seemed to be the point - to elicit emotions from the viewers. From the film's website:

Davis Guggenheim's award-winning documentary reveals the human side of the story: showing what it takes to survive the first year teaching in America's toughest schools. (emphasis mine)

Based on many of the Tweets posted to #TEACH, the movie accomplished its goal and touched many of those who watched.

It is important to recognize this as Mr. Guggenheim's vision for the film and understand that it must have influenced how he edited his footage. Just as a director who is adapting a book for a movie must consider what to keep and what to cut, Mr. Guggenheim could not hope to communicate the complexities of teaching in 35 minutes. Consequently, he presents a picture of teaching that diminishes the importance of certain aspects of the practice in order to focus on the teachers' stories.

I started this blog as a way to chronicle my teaching practice. In particular, I wanted to focus on the framework known as the Teaching-Learning Cycle. To Mr. Guggenheim's credit, his documentary acknowledges each of the Cycle's phases. However, it gives too much attention to assessment. Here are some examples:

• Several scenes where teachers and administrators wait by Scantron machines for testing results;
• A teacher expressing her concern about whether the test will "validate" her efforts; and
• Another teacher sharing her frustration that students were bubbling in the wrong answers when they should have known better.

Given the current national obsession with test scores, it is understandable why assessment became the focus of the film. However, it is also important to see how the assessments added tension to the storytelling. We want to find out, "Will the students pass the big test?"

While this plot device was effective in driving the stories in TEACH (see also Stand and Deliver and Lean on Me), it is unnecessary. As Kathy said, then Tweeted, at the end of the film:

As I recall, these themes have been successful in many other stories.

Part of the problem is that the assessment storyline is also taking over the current education reform narrative. The intent again seems to be to create unnecessary tension to manipulate people's emotions. My concern is that we are being swept up in that emotion, thereby, attending solely to teaching and testing while diminishing what's really important - learning.

Where should we look?

It is morel mushroom season in the Upper Peninsula of Michigan. Finding these elusive, tasty morsels can be as challenging as it is rewarding. Mushroomers are secretive about where they hunt, but even if you know where to look, they can be hard to find among the leaf litter.

On a recent walk through the woods, I began to contrast the difficulty in finding morels to assessment (finding learning). I did not want to lose my thoughts, so I tried to place them in a mnemonic: ASSESS. Such an approach can be useful for memory but it has limitations (e.g. a set number of words that sometimes requires synonyms that don’t quite match the meaning of the original words).  Still, this blog is intended to share my thinking and I hope that you will challenge it and extend it in the comments.

Authentic

Learning happens in authentic settings (notice I did not say it had to be real-life), therefore, that is where we need to look for it. You will not find mushrooms where they do not grow. A test can be a learning experience but these can be difficult to write; that is why a prefer projects and portfolios as ways to demonstrate learning.

Standards

We need to know what we are and are not looking for when it comes to learning. When it comes to mushrooms, not knowing what you are looking for can be deadly. Metaphorically speaking, the same can be said about learning. Focusing on memorized facts can “kill” a person’s interest in learning.

Unsafe

Slow

It is difficult to observe learning under unreasonable time constraints. Trying to identify learning using 10 questions on a 60-minute test is akin to trying to spot mushrooms on a fast-moving motorcycle. Someone might be able to do it, but I cannot. It takes time and patience to assess learning.

Experience

With experience, it becomes easier to know where to look, when to look, and what to look for; this is true whether you are talking about mushrooms or learning. Lacking experience we tend to rely on tales or traditions.  We get stuck using the same types of tests that we encountered as students without considering other places we might find learning.

Support

Until we gain the experience we need to accurately assess learning, we ought to seek out others with more experience for support. It takes a trained eye to find mushrooms and a skilled teacher to help others find them for themselves. A teacher who has used effective assessments for identifying learning cannot simply pass these along to other teachers. The mentor teacher must share the process as well as the product. Without the appropriate support, the apprentice teacher can fall into the old rut of using the product to assess what is easy instead of what is meaningful.

Spontaneous

One never knows when learning might be demonstrated, so teachers must always be on the lookout. If we only look where we have always looked, we may be missing the morels in our own backyard. Teachers who use observational assessments often are able to identify learning that might not show up on a test. Furthermore, if teachers inform students of what learning is expected, the students can help to demonstrate that learning occurred even if it happened in ways the teachers did not anticipate.

This is what I thought about as I hunted mushrooms and considered the parallels to assessing learning. I certainly refined some of my thinking as I wrote this post, but the key point remains: we need to rethink our assessment practices if we are to find true learning. Hopefully, ASSESS will remind me of what this shift might entail for me.

In what ways do you think assessment might need to change and how will you remember to make these changes when the new school year begins?

WTH?

The analysis of pointers is pointless.

Eckhart Tolle (at 2:12)

Yesterday, I sat with a group of concerned parents and teachers as they discussed what was happening at their children's school. They had chosen this school because it was a public (charter) school dedicated to developmentally appropriate methods of teaching and learning. But recently the school had been implementing more test-prep based on the recommendations of certain "experts" who were concerned about the school's standardized-test results.

You see, when Michigan decided to change their cut scores on the state test the school's pass rate had plummeted. I tend to think of this as a manufactured crisis meant to sell more test-prep programs but that rant may have to wait for another post. The consequences to these parents and teachers are real - and heartbreaking - and demand the attention of this post.

The parents and teachers had selected this school specifically because of its  commitment to being something other than a "factory" school; a commitment to being a school where natural learning was valued above all else. Now, however, the school seemed to be succumbing to the push to standardize teaching and learning. Parents had attempted to express their concerns to school leaders with no success. This group was now thinking about alternatives - home schooling, starting a private school, starting their own charter.

I did my best to explain what I understood to be the some of the pros and cons of each of these options, all the while empathizing with their situation. These were not people of means who could afford to stay away from a job to home school their children or pay the tuition at a private school. They wanted to know what assurances they would have that a new charter school would not result in the same heartbreak somewhere down the line. Unfortunately, I had no assurances to give.

It was energizing to find these parents so passionate about developmentally appropriate education but I left the meeting angry. I am angry that we have developed this unhealthy obsession with standardized-test results. I am angry that parents committed to learning, not test scores, find themselves looking around for alternatives and find nothing available/affordable. Pardon the violence in this video (and the earlier comic meme), but the situation has left me thinking, "What the heck!?!" and makes me want to slap someone. [Fortunately, I chose to use my words instead.]

[Important part starts at 1:15]

We need to quit focusing on the finger (test-scores) and begin looking to the moon (learning). Our current approach is shortsighted. And if we continue down this path we will not have the educated citizenry we need to survive.

Can we follow the trail?

It snowed last night - a lot.

from NOAA

This morning, as I began the process of digging us out, I began noticing how the snow kept a record of my progress.

This got me thinking about assessment and monitoring the progress of learning. Too often, it seems that the assessment information we collect provides a picture of where learners are at but not enough about how they got there. It's as if they magically appeared in the snow.

But moving through the snow forces us to leave tracks. If the tracks are clear enough, we might be able to follow the path. One of the reasons I have learners write metacognitive memoir is that it leaves a trail for me to follow - it makes their thinking visible.

Sometimes the trail can be confusing, however. It's not always clear the path that was taken or even which footprints belong to the traveler and which were left by someone else.

That's why I often like to interrupt learners in the middle of their efforts -- to catch them in the act of thinking.

It is also the reason why I like Ewan MacIntosh's idea of engaging learners in problem finding rather than problem solving. Perhaps a pristine area of snow will make it easier to follow their path.

Thanks for indulging me this metaphor. It was a great day to play in the snow and think about assessment.

What if we gave them the answers?

My experiment of teaching a course where preservice and inservice teachers share two hours of class time has been going well. (I introduced the concept here.) In fact, one of the preservice teachers said recently, "I wish all of my education classes had classroom teachers in them." I believe the following example explains why he feels that way.

That same preservice teacher was part of a small group (along with an inservice middle school math teacher and a community college instructor) who were analyzing middle school students' work on an algebra assessment. They were talking about how difficult it is to get students to share their thinking especially once they assume they have arrived at an answer. I concurred and explained that this was one of the reasons I focused on using metacognitive memoirs, saying, "I know the answer but I don't know what you're thinking." This gave the inservice middle school teacher an idea.

He wondered what would happen on the next test if he gave the answers and asked the students to focus on their thinking. A few days later, I (along with the preservice teacher and the community college instructor) received the following email:

Hi, I gave a test yesterday in my 8th grade math class and I gave them all of the correct answers at the beginning of the test to see if it would improve the work that they showed and how well they explained their thinking.  They were shocked, but they actually caught onto the idea quickly, I didn't even have to tell them why I was giving them the answers, they came up with it themselves.  While the test responses weren't perfect, students did a MUCH better job sharing their thinking than they ever have before.  I am excited about how this turned out and I anticipate doing this more often in the future.

I asked the teacher if he would mind me sharing this experience and the test on my blog and he agreed. Not only that - he also provided how he implemented this new approach, a sample of students' work, and students feedback.

After handing out the test, the teacher began:

• Teacher: "Listen closely. This is a test. You know the rules as far as talking, etc."
• Teacher begins reading off answers.
• Students are following directions, no questioning until after the page flip.
• Student 1: “Why are you telling us all the answers?”
• Student 2: “I like this!”
• Student 3: “Don’t stop him.”
• Teacher keeps reading answers. There is no contesting of getting the answer and the kids keep filling in right answers for remainder of test.
• Student 4: “I don’t understand this...”
• Student 5: “Why did you just give us the answers?”
• Student 6: “Do we have to explain what we did for the answers you just gave us?”
• Teacher: “You’re not going to get any credit for having the right answers. You’re only going to get credit if you can explain how you get the right answers. So all of you are starting right now with all the answers and a 0%.”
• Student 4: “I like the other way better.”
• Teacher: “Let me just say one more time...You all have the right answers, so the explanations are where you can earn the points. With that in mind, go ahead.”

Here is what the test looked like after the teacher had read the answers.

And here are some examples of what students wrote:

After the test, the teacher asked for students' feedback on this approach to assessment. These represent some of their responses:

• "My head hurts because I actually had to think."
• "I realize now that I've never done a very good job explaining my answers."
• “This was like an English test!”
• “It took forever...like, I know what I want to say but I can’t explain it.”
• “Didn’t like volume of writing and repetition.” (Felt like there was too much writing and they were answering the same questions over and over.)
• “Didn’t see the point of giving out the answers because you have to do all that thinking to get the answer anyways.”
• “Liked it. I always spend time figuring out the problem so I don’t explain. This helped cut out the calculation step.”
• “Didn’t like because I don’t like explaining myself.”
• “Would have prefered to find answers instead of trying to explain because sometimes I can just get it (in my head).”
• “Liked having answers, otherwise I spend a lot of time trying to get the answer. This way I know the answer is right.”

I hope that I was able to adequately articulate this approach to assessing students' mathematical thinking. If you have questions or ideas, please leave them in the comments and I'll be sure to pass them along. We have 8 more weeks together in this course. I'm looking forward to whatever else they come up with in that time.