NCTQ Yearbook Earns a "D"

NCTQ EARNS A "D" FOR ITS GRADING OF

TEACHER EFFECTIVENESS POLICIES

Press Release

February 1, 2014 (Michigan) – The STAB¹ has released its first annual Report Card on Education Policy Grading, which judges the grades given by the National Council on Teacher Quality (NCTQ) in the 2013 State Teacher Policy Yearbook: Michigan (report). While many different factors went into the grade earned by the NCTQ for their report, we realize that the media has limited space available to analyze our findings (given all the recent news about Justin Bieber and all) and decided to boil it down to a single grade that is familiar to the public².

The "D" earned by NCTQ’s Yearbook communicates that it includes some nice graphics but it is incoherent in its analysis³. Furthermore, it appears that their expectations are based solely on opinion with no references provided to support their positions. In fact, some of the expectations seem to be biased by a prior position held by the NCTQ President, Kate Walsh⁴.

Essentially, the NCTQ’s Yearbook does not match our vision of sound educational policy. If it had, we would have rewarded the report with an “A” grade. Because it did not align with our ideology, we will shame it with a low grade that we trust the public will interpret as unsatisfactory. Once the NCTQ comes to view educational policy the same way we do, then we anticipate their grade will improve⁵. After all, isn’t the role of grades to manipulate others to match our view⁶?

¹ The Sarcastic Teachers of America Board

² We all agree on what an "A" means, right?

³ You can’t want higher standards for teacher preparation and more flexible pathways to teacher certification⁷.

⁴ Kate Walsh started the first alternative certification program for teachers in Maryland. (see ³)

⁵ We suggest they keep the graphics – very nice.

⁶ No, it is not! But that seems to be the idea behind grading schools by groups like the NCTQ.

⁷ Because alternative teacher certification often means little teacher preparation coursework.

⁸ Are you still reading this? Why?

What did/does the 'A' say to me?

In the previous post, I used the song What Does the Fox Say? to frame a discussion about what grades communicate to various stakeholders in education - in particular students and teachers. It is my view that we do not share a common understanding of what a grade means and this impacts learning. I suggested some of the different ways we interpret an A by modifying the song's lyrics. Readers responded in the comments with what an A grade meant to them as a student and what they hope it says to their students as a teacher. Now is the time for me to share my perspectives about what did/does the A say to me.

When I got an A, I thought that I had pleased my teacher. I distinctly remember having a conversation with a friend who was struggling in school about how I achieved success. I tried to find out what the teacher wanted and then went about meeting that vision. The grade I got would tell me how close I came to giving the teacher what he/she had in mind. It did not matter the subject, the teacher was the all-knowing arbiter of my work. For me, school was not so much about learning as it was mind-reading.

As a new teacher, I simply flipped this perspective. An A meant that the student had done at least 90% of what I expected (i.e. what I would do). To my credit, I did not want my students to read my mind so I made my expectations very clear. I got a lot of student-work that looked just like my work. Instead of mind-readers I was fostering mimics.

Now, I see an A as representing what Joyce and Showers (2002) called Executive Use. The student has demonstrated complete content competency (I was uncomfortable with idea that there might be a 10% gap in a teacher's knowledge) and an ability to analyze under what circumstances the learning could be applied appropriately (phronesis) or how to adapt it to new situations. Granted, because I mostly teach teachers, this standard might be easier to implement now than when I taught middle school math. Still, I have applied a similar idea in a College Algebra course with some success - it was a tough sell.

Basically, I want an A to say to students that they have achieved sustainability in the topic being graded. They can apply what they have learned beyond what we talked about in class, and they can learn more on their own if needed. The teacher (me) has become obsolete.

What does the 'A' say?

I was introduced to this little ditty for the first time last week. 
[Warning:  possible earworm]

Dog goes woof
Cat goes meow
Bird goes tweet
and mouse goes squeek

Cow goes moo
Frog goes croak
and the elephant goes toot

Ducks says quack
and fish go blub
and the seal goes ow ow ow

But there's one sound
That no one knows

What does the 'A' say?

Well-done-well-done-well-donedonedadone!

Well-done-well-done-well-donedonedadone!

Well-done-well-done-well-donedonedadone!

What does the 'A' say?

You got 90% or better!

You got 90% or better!

You got 90% or better!

What does the 'A' say?

De-de-de-de-de-distinguished!

De-de-de-de-de-distinguished!

De-de-de-de-de-distinguished!

What does the 'A' say?

Ah-ah-ah-ah-ah-ah-ah-ah!

Ah-ah-ah-ah-ah-ah-ah-ah!

Ah-ah-ah-ah-ah-ah-ah-ah!

What does the 'A' say?

Big bold type

Pointy head

Causing joy

and spreading dread

...

So that's enough of that. But in all seriousness, the song reminded me of a project I have been interested in for some time: what does a grade of 'A' communicate to students, to their parents, to other educators, to the community? I have often wondered if we are all on the same page when it comes to grades and what they say. At one point, I thought about putting a questionnaire in the mailboxes of my colleagues asking this question, but for some reason I never got around to it.

Until now.

In the comments, please share what an 'A' said to you as a student and what you hope it communicates to your students as a teacher. I'll share what I really think the 'A' says after you all get the ball rolling. Thank you in advance for your participation.

Update: Thanks everyone. My response got a bit long for a comment - so, for what it's worth, you can find it in the next post.

Was the scoring consistent?

The following is a statistics workshop I have used in a class for preservice K-8 math teachers. Because the topic being addressed, Mean Absolute Deviation (MAD), represents a specific procedure, parts of this workshop are more direct than what I typical do in a lesson. Still, I try to leave enough space for learners to explore the procedure and make sense of it.

The goal of this activity is to consider how to quantify consistency while addressing the idea of inter-rater reliability. It is based on the article Means and MADs and is intended to help introduce the concept of deviation using a measurement that is more accessible to middle grade learners. I updated the activity to reflect the upcoming Common Core State Standards (CCSS).

Grade 6 CCSS in Statistics and Probability [6.SP]

• 6.SP.2. Understand that a set of data collected to answer a statistical question has a distribution which can be described by its center, spread, and overall shape.
• 6.SP.3. Recognize that a measure of center for a numerical data set summarizes all of its values with a single number, while a measure of variation describes how its values vary with a single number.

Schema Activation: Turn and Talk

With your partner, discuss which set represents the greatest consistency.

Focus: Quantifying Variation

Consistency is a major concern when it comes to grading.  We want the scores that different teachers assign a piece of work to be relatively close. While sometimes variation is obvious, quantifying it can be helpful in identifying inter-rater reliability. In this workshop we will learn about a particular measure of variation called the Mean Absolute Deviation that can be used to describe how values vary with a single number.

Activity: Small Group

The table and graphs below show the scores one group of teachers assigned Ross for his exploration project in each Math Thematic assessment category: Problem Solving (PS), Mathematical Language (ML), Representations (R), Connections (C), and Presentation (P).

PS	ML	R	C	P	Total
3	4	3	5	5	20
5	4	5	1	4	19
3	4	4	5	5	21
5	4	4	5	2	20

Predict

• In which category do you think the scores were most consistent?
• In which category do you think the scores were least consistent?
• Which representation (table or graph) was most useful in addressing these questions? Why?

Determine

Find the mean distance (direction does not matter – why?) each data point is from the mean of the data set and compare it with your predictions.  Which representation (table or graph) was most useful in finding this Mean Absolute Deviation? Why?

Reflection: What? So what? Now what?

1. What were your results?
2. So what do the results say about consistency?
3. Now what are some ways to improve scorers’ inter-rater reliability?

What does the grade say?

The following workshop is based on an activity used in a class for preservice elementary math teachers. The original intent of the activity was to reinforce measures of centers while considering philosophical issues associated with grading. It has been updated to reflect my current understandings about teaching and learning and the Common Core State Standards (CCSS).

Grade 6 CCSS in Statistics and Probability [6.SP]

Develop understanding of statistical variability.

Understand that a set of data collected to answer a statistical question has a distribution which can be described by its center, spread, and overall shape.

Schema Activation:  Journal Jot

What does an A grade communicate?

What does a B grade communicate?

What does a C grade communicate?

What does an F grade communicate?

Focus: Use the statistics that we have explored so far (mean, median, mode, and range) to describe the learners’ scores provided below.

Activity:  Small Group

Math Scores (Each project is out of 20)

Name	One	Two	Three	Four	Five
Anita	15	15	15	15	15
Ben	6	9	20	20	20
Charlie	20	20	15	10	10
Dale	13	14	15	16	17

Reflection: Think-Pair-Share

• Based on your analysis of the scores and your responses to the Journal Jot, assign a math grade for each learner.
• Please provide some insight into how you assigned these grades and what you hope they communicate.
• Historically, teachers have used the mean of a set of scores to assign grades. Do you think this is an appropriate strategy in this case? Why or why not?
• In general, why do you think teachers typically average scores to determine grades?

Exit Ticket: Now what grade do you think each learner earned and why?