What does the research really say? (Math Edition)

The American Educational Research Association (AERA) recently published a study, Which Instructional Practices Most Help First-Grade Students With and Without Mathematics Difficulties?, that caught the attention of several media sources. Many of these sources misrepresented the research with headlines like:

• Here's How Not to Teach First-Grade Math
• Drill, Baby, Drill? Study Finds Worksheets, Practice Improve Math Skills More than Fun Stuff

Other outlets were truer to the research:

• Explicit Instructions Works Best for Struggling Math Students
• Music, Manipulative Are Fun, But Basics Best for Struggling Math Students

However, these pieces still ignore some of the nuances of the research. To his credit, one of the author adds some perspective in this short video.

Still, achievement in mathematics needs to be defined. From the paper:

We used the ECLS- K’s Mathematics Test to estimate children’s mathematics achievement. ... The test was based on the National Assessment of Educational Progress (NAEP)’s specifications. (p. 6)

If your idea of mathematical achievement is simply scoring well on a standardized-test, then direct instruction probably is the most efficient way to prepare students to achieve. 

For example, when putting together a piece of furniture from IKEA, it makes sense to follow the directions provided explicitly. Why would I explore assembling our coffee table when the directions provide step-by-step instructions on how to achieve my desired outcome? How else can I be sure that my table will look exactly like the one I ordered?

Item Number: 801.762.84

And, therefore, like every other IKEA coffee table, 801.762.84.

But what if I want something unique? In this case, the directions might not apply (or even exist). My stepson made this coffee table for us out of a part of a tree that was from my childhood home. It is one of my most prized possessions.

Now, he is a talented woodworker who can (and has) put together a lot of furniture using directions. When he wants or needs to be creative, however, he can be. For me, that is true achievement.

We need to apply the same approach to teaching mathematics. Focusing solely on achievement based on content assessed through a standardized-test misses an important element of mathematical ability - thinking beyond what can be explicitly taught. Dr. Yong Zhao does a good job of addressing this disconnect in this post. So I leave you with this:

Source

What does the research say? It depends on what one is looking to achieve.

Update (7/4/14): And now an example related to teaching language arts has surfaced.

Where should we focus our efforts?

During the Second World War, Allied bombers were sustaining heavy damage from flak and bullets while flying missions over Germany. Many of these bombers failed to return to their bases in Britain. A study was conducted of the damage done to the planes that did make it back which resulted in a proposal to add armor to those areas that had sustained the greatest number of hits. This would have been a costly undertaking as it meant spending valuable resources on refitting the planes. Furthermore, the extra armor would make the bombers heavier and less maneuverable. Had this effort been carried out, it most likely would have failed.

From Digital Roam

Fortunately, a Hungarian mathematician, Abraham Wald, took another perspective. Why not reinforce those parts of the bomber that had not been hit? He had noticed a different pattern in the shot-up planes. Each had avoided damage in similar spots, and Wald reasoned that these were vital areas that needed to be protected. After all, these bombers had successfully returned from their missions in spite of the damage they had received.

The Allied Command followed Wald's suggestions. (A collection of his memos can be found here.) However, I believe some loud and persistent voices in education reform are missing the wisdom of his message. Recent videos developed by Students First exemplify this misunderstanding of what is vital to the success of the U.S. education system. (Students First is the group headed by former Chancellor of the Washington D.C. school system, Michelle Rhee.)

The video shown here suffers from the same problem that distracted the original researchers of the WWII British bombers. Students First is focusing on the testing data (where we were "hit") and ignoring what is really important in education. To continue the Olympic metaphor, it would be like judging our divers against those of other nations simply based on how long it takes to climb the ladder. It does not matter, which makes it a distraction, and a potentially costly one.

Dr. Yong Zhao explains the problem with this focus on using only tests to compare our educational system to other country's in his keynote at the AACTE 2012 Annual Meeting. The keynote starts about 18 minutes into the video.

Education reform groups like Students First say that our current standardized test results show that our education system is full of holes. The truth is that by that reasoning, "American education has always been bad." [32:33] Poor test scores are not a recent phenomenon. "We have had over a half-a-century of bad education according to some measures." [40:49]

Much like Wald did, Zhao interprets these "hits" differently. In the figure below, the countries on the left are the top ten countries in the Programme for International Sudent Assessment (PISA). Those on the right represent the leaders from the Global Entrepreneurship Monitor (GEM). 

"As you can see the two lists don't go together. Countries that have higher PISA scores have lower entrepreneur capabilities." [48:40] In other words, focusing on high test scores means embracing conformity at the expense of what has always been our national strength - creativity.

Unfortunately, the current wave of education reform focuses on the easy to observe gaps rather than our subtle and vital successes. Yes, we have holes but these holes have not kept us from being successful. Still, we seem committed to embracing standards-based test-prep while eliminating the aspects of our educational system that nurture creative thinking. If we do not listen to the wisdom of Wald and Zhao, we will spend a lot of resources to fix what is not really important and make our education system less flexible as a result.