Value-Added Fears

In recent months there has been a lot of talk about value-added assessment--a method of tracking student growth and using that data to evaluate the effectiveness of a teacher. This sounds like a really great plan, except for the fact that the only data that seems to be used in value-added assessment is data garnered from standardized test scores. I have blogged, tweeted and talked to both parents and educators about why I feel standardized test scores are not a reliable enough measure to gauge student growth. Here are my arguments:

• Standardized tests are norm-referenced. The norms for these tests are supposed to represent a microcosm of U.S. society. If the school where students are being tested does not match that microcosm, the norms are invalid for the population taking the test. For example, my children are biracial. They attend a school that is 64% culturally/linguistically diverse. If their performance on a test is compared to the normed performance of students in norm-group that is only 4% culturally/linguistically diverse, the results are not valid. The group taking the test does not have the same characteristics that earned the norm-group their scores. 
• Standardized tests do not necessarily test students on topics they have learned in the classroom. Test-makers are not educators or administrators. They are corporate business-people (who may have been educators at one time, but chose to leave the profession.) Corporations that make tests decide what each 3rd grader in the country should know and then develop the test based on their opinion. A school district or a state may adopt their test to measure growth because they feel it most closely matches their curriculum, but the test is not designed specifically to measure what their students are learning. For example, a district may choose to use the Iowa Test of Basic Skills to measure student growth. The third grade computation section may include problems in long division. Yet, in my child's school, they do not have the opportunity to learn long-division until 4th grade. The test-makers think that long-division is something every third grader should know, but my school district does not agree. As a result, many children will do poorly on Math Computation because they haven't learned the material. Poor scores on a standardized tests do not reflect poor teaching or lack of growth in this situation; they reflect the use of an inaccurate tool to measure.
• People who use standardized tests assume that standardization is actually occurring. This means that all teachers are reading the directions exactly as written and that all testing environments are similar. There is no way to ensure that standardization is actually occurring. Testing environments vary greatly. To be truly valid, we would need to ensure that each test-taker came with the same amount of sleep the night before, the same amount of food in their bellies, and even the same temperature in the testing environment! What if one room of test-takers is roasting and another is freezing? Wouldn't those factors affect your ability to perform on a test? We cannot control for all of the factors that affect student performance. In science, if you cannot control all of the variables in an experiment, the results are invalid. I argue that since we cannot control all of the factors in mass-testing, those tests are not actually standardized; if they are not standardized, the results are not reliable.

On a personal or gut-level, I have several other problems with using value-added assessment. 

• I am an alternative educator. My students are deemed "at-risk" of dropping out due to a host of social and familial issues. I am good at my job, but my student's test scores will never reflect that I am good at my job. No matter how I much I stress the importance of the test or how intelligent I know my students are, there is always at least one who fills in the bubble sheet with pretty patterns.  It is not an accurate assessment of what that student knows.
• Adjustments are not made for students with disabilities. I have students in 11th grade who read at the 4th grade level. They will never be proficient in the terms laid out by NCLB. That doesn't mean that the student isn't achieving or that they will be unsuccessful in life. It means that they will never do well on standardized tests.
• Bad teachers can have students with good test scores. There are teachers who have been teaching for a long time. We all know them. They get through the material at all costs without taking the time to build relationships with their students. They complain in the teacher's lounge and count the days until their retirement package kicks in. In a value-added situation, if the students in those classrooms achieve well on standardized tests, that teacher will keep teaching. Meanwhile, a teacher who has built relationships with his/her culturally/linguistically diverse, socio-economically disadvantaged, or special education students could lose his/her job because the invalid/unreliable norm-referenced test scores say that his/her students haven't shown enough growth.

I do agree that we need to find a way to evaluate teachers more effectively. But I really don't think that value-added assessment is the way to go if standardized norm-referenced tests are going to be the only measure used.  Let's put into place a system that uses valid criteria. A valid system is one where local standards are tested, students and parents are asked for evaluative input, peer and administrative teams are conducting classroom evaluations. That kind of information is much more valuable in measuring both the growth of students and the effectiveness of teachers than any norm-referenced test.