Current Issue Back Issues Cards
Issue 12, Fall 2014
Click here to have this issue shipped directly to you.


From Socratic Seminar to Space Science,
   Brian Delgado
Assessing Quality Teaching,
   Kathleen L. Gallagher
72-Mile Classroom,
   Erina Chavez
Failure: The Mother of Innovation,
   Michael Martin & Christine Hoyos
Doing the Project First,
   Kelly Williams
When Teachers Exhibit,
   Joan S. Soble
Sizzle and Steak,
   Alec Patton
Designing a Collaborative Learning Environment in Math,
   David Corner
Home Visits,
   Melissa Agudelo


Cards:
1: Toy Story
2: Practicing English by Playtesting Games
3: Wild Pond Protectors
4: In Their Skin
5: Zoomanity
6: Creative Collective:
    An Integrated Project of the Arts

7: Making New Members Feel Welcome:
    A Design Thinking Challenge

8: Food for Thought


HTH GSE » UnBoxed » Issue 12 » method

Assessing Quality Teaching











I always strived to be an excellent teacher, but it didn’t always come easily or naturally. I am not sure I ever would have realized that my instruction could be different if I didn’t read in the company of wonderful colleagues who cared about teaching and learning. In the first professional book study of my career I read The Art of Teaching Writing (Calkins, 1986). My colleagues and I used the book to transform our beliefs about writing instruction. Over the course of my career, many other professional texts written by powerful, practicing teachers taught me how to bring meaning to the content I was teaching. Marilyn Burns did it for me in mathematics. In addition to her books, I was lucky to have access to a group of gifted role models who showed me how to bring her ideas alive to create a classroom that buzzed with the joy of mathematical thinking and reasoning. These authors who wrote passionately about their craft, combined with the interactions I had with colleagues in professional learning communities, inspired me to continually transform and improve my professional practice.

After 17 years in the classroom I ventured into the arena of instructional coaching. In my first year, I quickly realized that everyone did not have access to the resources and role models I had experienced. Working at two schools and in about 40 classrooms, I calculated that only about 25 percent of teachers were actively working to improve their instruction. These top teachers were the ones who most desired professional collaboration and support, but as a coach, I was challenged to also support the teachers who did not have an intrinsic desire to change. Many of these teachers seemed to feel that the potential for change in their classrooms and students was sourced outside themselves. In the course of learning to do this work, I realized that there really weren’t any concrete tools for observing and providing feedback to teachers in ways that allowed them to reflect on the quality of their instruction.

The Art of Instructional Measurement

Accurately measuring the contributions that effective teachers make to children’s learning is important not only to understand the characteristics and attributes of high-quality teaching, but also to assist the teachers who are having difficulty (Weisberg, Sexton, Mulhern, Keeling, Schunck, & Palcisco, 2009). One problem with standardizing a practice for the sake of measurement is that as soon as one begins to try to define it, we reduce it to its most technical aspects. We look at things like lesson plans, standards, learning objectives and test scores because those are easy artifacts to reference. Observations are strengthened when we consider the quality of teacher questions, student responses, and work products, but unfortunately, methods for evaluating these processes are implemented with varied levels of reliability and little or no consistency over time (Darling-Hammond, Amrein-Beardsley, Haertel & Rothstein, 2012; Hargreaves & Shirley, 2009). So we have the dilemma of measuring ineffectively or not measuring at all and both have the same deleterious effects on the improvement of teaching.

In order to effectively capture what Richard Elmore (1996) calls the instructional core of teaching, we have to be able to examine the constant and ongoing interactions that happen between teachers and students as they engage meaningfully with content. Since almost any person can walk into any classroom and in a minute or two assess whether or not learning is taking place, I thought it might be possible to describe that process using Elmore’s instructional core as a guiding framework. I began by inviting the teachers at my school to join me in an action research project. Twelve teachers gave up a few prep periods to observe our students learning. We collected examples of what students looked liked and sounded like when strong instruction was taking place. We generated a long list of student behaviors associated with student engagement. Our list included: students raising their hands excitedly, asking questions, learning cooperatively, enjoying their work, working out problems, reading for meaning, writing, persisting through difficulty, listening to the teacher, listening to each other, making a personal investment in the learning, presenting to their peers, getting feedback from the teacher, getting feedback from their peers, using materials appropriately, being creative, thinking hard, talking about their work, being accountable, sharing ideas, using manipulatives, and behaving responsibly.

Three general categories emerged from our observations in classrooms where learning seemed to be the most powerful. Students were actively participating. They were thinking critically about the content they were learning. And they were talking a lot to each other about their work.

We began experimenting with ways to measure these variables so as not to interfere with the creative work that teachers do to make learning come alive. Participation seemed the easiest to measure. We learned about Depth of Knowledge (DOK) levels for assessing the cognitive demand of learning tasks and used that to assess the thinking component (Hess, Jones, Carlock, & Walkup, 2009; Webb, 2007). As we studied classrooms where students were engaged in collaborative, content-focused conversations, we realized that the most artful teachers facilitated talk in their classrooms that gave students many opportunities to use academic language in contexts that made sense to them. We came up with a four-point rubric for each of the variables and noticed that if we tracked the variables throughout the course of a lesson, we could see what teacher actions were associated with the highest and lowest levels of student engagement.


  

Teacher participants in many schools allowed me to begin using the measurement protocol to assess their instruction. As teachers taught, I scripted as much of the classroom dialogue as I could and did my best to capture the essence of the tasks on which students worked. For every five minute lesson segment, I assessed the class’s participation, the cognitive demand of the task students were engaged in, and the academic language that students were using while they were learning. We figured out how to graphically display the feedback to show how each of the variables changed as a result of the instructional decisions the teacher made. This was the birth of a process we called the Protocol for the Assessment of Quality Teaching, or PAQT (see below). More recently, as teachers strive to align their instruction with Common Core State Standards, we realized that feedback was critical to the process and the Protocol for the Assessment of Common Core Teaching (ProACCT) was born.


  

In the 7th grade pre-algebra class referenced in the table, 40% of the students were English learners, 5 students had IEPs, and 60% had scored Below Basic on the California Standards Test that had been given the previous year. While this lesson received a ProACCT score of 93, the 7th grade lesson next door, which was taught to students with similar demographic characteristics, received a ProACCT score of just 28. This means that only a small percentage of students participated, the cognitive demand of the task was very low, and the students had few opportunities to use academic language in contexts that made sense to them.

The Characteristics of Quality Teaching

As we began to look at this data over time, we saw that the best teachers were scoring eighty and above and the struggling teachers were scoring in the fifties and below. We found that we could use the data to inform professional development and coaching sessions. We could team up struggling teachers with more competent peers so they could see high-quality teaching in action. We felt that we were beginning to get a sense of how to measure instruction without interfering with a teacher’s capacity for creativity and innovation. We found that the protocol worked in all subject areas and at a range of grade-levels.

A careful analysis of the data from expert teachers allowed us to identify eight practices that resulted in high levels of student engagement. The practices include:

1. Immediate engagement. Teachers began teaching as soon as they had contact with the students and students knew exactly what to do upon entering the classroom. There was no wasted time or downtime, and students enjoyed the challenge of beginning their work right away. Usually the activity was a follow-up to some previous work, an assignment that was explained the last time they were together, or a warm-up activity that primed students’ minds to engage in the next task.

2. Scaffolded academic dialogue. This is sometimes referred to as partner talk, but traditional partner talk does not always result in teachers providing adequate scaffolding (Kirschner, Sweller, & Clark, 2006; Westgate & Hughes, 1997). In our observations, scaffolded academic dialogue ensured that all students engaged with the material. Students saw themselves as co-teachers when they used academic dialogue and understood that being able to explain the concepts they were exploring was important evidence that they were learning.

3. Real-world connections. These sometimes served as springboards for studying a concept in depth, while at other times were woven into the tasks teachers assigned. The important thing was that students connected the concepts they were learning to real-life experiences that made sense to them.

4. Front loading. This is a strategy often used with English Learners who struggle with academic vocabulary. The teacher “front loads” the vocabulary by providing meaningful experiences with key words prior to students being confronted with them in challenging texts or tasks.

5. Differentiated instruction. Because students at different ability levels or with different prior experiences will engage with the concept differently, knowing the strengths and needs of students enables teachers to plan activities that all students can access. Differentiated instruction requires that the teacher understand both the preconceptions and misconceptions that different children bring to the learning process and adjust the task accordingly to meet the needs of students at a variety of levels.

6. Feedback and conferring. Teachers provided immediate feedback and coaching to students as they tried on the skills they were learning. Teachers made a point of constructing tasks that allowed them to observe children’s thinking. There was a sense that all of the students knew that the teacher cared about what they thought and that any misconceptions would be recognized and addressed.

7. Structured reflection. This proved to be a critical component of the learning process in all of the high-quality lessons that were observed. Reflection requires that students think on their own and reflect on the meaning of their work in order to improve their understanding. Reflection was not something that happened after the learning; rather, it was an integral part of the entire process.

8. Lesson closure with connected homework. Teachers brought the class together as a group and facilitated a synthesis of the big ideas of the lesson. Effective lesson closure is a result of the teacher paying careful attention to the different ways students interacted with the concepts of the lesson so that the best thinking could be made public and a bridge could be built between yesterday, today, and tomorrow’s work. Connected homework means that assignments were directly related to what students worked on in class and served as an extension or reinforcement of what was learned. The most artful teachers integrated these practices based on the feedback they received from their students as they taught their lessons. Some teachers used more than one practice at a time to create learning tasks that maximized students’ participation, critical thinking, and students’ use of academic language.

Ineffective Practices

Conversely, in classrooms scoring 50 or below, we identified six ineffective practices that diminished student engagement and achievement. These included: (a) students waiting with no academic expectations; (b) students copying from the board, texts, or each other; (c) rapid-fire questioning with one-word answers; (d) students being called on one at a time; (e) teachers answering their own questions; and (f) public reprimands for off-task behavior. In these classrooms, it was evident that mastery of the content would be difficult for students. The opportunities students had to make sense of concepts through critical thinking and discussion were limited due to ineffective decision-making on the part of the teacher resulting in low levels of student engagement and achievement.

Professional Learning

Of course the method presented here is not the only way to measure high-quality instruction, but it turned out to be a powerful first step. It allowed us to celebrate teachers who were making learning a rich and wonderful experience for children and also begin having conversations with teachers who were truly doing damage. The protocol has been field-tested in many different contexts, and the reactions from teachers have been overwhelmingly positive. Often, at the end of a lesson debrief, teachers are amazed that such explicit feedback is even possible. One teacher said, “Wow. I have never received feedback that has been so detailed.” Another teacher stated, “When I saw my data, I wanted to figure out how I could make the low points more exciting and engaging for the kids. I would like to get this kind of feedback more often and if I could look at it with my peers, I feel I could really improve my teaching.” Another teacher commented that it would be helpful to use the protocol with her peers to assess participation, cognitive demand, and academic language after collaborating on planning a lesson. She said, “Then we could really see where we need to tweak our own language to make the instruction stronger.” These comments represent an example of the new kinds of conversations needed regarding the public versus private nature of the teaching profession. Providing teachers with empirically valid data that allows them to understand the impact that their instruction has on students can be a powerful lever in fostering this level of professional dialogue.

While I look back fondly on my own experiences in professional learning communities, although they were rich and wonderful experiences, I never had an opportunity to get useful feedback on how I actually interacted with my students or on the quality of my lessons. I thought I was doing a pretty good job, but I didn’t have any real way to know for sure. The feedback I did receive was generally positive, but not related to specific ways I could improve my practice. The protocol presented here attempts to provide useful data to teachers that inspires reflective practice. It is meant to foster more active engagement in the improvement process and more focused collaboration when working with colleagues to improve teaching and learning.

References

Burns, Marilyn. (2000). About teaching mathematics: A K-8 resource. Sausalito, CA: Math Solutions Publications.

Calkins, Lucy McCormick. (1986). The art of teaching writing. Portsmouth, NH: Heinemann Educational Books Inc.,

Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8-15.

Elmore, R. F. (1996). Getting to scale with successful educational practices. Harvard Educational Review, 66(1), 1–26.

Hargreaves, A. & Shirley, D. (2009). The fourth way: The inspiring future for educational change. Thousand Oaks: Corwin.

Hess, K. K., Jones, B. S., Carlock, D., & Walkup, J. R. (2009). Cognitive rigor: Blending the strengths of Bloom’s taxonomy and Webb’s depth of knowledge to enhance classroom-level processes. Online Submission. Webb, N. L. (2007). Issues Related to Judging the Alignment of Curriculum Standards and Assessments. Applied Measurement In Education, 20(1), 7-25. doi:10.1207/s15324818ame2001_2

Weisberg, D., Sexton, S., Mulhern, J., Keeling, D., Schunck, J., Palcisco, A., & Kelli Morgan (2009). The widget effect: Our national failure to acknowledge and act on differences in teacher effectiveness. Second Edition. New Teacher Project.

To read more of Kathleen’s dissertation visit:
search.proquest.com/docview/1429525668