Understanding measures to assess research use by practitioners

Co-Principal Investigator, 1.2017-present.
Funding: W. T. Grant Foundation, $50,000.
The ways in which research and evidence are used by practitioners (including policy-makers, school district administrators, and principals) is surprisingly not well understood. To help the W. T. Grant Foundation develop research in this area, we are cataloging ways to measure research use and areas that have particularly little information.

Crossing boundaries: A mixed-methods study of research interaction

Co-Principal Investigator, 10.2014-present.
In this multi-stage, mixed methods study that, we are seeking to better understand the dynamics the academic research community. Because the collegial interactions of researchers is central to the importance of public research, we decided to build an empirical basis of what it looks like in different disciplines. We first interviewed academic leaders to understand aspects of the professional community that were important and have designed a social network analysis and survey to administer to all academic departments. While I am working with two other colleagues, I have been primarily responsible for the design of the online survey system and the quantitative social network analyses.  The intro page can be found here: http://crossingboundaries.kevin-crouse.net 

 

Aggregation models of measures in new teacher evaluation systems

Principal Investigator, 5.2013-present.
In most new state teacher evaluation models, teachers receive assessment scores on multiple measures of teaching quality that are then combined through a series of decision rules and translated into a discrete set of performance levels (like Effective or Ineffective) that are linked to personnel requirements ranging from specifications for professional growth plans to dismissal. The characteristics of the decision rules vary substantially and have not been investigated at length. In this project, I use a data set of research-based teacher assessment measures and model how different rules common in Race to the Top teacher evaluation systems lead to drastically different numbers of teachings being assigned to different performance levels, even if their scores on the actual measures remains the same.

I have written a small program (in R) to let you examine some of these effects yourself at : https://krcrouse.shinyapps.io/aggregation-shiny/.  It allows you to view and manipulate a subset of the variables that I am looking at (relative measure weights, cut-offs, number of performance levels, and rounding).

Student Learning Objectives in new teacher evaluation systems

Principal Investigator, 7.2014-8.2015.
Student Learning Objectives [SLOs] are increasingly included in teacher evaluation systems to assess teachers in non-tested subjects. While there have been promising pilot implementations in Denver, Austin, and Charlotte-Mecklenburg, SLOs have not been used as part of high-stakes accountability systems like they are today. Along with two colleagues, I investigate the new uses of SLOs in teacher evaluation systems, consider implications for their validity in high-stakes systems, and propose a framework that districts or states may find to be more effective to use SLOs as one measure in teacher evaluations. A chapter emerging from this work will be included in the upcoming edited book, Student Growth Measures in Policy and Practice: Intended and Unintended Consequences of High-Stakes Teacher Evaluations.

 A comparative policy analysis of Race to the Top teacher evaluations

Co-Principal Investigator, 6.2013-12.2014.
Funding: American Federation of Teachers, $40,000
Analyzed all Race to the Top teacher evaluation systems and provided a portal to compare and analyze specific attributes. This work included developing a collaborative database to annotate documents and discuss approaches with collaborators.

Analysis of Washington, D.C.’s IMPACT teacher evaluation system

Co-Principal Investigator, 4.2014-8.2014.
Based on our current work on Race to the Top teacher evaluation systems, we were commissioned by the the National Academy of Sciences to review Washington, D.C.’s system for their own work. That report is available at the National Academies of Sciences

Infusing digital tools into a middle school vocabulary program

Research Project Coordinator, 3.2013-11.2013.
We received a small grant to purchase iPads to see if it would help in vocabulary instruction for students in an academic program for at-risk youth. We designed it as a quasiexperiment in which classrooms across two grades were assigned to a treatment or control and closely monitored for instructional time on vocabulary learning. I encouraged the project supervisor to add in an ethnographic element to better understand what happened in class in addition to the actual vocabulary tasks. Our findings found that student vocabulary achievement did not change with iPad use, but the time students independently engaged with the tablets provided significant time for teachers to  work independently with specific students, often on lesson goals that they felt were more important. This led to our conclusions that there are numerous benefits to iPad use in this context, but an increase in vocabulary-related test scores was not one of them.

Ethnographic case study of teaching challenges in rural South India

Principal Investigator, 5.2012-3.2013.
I spent the summer of 2012 in a rural, south Indian community to observe the school context in relation to school quality. While I provided teacher training classes in English language use to help the school out, I primarily conducted interviews and classroom observations to get a sense for the primary issues the school faced compared to the controversies in the mostly western research literature. While much of the current thought in academic journals focuses on the lack of facilities in such schools, I found the primary concern to surround teaching, especially that there were few teacher training programs or professional development options in rural areas and city schools provided salaries for teachers that far exceeded what rural schools could afford. 

Youth participatory action research in the classroom

Research Assistant, 8.2011-3.2013.
Facilitating PAR in urban high schools; funded by the Shulman Foundation