Metaevaluation: Interrater Reliability Results; Discussion

These slides, provided by EvaluATE: Evaluation Resource Center for Advanced Technological Education, are from a webinar presented by Lori Wingate. This webinar discussed the results of a study of interrater reliability of metaevaluation ratings. The metaevaluation (evaluation of evaluations) that served as the context for this study was a review of ten evaluations of National Science Foundation-funded Advanced Technological Education projects and centers. The reviewers included ten doctoral students in evaluation, ten evaluation practitioners, and ten evaluation scholars. The primary purpose of the metaevaluation was to identify the strengths of the evaluations and areas in need of improvement. The ratings were also subject to analysis to determine the extent of interrater reliability.