Run in-person and virtual judging panels from one scoring screen, with every judge on the same scorecard and no binders.
For many award programs, judging day still runs on paper. Staff print and tab a binder for every judge, the panel spends a day in one room, and someone types the scores into a spreadsheet afterward. Judges who can't travel join by video with a PDF, or don't join at all.
The binders aren't the only problem. When a panel talks before it scores, the first strong opinion often sets the outcome. Judges outside the room carry less weight than judges inside it. And when a nominee asks how the winner was chosen, the reasons are hard to find.
This session draws on two of the five dimensions of Review Integrity, a framework for judging whether an application and review process is fair, consistent and defensible.
The first is Reviewer Calibration: are judges working from the same criteria, are conflicts kept out, and is it clear when one judge scores much harder than the rest? On a mixed panel, that means the judge at home and the judge at the table see the same entry and the same scorecard.
The second is Decision Traceability: can any award decision be reconstructed and defended? If every judge scores before the discussion, the scores and the final choice sit in one record, not on sticky notes.
When judging runs on binders, staff lose days on both ends, busy members and out-of-town jurors turn down the invitation, and judges who felt unheard don't come back. The award itself gets harder to defend.
Award nominations typically run from fall into late winter. How entries reach your judges, which scorecard they use and who judges which category all get settled before nominations close.
You'll also see how Reviewr's team builds your program from the nomination forms and scoring sheets you already use, in about 30 days.
Association executives, awards staff and committee chairs who run judged award programs: REALTOR® of the Year and other member awards at REALTOR® associations, society awards and foundation scholarships at CPA societies, and design awards juries at AIA chapters.
Can some judges meet in person while others score remotely? Yes. Every judge scores on the same screen with the same scorecard, wherever they are. The session covers how to run a panel that's part in the room and part remote.
What if a judge wants printed entries? They can have them. Their scores are entered into the same record as everyone else's, so nothing gets re-typed from the margins later.
We only get a handful of nominations. Is this worth 30 minutes? Yes. Most of the session is judging practice you can use with any number of entries and any tools.