How to judge a contest fairly
A judged contest rewards the best entry rather than the luckiest one. That’s why people put real effort into entering, and why a well-run one is worth the work.
The trade is that someone has to decide one entry beat another, and be ready to say why. Get that part right and the result holds up to anyone who asks.
Most of the work happens before a single entry arrives.
On this page: what makes judging fair · setting criteria · how many judges · scoring scales · bias and conflicts · handling ties · what to tell entrants · FAQ
What makes a judged contest fair?
A judged contest is fair when the criteria are fixed and published before entries open, every entry is assessed against the same criteria, and the judges have no undisclosed stake in who wins.
Fairness in judging isn’t about removing opinion. Opinion is the point, otherwise you’d run a draw. It’s about making sure the opinion is applied consistently and for stated reasons.
So there are three things to get right: the criteria, the panel, and the process.
What should you decide before entries open?
Everything that could change who wins. Once entries are in front of you, any rule you invent looks like it was invented to produce a result.
Settle these first:
- The criteria, and what each one means in practice
- The weighting, if some criteria matter more than others
- The scoring scale, and what its numbers represent
- How many judges, and who they are
- How ties get broken
- What disqualifies an entry
- Whether judges see entrant names
Publish the criteria with the contest rules. Entrants who know what’s being assessed submit better work, which improves the contest for everyone, and it removes the suspicion that the criteria were shaped afterward to fit a favorite.
How do you write criteria that hold up?
Write criteria that two different judges would apply the same way. If a criterion can’t survive that test, it’s a personal preference with a score attached.
“Originality” fails the test. Two judges will read it differently and neither is wrong. “Uses the product in a way not shown in our own marketing” is checkable, and it’s probably what you meant.
A workable rubric usually has three to five criteria. Fewer and you’re just ranking on overall impression. More and judges start double-counting the same quality under different names, which quietly weights it twice.
For each criterion, write one sentence saying what a high score looks like and one saying what a low score looks like. That single step removes most scoring disagreement before it happens, because judges calibrate against the description rather than against each other.
A scoring rubric is the fixed set of criteria, their weights, and the scale used to score each one. It exists so that a score means the same thing regardless of who awarded it.
How many judges do you need?
Three is the practical minimum for anything with a real prize. One judge is an opinion, two can deadlock, and three produces a usable average while limiting how far one outlier can move a result.
Above five, the coordination cost grows faster than the benefit. Judges drop out, scores arrive late, and the schedule slips.
Large contests usually solve volume with rounds rather than headcount: a wider group shortlists, and a smaller panel judges the shortlist properly. That keeps deep assessment for the entries where it matters, and it’s a lot easier to staff.
If you only have one judge, say so publicly. A single named judge with published criteria is more defensible than a vague “our team” that might have been one person anyway.
What scoring scale should you use?
Use a small scale, between three and five points, with each point described.
Ten-point scales feel more precise and aren’t. Judges cluster in the middle, avoid the extremes, and the difference between a 6 and a 7 varies more between judges than it does between entries. You end up with numbers that look exact and mean very little.
A five-point scale where 1 through 5 each have a written description produces more consistent results, because judges are matching entries to descriptions rather than to a feeling about numbers.
Whatever you choose, use the same scale for every criterion. Mixing scales across a rubric makes the weighting accidental instead of deliberate.
How do you keep bias out of it?
Hide entrant identities where you can, and require judges to declare conflicts before they see anything.
Blind judging means judges assess entries without knowing who submitted them. It’s the single most effective tool here, and it’s straightforward for written work, photography, design, and most creative categories. It’s harder when entries are inherently identifiable, like video featuring the entrant, and you shouldn’t pretend it’s blind when it isn’t.
Conflicts of interest have to be declared before judging starts, not after a problem surfaces. The usual ones: a judge who knows an entrant personally, who works with them, or who has a financial interest in the outcome. The remedy is usually that the judge recuses from that entry rather than leaving the panel.
Write the rule down before you need it. A published policy makes it routine. Deciding on the day makes it something people talk about.
How should ties be broken?
Decide the method in advance and publish it. Ties are common in scored contests, especially with small panels and short scales.
Reasonable options, in rough order of preference:
| Method | How it works | Note |
|---|---|---|
| Priority criterion | The tied entry scoring higher on a nominated criterion wins | Cleanest. Say which criterion when you publish the rubric |
| Head to head | Judges compare only the tied entries and pick | Works well with a small panel |
| Chair decides | A nominated judge has the casting vote | Fast, and only defensible if the chair was named up front |
| Random draw | Break the tie by chance | Honest, and better than an invented reason. Publish that you did it |
The one to avoid is deciding after you see who’s tied. That’s when the reasoning starts bending toward the entry you privately prefer.
What should you tell entrants afterward?
Publish the winners, the criteria they were judged against, and who judged. That combination answers almost every question people actually have.
Individual scores are a separate decision. Publishing them is maximally transparent and can be unkind, especially for creative work where a low score is a public verdict on someone’s effort. A reasonable middle ground is publishing the winners’ scores, or offering scores privately on request.
What you should not do is publish nothing but a name. That’s the version that generates questions, because from outside it looks like a decision made privately.
What goes wrong most often?
Criteria written after entries arrive. Even when done honestly, it’s unprovable and it looks exactly like the dishonest version.
Adding a criterion nobody was told about. If professionalism of presentation matters, say so before entries open. Discovering it during judging penalizes people who would have done it.
Judges scoring inconsistently over time. Panels drift, usually getting harsher as they go. A short calibration round on three sample entries before scoring properly fixes most of it.
One judge dominating a discussion. If judges score independently before discussing, you get their real opinion. If discussion happens first, you often get the loudest person’s opinion three times over.
No plan for a disqualification. Late entries, wrong format, ineligible entrants. Write the rule in advance and apply it visibly.
Frequently asked questions
Is public voting fairer than a judging panel? It’s different, not fairer. Public voting measures popularity and reach, so it rewards the entrant with the biggest network rather than the strongest entry. It’s also the format most exposed to manipulation. A panel measures the entry against stated criteria. Which fits depends on whether you want the best submission or the most shared one.
Can I be a judge in my own contest? Yes, and it’s common for small contests. Declare it, and avoid being the only judge when the prize is significant. If you have a relationship with an entrant, recuse yourself from that entry.
How long should judging take? Plan for real time per entry rather than a quick skim. Budget it properly, multiply by the number of entries and judges, and add a buffer for chasing. Judging deadlines slip more often than any other part of a contest.
Should judges see each other’s scores? Not before they’ve submitted their own. Independent scoring first, discussion afterward. Seeing another judge’s score first anchors your own, which quietly turns a panel into one opinion.
What if a winner turns out to be ineligible? Apply the rule you published. If it says the next highest scoring entry takes the place, do that and say so. This is why eligibility and substitution rules belong in the rules before entries open.
Do I need to publish the losing entries? No, and often you shouldn’t. Entrants submitted for a contest, not for publication. If you want to publish entries, ask for that permission explicitly when they enter.
Raflia runs panel-judged contests with scoring, rankings, and results your entrants can check. See jury selection, or join the waitlist and we’ll email you when Raflia opens.
Running a random draw instead? See how to prove your giveaway was fair.