Question pools explained

Updated 15 November 2026 · About 7 minutes to read

When everyone gets the same questions in the same order, answers spread quickly — a screenshot in a group chat is enough. Question pools and shuffling make each attempt different, so sharing answers stops paying off.

What a question pool is

A pool is a group of interchangeable questions about the same topic and of similar difficulty. Instead of putting every question in the test, you tell the test to draw a number of them at random for each attempt.

Example. You have 30 questions tagged fire-safety and 20 tagged first-aid. Your test has two sections:

  • Section 1: draw 8 from fire-safety
  • Section 2: draw 5 from first-aid

Every attempt gets 13 questions, but two people sitting side by side will usually share only a few of them.

How big should a pool be?

The ratio between the pool and the draw decides how much overlap people see. If a pool has N questions and each attempt draws n, two attempts share on average n² ÷ N questions from that pool.

Average shared questions between two attempts
Pool sizeDrawn per attemptShared on average
101010 (no variation)
20105
30103.3
50102

A pool two to three times the size of the draw is a good target for workplace tests. Beyond that, gains are small and writing equally good questions gets hard.

Keep draws fair

Randomization is only fair if any draw is about as hard as any other. Practical rules:

Shuffling questions and options

Shuffling the order of questions within a section and the order of options within a question adds variation even when the question set is the same. It also removes position clues — authors tend to put the correct answer in a predictable spot. Don't shuffle when order carries meaning (“Step 1, Step 2…”) or for true/false.

Grading randomized tests consistently

A common worry: if everyone's paper is different, how do you grade it and handle disputes? In ExamKit each attempt gets a random number (a seed) when it starts. The question draw and the option order are computed from that seed with a deterministic generator, so the server can rebuild exactly the paper a person saw — for grading, for resuming after a reload and for reviewing an attempt later. Publishing a test freezes a version with all candidate questions, so editing the bank afterwards never changes a paper that was already given.

What randomization doesn't do

Pools and shuffles make copying and answer-sharing much less useful. They don't stop someone from looking things up or getting help during the test. ExamKit has no proctoring; combine randomization with a sensible time limit (enough for someone who knows the material, not enough to research every question), and for anything high-stakes use a supervised room.

Quick setup in ExamKit

  1. Tag questions by topic in the question bank.
  2. In the test builder, add a section and choose Draw from tags; pick the tag and how many to draw. The builder shows how many questions are available.
  3. Turn on Shuffle questions and Shuffle options.
  4. Use Preview a few times to see different draws, then publish.

Build a randomized test

Tag your questions, draw from pools, and let the server keep the time. Free.

Create a free workspace