Trial Bench: A/B testing
Two or three versions of the same idea, compared fairly.
How it works
The A/B testing tab compares two or three versions of the same idea. Choose what you are varying, then for each variant (A, B and optionally C) upload the video, note what changed and write its caption. knit.bio schedules the variants as real posts, and they publish within a few minutes.
Variants are compared on follows per person reached, using a standard two-proportion significance test. The confidence shown is the lowest across every comparison with the current leader, so the leader has to credibly beat all the others, not just the nearest one. A test is not called until each variant has reached a minimum audience, and while the lead is still inside normal variance the tab says so.
When the window closes the winner is named, with how far it beat the runner-up. That is advice, not an action: open Instagram and tap Share with everyone on the winner.
One limit is built into the method. Instagram does not guarantee that variants reach entirely separate audiences, which technically bends the assumption behind the test. The tab’s confidence note says so instead of hiding it.
Set it up
- 1
- 2
Choose what you are varying
Pick one thing: the opening, the cover, the length, the caption. Changing two things at once tells you nothing.
- 3
Add your variants
For each of A, B (and C), upload the video, describe what changed and write the caption.
- 4
Wait for the window
Read the leader and its confidence as the numbers arrive, and act when the tab calls it.
Tips to get the most out of it
- Change exactly one thing between variants and keep the topic, length and posting hour as close as you can.
- Treat “still inside normal variance” as an answer. If two versions perform about the same, keep whichever is easier to make.
- With a small audience, differences of a few follows are noise. Give a test more time before you trust a winner.