Docs

Experiments

Start with the main constraint: you cannot A/B test a card across viewers. A social platform fetches your image once and shows that single copy to everyone who sees the post. There is no per-viewer request to split on. Any tool claiming otherwise is either splitting something else or not doing what it says.

What does work is randomising by page. Half your articles get design A, half get design B, and you compare the two groups. That is a real experiment. It needs more pages than a viewer-level test would need viewers, and it answers “which design works better across my content” rather than “which works better for this one post”.

https://ogsmith.app/api/og?key=og_yourkey&exp=headline-test&k=post-123&title=My%20post

k identifies the page and must never change for it, so a post slug or database id works best. With &url= it defaults to that URL. OGsmith picks a page's variant once and stores it, so editing an experiment later never changes the artwork on posts already shared.

Never key on the headline. It is content, and content gets edited. Fixing one typo would change the key, and a changed key is a new page: it would be counted twice in the denominator and land in the other variant about half the time, with its history split across both arms. The API refuses the title for exactly this reason and asks for an id instead.

Editing during a test

Fixing a typo in a card is fine. Change the text, bump your cache version so platforms refetch, and the experiment is unaffected: the key didn't change, so the page keeps its variant and its numbers.

Changing what a variant looks like is different. The numbers you have already describe the old design, so leaving them in place pools two different cards into one rate. The dashboard warns when you do this and offers Reset results, which clears the counters while keeping every page on the variant it already has. Measurement starts over, and nothing already shared changes appearance.

Measuring the outcome

OGsmith can count how often a card was rendered, but a render is a crawler fetch, not a person or a click. Nothing about what a human does with your post ever reaches us. So outcomes have to come from the side that can see them: your analytics.

Ask which variant a page is in, tag your own analytics with it, and report back what happened:

GET https://ogsmith.app/api/experiments/assign?key=og_yourkey&exp=headline-test&k=post-123

{ "experiment": "headline-test", "variant": "b",
  "label": "B — terminal", "params": { "template": "terminal" } }

POST https://ogsmith.app/api/experiments/convert
Authorization: Bearer og_yourkey

{ "exp": "headline-test", "k": "post-123" }

Asking for an assignment doesn't count as a render, and neither call costs quota. Results appear under Dashboard → Experiments, which reports a rate per page and a two-proportion test. It won't call anything a winner until each variant has at least 20 pages and 5 reported outcomes. Stopping at the first good-looking number is the way most split tests go wrong.