Devlog

The fourth dimension: comparing the models that draw the people

Five text-to-image generators, ten peoples, 200 renders scored against real-photo baselines, and a blind vote with the model names hidden.

  • AI
  • Research
  • Community

The site had three catalog dimensions: peoples, phenotypes and poses. On September 8 the models that draw the people became the fourth. The rule is simple: the people stays constant and the model varies, so every comparison sets generators side by side on one people, and no surface orders peoples by a number.

The first cohort is five generators and ten peoples, 200 test renders in all, each scored against real-photo baselines where one exists. A blind pair card shows readers two portraits of the same people with the generator names hidden until the vote lands. A public catalog at /generators describes each model and its behavior with adult content, and cites published bias research where it exists.

What a generator cannot do is part of the record. One generator drew men for Balochi women eight times out of eight, and that empty cell stays on the page as its result. The comparison pages opened to readers on September 16.

Under the hood

  • The unit is a cell: one generator, one people, one prompt register. Ten peoples across five generators makes 50 cells at four renders each. Every generator gets the same prompt template and a square frame, with any built-in prompt rewriting switched off, a seed recorded where it accepts one, and a hash of each cell's prompt.
  • Scoring reuses the vision analysis that built the real-photo baselines, on the same bucketed scales, restricted to the dimensions that transfer from notable-people photographs to a generated portrait: skin tone, eye color, eye fold, hair texture and hair color.
  • The design rule is enforced by the shape of the code. Comparison pages live under the people, at /ethnic/{slug}/generators; no URL nests a people beneath a model; no function computes a per-people score or sorts peoples by one; and the export sorts by people, then generator.
  • The blind pair deals two renders of the same people and register from different generators, with sides assigned by a coin flip on every serving. The pair's identity is derived from its two render IDs, so a reader who meets the same pair dealt the other way updates a single vote, and the side each model sat on is recorded so position bias can be measured. Readers answer left, right, both or neither.
  • Reader pair votes publish as preference per people and are never folded into the automated score. Renders are stored apart from the catalog, and the render and pair-vote tables join the open dataset.
text-to-image generators
5
peoples in the first cohort
10
test renders
200

Peoples in this entry

See it on the site

Sources and open data

Read the full log