Treat export as a transformation worth testing
A reviewer can record a careful judgment that disappears during export. A receiving tool might accept only a chosen answer and a rejected answer, while the review system also records ties, shared failures and comparisons lacking essential context.
The engineering question is not simply whether the exported file loads. It is whether each exported record still means what the reviewer decided. Obtain the receiving format's actual requirements: different training and evaluation consumers may support different outcomes. This is proposed technical guidance, not a report of client work or a universal format specification.
Build a small semantic test fixture
Create a synthetic fixture containing one clear example of every supported review outcome. Give each record a stable identifier and keep answer identity separate from screen position.
- P01: A preferred, both acceptable. Export A as chosen if the consumer supports that meaning.
- P02: equal preference, both acceptable. Retain separately or explicitly exclude.
- P03: both answers unacceptable. Preserve failure information rather than inventing a preference.
- P04: essential document missing. Mark unjudgeable rather than turning missing context into a tie.
These are invented test records, not observations from a project. Include another record with the preferred answer displayed second. That tests whether the exporter follows answer identity rather than assuming the first answer won.
Reconcile every input record
Require every input identifier to appear in exactly one documented disposition: exported, retained outside the receiving format, excluded for a stated reason, or rejected as malformed. For a one-record-per-comparison design, input count equals exported plus separately retained plus excluded plus rejected. Make these categories mutually exclusive.
If a transformation creates several rows, document its expected multiplicity and reconcile identifiers as well as row counts. Count reconciliation does not prove semantic correctness, but it exposes unexplained disappearance and duplication.
Inspect the exported meaning
Read the fixture through the receiving parser where available. Check that chosen and rejected answers retain their identities; missing judgments have not become default wins; shared failures have not become positive examples just because one answer is less poor; and exclusions remain connected to original comparisons. Re-running the transformation should not duplicate records.
If the consumer cannot reconstruct the original judgment, preserve a separate review record with the export mapping. That record is not permission to add unsupported fields to the consumer's file.
Report the cost of compatibility
In a synthetic batch of 100 reviewed comparisons, only 60 might contain explicit preferences suitable for a winner-only export. Report 60 exported preferences from 100 reviewed comparisons, together with the other dispositions, not 100 preference pairs.
Excluding non-winner outcomes may be necessary for compatibility, but changes the dataset's composition. Preserving them costs storage and maintenance while retaining information for future consumers. Verify actual format details against the selected consumer's documentation; this procedure does not establish its supported fields.
The London preference-outcome guide addresses commissioning decisions. UK AI Data & Evaluation services distinguish this export problem from other data workstreams.