A documented implementation, not an XIVTech engagement
This is an independent analysis of OpenAI's publicly documented InstructGPT project. XIVTech is not identified as its implementer or a delivery partner. The project facts below come from the named authors' publication and research paper; the buyer questions are our synthesis, not claims about an XIVTech client result.
In Aligning language models to follow instructions, dated 27 January 2022, OpenAI described deploying InstructGPT models through its API. The accompanying Training language models to follow instructions with human feedback, submitted 4 March 2022, describes the experimental approach and findings. Both are primary sources from the same research team, not independent external replications. This analysis does not reproduce the experiments or assert what today's model services do.
Demonstrations were one data product
The paper describes collecting labeler demonstrations of desired behaviour and using them for supervised fine-tuning. A demonstration supplies an example response. Its role differs from a judgment choosing between existing candidate responses. Calling both simply labelled data obscures what a downstream process can actually use.
For a buyer, the practical question is what one record contains and what task it supports. A request for examples of desired behaviour should not be treated as equivalent to a request for preference rankings merely because both involve human review. The paper establishes that distinction for this project; it does not establish that every current engagement needs both.
Preference comparisons supported another stage
OpenAI's publication describes collecting comparisons between model outputs, training a reward model to predict labeler preference and then using it when fine-tuning the policy with PPO. These are attributed descriptions of that implemented approach, not a proposed XIVTech training architecture or a commitment to provide reinforcement-learning services.
