OpenAI's published Morgan Stanley account describes implemented evaluation datasets and expert review for internal retrieval and meeting summarization. This is independent analysis of an external project. XIVTech did not deliver it; no client relationship or partnership with either organization is claimed.
The documented implementation
According to OpenAI, Morgan Stanley Wealth Management used GPT-4 in an internal assistant for advisors accessing its knowledge base. The account also describes Debrief, a meeting-summary tool using Whisper and GPT-4. It reports client consent for meeting recordings and advisor review of generated outputs before finalization.
These are a participating vendor's published statements, not independent XIVTech observations. The source does not expose the datasets, full sampling methodology or internal deployment configuration.
Evaluation followed the task
OpenAI describes summarization evaluations in which advisors and prompt engineers graded accuracy and coherence. It also reports translation evaluations and work on retrieval methods as the document library expanded. For Debrief, it describes datasets representing different meeting types and tests of important action-item capture.
Retrieval, summarization and action-item extraction are different acceptance questions. A broad score would not explain which behaviour failed. That is an editorial interpretation; the public source provides no transferable acceptance threshold for another organization.
Human review and regression testing
The account says advisors review and adjust generated outputs and describes daily testing with sample questions. Reviewing a particular output and assessing a system change are distinct controls. One does not automatically replace the other. The public description does not establish every mechanism connecting reviewer findings to regression cases; inventing an architecture would overstate the evidence.