I gave GPT 5.6 Sol Max, Fable 5 Max, and Grok 4.5 Speed the same six fictional startup documents and asked them to prepare a launch brief a writer could safely use.
GPT produced the most cautious response and scored 97/100.
Fable and Grok identified several contradictions but still included disputed information in sections labelled as confirmed.
GPT also noticed a security disclosure problem missing from the original answer key.
| Model | Score |
|---|---|
| GPT 5.6 Sol Max | 97/100 |
| Fable 5 Max | 92/100 |
| Grok 4.5 Speed | 80/100 |
Key finding: Spotting a contradiction is not the same as keeping it out of the confirmed section.
I designed and conducted the evaluation, selected the models, reviewed the documents, compared their responses, and assessed the grading.
I also restarted the experiment after noticing a potential source of bias in my first attempt.
Gemini created the fictional company documents and answer key. Other AI tools assisted with grading, but I reviewed their conclusions against the original materials and made the final assessment.
When GPT identified a problem missing from the answer key, I checked the source documents, compared the other responses, and confirmed that the answer key had made an unsupported assumption.
Founders and content teams increasingly use AI to turn internal information into launch announcements, product messaging, and customer-facing content.
The difficulty is that internal documents do not always agree.
A pricing sheet may contradict a founder’s sales pitch. A security document may describe data processing differently from a customer quote. A product specification may fail to support a feature mentioned in launch notes.
I wanted to know which model could identify those issues and produce a brief that a writer could actually use without accidentally repeating disputed claims.
In this project, a “confirmed” claim is one supported by the supplied documents without conflicting information elsewhere. It is not a claim that has been independently verified or approved for public release.