Inspectable walkthrough
What Changes When Three Models Receive the Same Request?
One 10,277-character lecture transcript and one request went to three live models. The run compared output behavior, latency, and cost against five criteria the user confirmed first.
The Run at a Glance
The request was simple:
Organize this lecture clearly without compressing the content.
“Do not compress” asks the model to preserve scope. “Organize clearly” asks it to impose stronger structure. The request never specifies what may be removed, so the models can operationalize organize differently.
For this run, the user wanted detailed notes capable of replacing the source. GPT-5.5 best fit that goal. A fast overview or a lower-cost priority could rationally produce a different choice.
Define the Desired Result Before Scoring
- Preserve flow: keep the lecture's progression from beginning to end.
- Preserve detail: retain reasons, examples, and supporting explanation.
- Clean selectively: remove repetition and filler without shrinking the information range.
- Make it navigable: use headings and subheadings.
- Stay grounded: do not add explanations or conclusions absent from the source.
These were stored user criteria, not a rubric invented after seeing the outputs.
The Same Passage Produced Different Behavior
One source passage explains why an evaluator's 3 out of 5 is less useful than the explanation behind it.
| Model | Behavior in the same passage |
|---|---|
| Claude Sonnet 4.6 | compressed it into a procedure: store score and explanation, then inspect the explanation |
| GPT-5.5 | retained why it was a three, what was missing, and what needed to improve |
| Gemini 3.5 Flash | summarized that explanations support debugging and root-cause analysis |
Another passage says that five to ten inputs are only a starting point for unit tests and that production evaluation needs larger, more diverse data. GPT retained the rationale; Gemini retained the user-simulator conclusion but omitted the concrete five-to-ten-input context.
None of the three was simply wrong. The difference was how much reasoning and example detail remained for a user who wanted source-replacing notes.
Recommendation Across Quality, Latency, and Cost
| Model | Preservation | No compression | Structure | Grounding | Final fit |
|---|---|---|---|---|---|
| GPT-5.5 | 93 | 94 | 96 | 84 | 92 |
| Gemini 3.5 Flash | 58 | 42 | 88 | 68 | 52 |
| Claude Sonnet 4.6 | 52 | 38 | 86 | 72 | 50 |
| Model | Output characters | Latency | Stored estimated cost |
|---|---|---|---|
| GPT-5.5 | 24,244 | 174.5 s | about $0.108 |
| Gemini 3.5 Flash | 5,556 | 21.0 s | about $0.021 |
| Claude Sonnet 4.6 | 4,264 | 64.6 s | about $0.091 |
GPT preserved the widest range of detail but expanded some explanation beyond the source, and it was the slowest and most expensive. Getsit therefore recommends fit for this input, intent, and operating constraint, not the universally best model.
Minimal Reproduction Protocol
- Before viewing outputs, define what must remain, what may be removed, forbidden behavior, and useful output form.
- Send the same source and request to each provider.
- Compare preservation, compression, and additions against matched source passages before reading the total score.
- Put latency, estimated cost, and failures next to the quality judgment.
- Limit the conclusion to this input, intent, and operating context.
Evidence Boundary
- This is one execution, not a repeatability study.
- Scores are judgments from an LLM evaluator, not objective truth.
- The stored product run was not blind.
- The original discovery conversation was not retained and is not this run.
- This run did not test improvement after model-specific prompt adjustment.