Summary
Using a scaled-down version of the IKP (Incompressible Knowledge Probes) method, we manually ran 320 closed-book factual recall questions against GPT-4.5 to answer one question: roughly what order of magnitude is GPT-4.5’s factual memory capacity?
Conclusion: GPT-4.5’s IKP-equivalent factual capacity is about 1.3T–1.8T. Combined with its pricing (input $75/M tokens, output $150/M tokens, roughly 15–30× GPT-4o) and generation speed, we infer its architecture is more likely a large-active MoE (1T–3T total parameters, 200B–600B active), rather than an ultra-sparse MoE or a pure dense model.
Important disclaimer: this is not a formal IKP benchmark evaluation. The sample was built post hoc and skews toward the long tail, so it can’t be treated as a paper-grade result. All numbers should be read as “equivalent factual capacity,” not the actual physical parameter count.
1. Motivation: why test GPT-4.5
GPT-4.5 is a model with an unusual character. It isn’t outstanding at reasoning, coding, or agent tasks, but its writing quality is exceptional — its literary sense, natural tone, and handling of emotional detail all show a very thick language-modeling foundation, not an effect assembled from post-training and routing.
At launch, OpenAI explicitly called it the “largest and best model for chat yet,” described it as “very large and compute-intensive,” and priced it far above GPT-4o.
These signals point to the same guess: GPT-4.5 may be a previous-generation giant model that prioritizes language modeling and has very large active parameters.
IKP offers a way to test that guess quantitatively.
2. A quick introduction to IKP
IKP (Incompressible Knowledge Probes) measures a model’s parametric factual memory, not its reasoning. It uses a large set of closed-book factual questions — especially long-tail facts that are hard to guess through reasoning — to measure how much world knowledge a model has memorized.
The original IKP benchmark has 1,400 questions across 7 obscurity tiers (T1 is common knowledge, T7 is the extreme long tail), 200 per tier. Its core finding: across 89 open models with known parameter counts, IKP accuracy is strongly linear in log parameter count (R² = 0.917).
IKP doesn’t measure reasoning, code, agent, or literary ability. It answers only one question: roughly what order of magnitude this model’s factual memory capacity is.
3. Setup
Since GPT-4.5 has no open API, all tests were done manually through the ChatGPT web interface:
- About 10 questions per batch, copied and pasted batch by batch
- A closed-book prompt that explicitly forbids search and tool calls
- Output as JSONL, scored automatically on a local machine
- No browser automation, no request capture
One key detail in the prompt: the model must be told explicitly that “if you have a specific memory, giving the correct subfield plus any one paper/system/institution/collaborator is enough.” Otherwise GPT-4.5 goes into an overly conservative mode and returns null for every uncertain researcher question.
4. Three failed attempts
4.1 Long tail only, 140 questions: severe underestimate
The first attempt used only T5–T7, 140 questions in total. The result looked bad, but this wasn’t a mini-IKP — it covered only the hardest tiers, had no T1–T4 baseline, and couldn’t represent overall capability.
Lesson: a long-tail subset can’t be used to estimate parameters.
4.2 Heuristic scoring: lots of false negatives
The second attempt used a balanced 20 questions per tier for T1–T7 (140 total), but with string-matching heuristic scoring. The result was 350B — far too low.
The reason: T5–T7 contain many researcher questions (“what is this person’s research subfield, name one paper or collaborator”). The model may have answered correctly but with different wording from the reference answer, and the heuristic scorer marked it wrong.
Lesson: researcher questions need semantic scoring; string matching isn’t reliable.
4.3 Balanced 140 questions with semantic scoring: heavy overestimate
Rescoring the same 140 questions with Gemini semantic scoring pushed the result up to 5.7T. The sample was simply too small — with only 20 questions each for T6/T7, a few lucky correct answers could inflate the estimate dramatically.
After expanding the sample, the number quickly came back down.
Lesson: 140 questions don’t give enough confidence to support estimates in the high range.
5. Final result: 320 questions, formally scored
5.1 About the sample
The final set has 320 questions, scored semantically via OpenRouter. The distribution:
| Tier | Questions |
|---|---|
| T1–T4 | 20 each |
| T5 | 80 |
| T6 | 90 |
| T7 | 70 |
Because the sample is weighted toward the long tail, the main score uses an equal-weight average across tiers rather than a simple pooled total.
This sample was built post hoc; the sampling script, random seed, and possible data leakage can’t be audited, so it does not constitute a canonical IKP evaluation.
5.2 Per-tier results
| Tier | Accuracy | Strong correct | Weak correct | Wrong | Refused |
|---|---|---|---|---|---|
| T1 | 100% | 20 | 0 | 0 | 0 |
| T2 | 100% | 20 | 0 | 0 | 0 |
| T3 | 88% | 17 | 1 | 0 | 2 |
| T4 | 88% | 17 | 1 | 0 | 2 |
| T5 | 34% | 28 | 9 | 5 | 38 |
| T6 | 19% | 24 | 3 | 8 | 55 |
| T7 | 0% | 4 | 4 | 15 | 47 |
This table sketches the shape of GPT-4.5’s factual memory:
- T1–T4 are extremely solid — not something a small model can do
- T5 still shows a sizable store of knowledge, but the refusal rate is already high
- T6–T7 fall off quickly, especially on researcher metadata questions
5.3 Aggregate scores and estimates
| Method | Accuracy | IKP-equivalent parameters |
|---|---|---|
| Penalized equal-weight (floored) | 61.26% | ~1.82T |
| Penalized equal-weight (unfloored) | 59.42% | ~1.37T |
| Post-stratified (floored) | 61.08% | ~1.77T |
| Post-stratified (unfloored) | 59.24% | ~1.33T |
Overall range: about 1.3T–1.8T equivalent factual capacity.
5.4 Researcher questions behave differently
The 320 questions include 109 researcher questions (asking for a CS researcher’s subfield plus supporting evidence). The results deserve a separate note:
- Only 9 strong correct (8.3%)
- 75 refused (68.8%)
- Only 7 wrong (6.4%)
GPT-4.5’s strategy on these questions is to refuse a lot rather than make things up. That doesn’t mean it “doesn’t know CS”; it means its memory of metadata about obscure researchers is thin, and its calibration leans conservative. It also explains why changing the share of researcher questions in the sample noticeably shifts the total score.
6. External evidence: pricing and speed
Pricing
| Model | Input price | Output price |
|---|---|---|
| GPT-4.5 | $75/M tokens | $150/M tokens |
| GPT-4o | ~$2.5/M tokens | ~$10/M tokens |
The gap is about 15–30×. That pricing doesn’t look like a small-active MoE.
Speed
Early third-party data showed GPT-4.5 at about 37 tokens/s versus about 111.5 tokens/s for GPT-4o (Artificial Analysis measurements). The surface ratio is about 3×, but given that OpenAI may have devoted more serving resources to GPT-4.5 around launch, the real compute gap may be 5–10×.
Together these point to: GPT-4.5’s active parameters are far larger than GPT-4o’s, but it’s not a multi-trillion dense model either (otherwise interactive serving would be nearly infeasible).
7. Architecture inference
Combining IKP performance, pricing, speed, and writing quality, the most reasonable explanation:
| Hypothesis | Consistent with the evidence? |
|---|---|
| Small model (GPT-4o class) | ❌ T1–T4 performance and pricing don’t support it |
| 5T+ dense | ❌ Serving cost is unrealistic, and T6/T7 performance isn’t strong enough |
| 50B-active ultra-sparse MoE | ❌ Writing quality and pricing don’t support it |
| Large-active MoE, 1T–3T total, 200B–600B active | ✅ Consistent with all the evidence |
Central estimate: about 1.5T–2T total parameters, about 300B–500B active.
8. Where GPT-4.5 sits in today’s model lineup
GPT-4.5 shouldn’t be seen as a new-generation reasoning-first model. It’s more a product of the previous era:
- Above GPT-4o
- Close to the GPT-4.1 / Gemini 2.5 Pro / Claude Opus range
- Below GPT-5, and clearly below GPT-5.5
Its defining trait is very thick language modeling — every token is processed by a very large number of parameters — rather than gaining capability through reasoning chains or tool calls. That explains its contradictions: stunning writing but ordinary reasoning; rich knowledge but a thin deep tail; a strong presence, but not the frontier.
9. Limitations
- The sample was built post hoc; this isn’t a canonical IKP evaluation
- No subset-specific calibration
- The sampling script and random seed can’t be audited
- 320 questions is far fewer than the standard 1,400
- The distribution is weighted toward the long tail and sensitive to the aggregation method
- High-range IKP estimates are inherently very sensitive to scoring method and question quality
For a more reliable conclusion, build a canonical holdout sample from scratch (50 questions per tier, 350 total) and lock all experiment parameters before running.
10. Conclusion
In an informal 320-question IKP test, GPT-4.5 shows about 1.3T–1.8T of equivalent factual capacity. Combined with pricing and speed evidence, its architecture is most likely a large-active MoE with 1T–3T total parameters and several hundred billion active.
It isn’t a small model, an ultra-sparse MoE, or a multi-trillion dense model. It’s a giant model with very thick language modeling, very high cost, and outstanding writing ability, but it doesn’t belong to the new generation of reasoning-first models.