Home/Journal

How big is GPT-4.5, really?

An informal estimate: a scaled-down IKP run of 320 closed-book factual questions puts its equivalent factual capacity at 1.3T–1.8T. Combined with pricing and generation speed, it points to an MoE with very large active parameters.

· Chiffonwind · 清凤 · 1,573 words
LLMGPT-4.5IKPBenchmark

Summary

Using a scaled-down version of the IKP (Incompressible Knowledge Probes) method, we manually ran 320 closed-book factual recall questions against GPT-4.5 to answer one question: roughly what order of magnitude is GPT-4.5’s factual memory capacity?

Conclusion: GPT-4.5’s IKP-equivalent factual capacity is about 1.3T–1.8T. Combined with its pricing (input $75/M tokens, output $150/M tokens, roughly 15–30× GPT-4o) and generation speed, we infer its architecture is more likely a large-active MoE (1T–3T total parameters, 200B–600B active), rather than an ultra-sparse MoE or a pure dense model.

Important disclaimer: this is not a formal IKP benchmark evaluation. The sample was built post hoc and skews toward the long tail, so it can’t be treated as a paper-grade result. All numbers should be read as “equivalent factual capacity,” not the actual physical parameter count.


1. Motivation: why test GPT-4.5

GPT-4.5 is a model with an unusual character. It isn’t outstanding at reasoning, coding, or agent tasks, but its writing quality is exceptional — its literary sense, natural tone, and handling of emotional detail all show a very thick language-modeling foundation, not an effect assembled from post-training and routing.

At launch, OpenAI explicitly called it the “largest and best model for chat yet,” described it as “very large and compute-intensive,” and priced it far above GPT-4o.

These signals point to the same guess: GPT-4.5 may be a previous-generation giant model that prioritizes language modeling and has very large active parameters.

IKP offers a way to test that guess quantitatively.


2. A quick introduction to IKP

IKP (Incompressible Knowledge Probes) measures a model’s parametric factual memory, not its reasoning. It uses a large set of closed-book factual questions — especially long-tail facts that are hard to guess through reasoning — to measure how much world knowledge a model has memorized.

The original IKP benchmark has 1,400 questions across 7 obscurity tiers (T1 is common knowledge, T7 is the extreme long tail), 200 per tier. Its core finding: across 89 open models with known parameter counts, IKP accuracy is strongly linear in log parameter count (R² = 0.917).

IKP doesn’t measure reasoning, code, agent, or literary ability. It answers only one question: roughly what order of magnitude this model’s factual memory capacity is.


3. Setup

Since GPT-4.5 has no open API, all tests were done manually through the ChatGPT web interface:

One key detail in the prompt: the model must be told explicitly that “if you have a specific memory, giving the correct subfield plus any one paper/system/institution/collaborator is enough.” Otherwise GPT-4.5 goes into an overly conservative mode and returns null for every uncertain researcher question.


4. Three failed attempts

4.1 Long tail only, 140 questions: severe underestimate

The first attempt used only T5–T7, 140 questions in total. The result looked bad, but this wasn’t a mini-IKP — it covered only the hardest tiers, had no T1–T4 baseline, and couldn’t represent overall capability.

Lesson: a long-tail subset can’t be used to estimate parameters.

4.2 Heuristic scoring: lots of false negatives

The second attempt used a balanced 20 questions per tier for T1–T7 (140 total), but with string-matching heuristic scoring. The result was 350B — far too low.

The reason: T5–T7 contain many researcher questions (“what is this person’s research subfield, name one paper or collaborator”). The model may have answered correctly but with different wording from the reference answer, and the heuristic scorer marked it wrong.

Lesson: researcher questions need semantic scoring; string matching isn’t reliable.

4.3 Balanced 140 questions with semantic scoring: heavy overestimate

Rescoring the same 140 questions with Gemini semantic scoring pushed the result up to 5.7T. The sample was simply too small — with only 20 questions each for T6/T7, a few lucky correct answers could inflate the estimate dramatically.

After expanding the sample, the number quickly came back down.

Lesson: 140 questions don’t give enough confidence to support estimates in the high range.


5. Final result: 320 questions, formally scored

5.1 About the sample

The final set has 320 questions, scored semantically via OpenRouter. The distribution:

TierQuestions
T1–T420 each
T580
T690
T770

Because the sample is weighted toward the long tail, the main score uses an equal-weight average across tiers rather than a simple pooled total.

This sample was built post hoc; the sampling script, random seed, and possible data leakage can’t be audited, so it does not constitute a canonical IKP evaluation.

5.2 Per-tier results

TierAccuracyStrong correctWeak correctWrongRefused
T1100%20000
T2100%20000
T388%17102
T488%17102
T534%289538
T619%243855
T70%441547

This table sketches the shape of GPT-4.5’s factual memory:

5.3 Aggregate scores and estimates

MethodAccuracyIKP-equivalent parameters
Penalized equal-weight (floored)61.26%~1.82T
Penalized equal-weight (unfloored)59.42%~1.37T
Post-stratified (floored)61.08%~1.77T
Post-stratified (unfloored)59.24%~1.33T

Overall range: about 1.3T–1.8T equivalent factual capacity.

5.4 Researcher questions behave differently

The 320 questions include 109 researcher questions (asking for a CS researcher’s subfield plus supporting evidence). The results deserve a separate note:

GPT-4.5’s strategy on these questions is to refuse a lot rather than make things up. That doesn’t mean it “doesn’t know CS”; it means its memory of metadata about obscure researchers is thin, and its calibration leans conservative. It also explains why changing the share of researcher questions in the sample noticeably shifts the total score.


6. External evidence: pricing and speed

Pricing

ModelInput priceOutput price
GPT-4.5$75/M tokens$150/M tokens
GPT-4o~$2.5/M tokens~$10/M tokens

The gap is about 15–30×. That pricing doesn’t look like a small-active MoE.

Speed

Early third-party data showed GPT-4.5 at about 37 tokens/s versus about 111.5 tokens/s for GPT-4o (Artificial Analysis measurements). The surface ratio is about 3×, but given that OpenAI may have devoted more serving resources to GPT-4.5 around launch, the real compute gap may be 5–10×.

Together these point to: GPT-4.5’s active parameters are far larger than GPT-4o’s, but it’s not a multi-trillion dense model either (otherwise interactive serving would be nearly infeasible).


7. Architecture inference

Combining IKP performance, pricing, speed, and writing quality, the most reasonable explanation:

HypothesisConsistent with the evidence?
Small model (GPT-4o class)❌ T1–T4 performance and pricing don’t support it
5T+ dense❌ Serving cost is unrealistic, and T6/T7 performance isn’t strong enough
50B-active ultra-sparse MoE❌ Writing quality and pricing don’t support it
Large-active MoE, 1T–3T total, 200B–600B active✅ Consistent with all the evidence

Central estimate: about 1.5T–2T total parameters, about 300B–500B active.


8. Where GPT-4.5 sits in today’s model lineup

GPT-4.5 shouldn’t be seen as a new-generation reasoning-first model. It’s more a product of the previous era:

Its defining trait is very thick language modeling — every token is processed by a very large number of parameters — rather than gaining capability through reasoning chains or tool calls. That explains its contradictions: stunning writing but ordinary reasoning; rich knowledge but a thin deep tail; a strong presence, but not the frontier.


9. Limitations

For a more reliable conclusion, build a canonical holdout sample from scratch (50 questions per tier, 350 total) and lock all experiment parameters before running.


10. Conclusion

In an informal 320-question IKP test, GPT-4.5 shows about 1.3T–1.8T of equivalent factual capacity. Combined with pricing and speed evidence, its architecture is most likely a large-active MoE with 1T–3T total parameters and several hundred billion active.

It isn’t a small model, an ultra-sparse MoE, or a multi-trillion dense model. It’s a giant model with very thick language modeling, very high cost, and outstanding writing ability, but it doesn’t belong to the new generation of reasoning-first models.