Knowledge base›Sources›Metehan Yesilyurt
Yesilyurt dissects the 200,000-entry vocabulary of the GPT-5 tokenizer: some brands exist as a single token, others split; URLs always split; prose is the cheapest format in tokens, tables the most expensive; over a thousand token slots are reserved for tools. His thesis: what is cheap to tokenize is easier for the model to process and reproduce.
As a mechanics hypothesis behind two of our own findings: first, ChatGPT partly phrases fan-out queries in English even for German questions (Share of English fan-outs) - English text tokenizes more efficiently in the o200k vocabulary, which would be a motive. Second, the single-token thesis makes it plausible why well-known brands surface so effortlessly in self-written queries. Both remain conjecture, but as the only mechanics explanation of this kind it belongs in the register.
None of this is observable in the client capture - the tokenizer works before everything we see. The claims are therefore all unverifiable, but externally checkable (tiktoken) and not in contradiction with our data.
| Claim | Status | Evidence |
|---|---|---|
| Brands like Google, Amazon, Reddit are single tokens; OpenAI and ChatGPT split into several | unverifiable | A tokenizer property, independently checkable with tiktoken - but not observable in our setup (HAR captures). No contradiction with our data. |
| URLs are never atomic tokens, they always split into fragments | unverifiable | Same situation: static vocabulary property, outside our measurement window. |
| Format efficiency: prose ~5.9 characters per token, markdown 4.8, JSON 4.0, tables 2.7 | unverifiable | Not checkable client-side; our snippets (202 characters index, 131-158 scrape) say nothing about token costs of the source page. |
| Around 1,075 reserved tool slots in the vocabulary point to planned tool expansions | unverifiable | A vocabulary observation; our captures only show the tools actually active. |