Tokenizer Lab

Explore Connor Love’s tokenizer alongside nine established tokenizer baselines.

10tokenizers
compared

02 Token count

Token count by tokenizer

0 / 10 measured

03 Tokenizer output

Byte-lossless Unigram · research candidate

o200k_base

Anthropic legacy tokenizer · local proxy

Gemma 4 12B · open model

Kimi K2.6 · open fallback

DeepSeek-V3.2

Grok-1 · open local proxy

Mistral Small 4

Qwen3