Beyond Semantics — The Unreasonable Effectiveness of Reasonless Intermediate Tokens

§ 01 — Setup

A gridworld where every thought can be checked.

Pre-trained models make trace-validity a matter of interpretation — natural-language traces are too ambiguous to formally audit. So we shrink the problem until it becomes auditable: 30×30 mazes, four orthogonal actions, a vocabulary of 944 tokens, and an A^* validator that replays every create/close operation against an open and closed list.

In-distribution · Wilson's algorithm (spanning tree)

start goal A^* path explored wall

Out-of-distribution · Drunkard's Walk (caves with loops)

5 generators total: Wilson · Kruskal · DFS · Drunkard · SF-style

Generate mazes

30×30 grids from 5 algorithms, acyclic trees and cyclic caves.

Run A^*, linearize trace

create x y cG cH / close x y cG cH token stream.

Train from scratch

0.5B Qwen2.5 variant, 944-token vocab, 32k context, 500k problems.

Formal verifier

Replays generated trace on open/closed lists. Flags parsing, wall, neighbor, plan-mismatch errors.

§ 02 — Findings

Four results that should not be true.

Finding 01

Correct plans, invalid traces.

Trace validity ≠ solution correctness.

Correct-trace-trained models still produce correct plans from invalid traces: up to 8% of correct plans for the Wilson model and up to 19% for the SF-style model. On the hardest length-generalization bin, only 52% of correct plans come from valid traces. If traces drove solutions, this number would be 100.

Finding 02

Swapped beats correct.

Irrelevant traces match or outperform real ones.

Train models on traces from other mazes (the "Swapped" setup) and accuracy doesn't drop — it rises on OOD. On the cave-like Drunkard set, the SF-style Swapped model hits 95.4% where the correct-trace model gets 62.1%. Trace validity: 0.0%.

Finding 03

RL widens the gap.

GRPO improves accuracy while eroding traces.

Post-training the SF-style correct-trace model with GRPO pushes plan accuracy up and trace validity down from 89% to 52%. The Swapped models keep improving too — still at 0% trace validity throughout.

Finding 04

Length is not thinking.

Trace length ⊥ problem complexity.

Pearson correlation between generated trace length and ground-truth A^* length: r = 0.0081 for Wilson, r = 0.02 for Wilson-model evaluated on OOD SF-style mazes. Longer traces are not "inference-time scaling" — they're sampling noise.

§ 03 — Main result

When nonsense traces learn better.

Plan accuracy on 1,000 held-out problems per maze type, following 500k-sample training runs. Swapped models see reasoning traces from completely unrelated mazes during training; the target plan is still the correct one for the presented problem.

Solution-only baseline Normal (correct traces) Swapped (irrelevant traces)

Wilson · baseline

10.0%

10.0

Wilson · normal

79.9%

79.9

Wilson · swapped

83.3%

83.3

Kruskal · normal

83.2%

83.2

Kruskal · swapped

85.0%

85.0

DFS · normal

54.0%

54.0

DFS · swapped

52.6%

52.6

Drunkard · normal

0.0%

0.0

Drunkard · swapped

11.7%

11.7

Normal (correct traces) Swapped (irrelevant traces)

Wilson · normal

40.8%

40.8

Wilson · swapped

45.8%

45.8

Kruskal · normal

44.6%

44.6

Kruskal · swapped

51.0%

51.0

DFS · normal

19.9%

19.9

DFS · swapped

29.0%

29.0

Drunkard · normal

62.1%

62.1

Drunkard · swapped

95.4%

95.4

SF-style · normal

56.2%

56.2

SF-style · swapped

89.1%

89.1

The U-shape: how much swapping is too much?

Instead of fully replacing the trace distribution, we sweep the proportion of examples receiving mismatched traces. A little swapping hurts — a lot of swapping helps. Plan accuracy traces a U; trace validity monotonically collapses. Wilson-trained, 50k samples.

Wilson Kruskal DFS Drunkard

(a) Plan accuracy vs % swapped traces

Add 25% swap: accuracy dips. Reach 75–100% swap: accuracy climbs back — and exceeds the normal baseline on Drunkard (OOD).

(b) Trace validity vs % swapped traces

Trace validity falls monotonically from ~97% (Normal) to 0% (fully Swapped). No U-shape here — accuracy and trace-meaning have decoupled.

What matters for transformers to improve accuracy with intermediate tokens is not their semantic correctness, but rather their consistency.

And RL makes it worse for the realist.

GRPO post-training checkpoints for both model families. Each cell shows plan accuracy / trace validity within valid plans, in %.

Table 4 Normal and Swapped Wilson models across maze distributions, by GRPO checkpoint.

Checkpoint	Model Type	Wilson	Kruskal	DFS	Drunkard	SF-style
0	Normal	79.9/ 97.1	83.1/ 95.6	54.0/ 87.4	0.2/ 0.0	0.0/ —
0	Swapped	83.3/ 0.0	85.0/ 0.0	52.6/ 0.0	11.7/ 0.0	0.2/ 0.0
70	Normal	88.9/ 97.9	89.1/ 98.1	62.3/ 92.1	0.0/ —	0.0/ —
70	Swapped	93.4/ 0.0	93.5/ 0.0	67.1/ 0.0	14.4/ 0.0	0.4/ 0.0
140	Normal	89.6/ 98.2	91.7/ 98.2	62.4/ 92.9	0.0/ —	0.0/ —
140	Swapped	99.5/ 0.0	99.3/ 0.0	81.9/ 0.0	14.4/ 0.0	0.4/ 0.0

Table 5 Normal and Swapped Searchformer (SF-style) models across maze distributions, by GRPO checkpoint.

Checkpoint	Model Type	Wilson	Kruskal	DFS	Drunkard	SF-style
0	Normal	40.8/ 89.0	44.6/ 93.0	19.9/ 85.4	62.1/ 85.2	56.2/ 81.1
0	Swapped	45.8/ 0.0	51.0/ 0.0	29.0/ 0.0	95.4/ 0.0	89.1/ 0.0
70	Normal	54.2/ 72.0	58.8/ 67.7	20.7/ 68.6	88.0/ 66.9	82.4/ 65.3
70	Swapped	62.3/ 0.0	67.5/ 0.0	39.4/ 0.0	99.6/ 0.0	96.9/ 0.0
140	Normal	63.7/ 52.1	68.8/ 49.9	27.9/ 50.5	91.2/ 45.7	89.1/ 43.4
140	Swapped	69.0/ 0.0	75.5/ 0.0	41.6/ 0.0	99.2/ 0.0	98.5/ 0.0

→ As GRPO pushes plan accuracy up (SF-style Normal: 56.2 → 89.1), trace validity falls (81.1 → 43.4). The swapped model meanwhile climbs to 98.5% with stable 0% trace validity — none of the solving is happening in the visible reasoning.

§ 04 — Trace length

Longer traces are not harder thinking.

A popular reading of CoT length says models "think longer" on harder problems — "inference-time scaling." If that were true, model trace length would track A^*'s ground-truth trace length. It doesn't.

Wilson model · in-distribution (Wilson tests)

Pearson r = 0.40 Spearman ρ = 0.98

Wilson model · out-of-distribution (SF-style)

Pearson r = 0.02 Spearman ρ = 0.02

→ In-distribution, the model learned to imitate the length distribution of its training data, not the computation. One maze-generator further out, that illusion collapses.

§ 05 — Implications

What the traces are not.

"Chain of Thought" is not a chain of thought.

Intermediate tokens help the model — that much is real and reproducible. But in our fully controlled setting, they help without needing to be about the problem they accompany. Anthropomorphic readings ("aha" moments, deliberation, stepwise reasoning) require evidence beyond surface form.

Consistency, not meaning, drives the gain.

A U-shaped curve in accuracy as we sweep swap-rate from 0% → 100% tells the story: well-structured but irrelevant traces work. Well-structured and correct traces also work. What does not work is inconsistent traces. The useful quantity is statistical regularity of the intermediate stream, not its semantic fidelity.

III

Think of intermediate tokens as prompt augmentations.

A reframing: the model is learning a Skolem function P_A = f_θ(T, LLM) that augments the task prompt into a form the base distribution handles better. Prompt augmentations have no need to be human-interpretable — and our results suggest they usually aren't.

§ 06 — Reference

Cite this work.

BIBTEX · TMLR 2026

@article{valmeekam2026beyond,
  title   = {Beyond Semantics: The Unreasonable Effectiveness of
             Reasonless Intermediate Tokens},
  author  = {Valmeekam, Karthik and Palod, Vardhan and Stechly, Kaya
             and Gundawar, Atharva and Kambhampati, Subbarao},
  journal = {Transactions on Machine Learning Research (TMLR)},
  year    = {2026},
  url     = {https://openreview.net/forum?id=gDE7YcRC3F}
}

Read on TMLR Download PDF View Code ← All Yochan projects

Companion work on pre-trained LLMs fine-tuned for QA: Bhambri, Biswas, Kambhampati — ACL 2026. Builds on Searchformer (Lehnert et al. 2024) and Stream of Search (Gandhi et al. 2024).