Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding

Structured coordinate generation for vision-language models, from axis-specific digit tokens to geometry-aware post-training.

Xiuyuan Zhu^1,2 Ke Lu^1,3 Kun Dong^1,2 Siwen Jiao⁴ Hao Wu^1,2 Zijin Du¹ Shun Mao^1,2 Dongming Zhang² Jian Xue^1,*

¹ University of Chinese Academy of Sciences

² State Key Laboratory of Communication Content Cognition

³ Peng Cheng Laboratory

⁴ National University of Singapore

arXiv Paper PDF Model BibTeX Contact

A coordinate becomes a short, structured sequence 478

x-axis

Flat <478>

Hundreds <x_hundreds_4>

Tens <x_tens_7>

Ones <x_ones_8>

Abstract

Generative Vision-Language Models commonly treat bounding-box coordinates as independent output symbols, leaving numerical order and axis semantics implicit. Hi-Token encodes each coordinate with axis-specific tokens for the hundreds, tens, and ones digits, adding coarse-to-fine structure and increasing token reuse without changing the VLM architecture. Hi-GAR complements this representation with geometry-aware GRPO rewards based on box overlap and coordinate accuracy at multiple scales. Controlled comparisons show that Hi-Token improves localization throughout the evaluated IoU range, while Hi-GAR further reduces low-overlap predictions. Experiments on three VLM backbones and the RefCOCO family demonstrate consistent gains across models and benchmarks.

Coordinates should carry structure

Flat location tokens make nearby values unrelated symbols. Hi-Token exposes numerical scale and axis roles through a compact, exactly parseable output vocabulary.

Flat tokenization 1,000 types

<323> <324>

Adjacent values use independent atomic tokens, and the same vocabulary must implicitly learn both axes.

Hi-Token 60 types

x_hundreds_3 x_tens_2 x_ones_3

x_hundreds_3 x_tens_2 x_ones_4

Neighboring coordinates reuse coarse tokens while preserving the fine change in the ones digit.

tokens per boxThree ordered decisions for each coordinate

coordinate typesSeparate x- and y-axis vocabularies

50×

mean supervisionMore observations per coordinate-token type

Hi-Token and Hi-GAR

Hi-Token changes the coordinate representation. Hi-GAR supplies geometry-aware feedback during post-training and adds no inference-time module.

Hierarchical tokenization

Factorize each coordinate into axis-specific hundreds, tens, and ones tokens.

Geometry-aware reward

Combine box IoU with coordinate checks at coarse, medium, and fine tolerances.

Validity gating

Apply coordinate-level rewards only after the prediction has meaningful overlap.

Overview of hierarchical coordinate tokenization and the Hi-GAR geometry-aware reward used with GRPO — Hi-Token produces a structured 12-token box sequence; Hi-GAR combines format, IoU, hierarchical coordinate, and threshold rewards behind a validity gate.

Results across the IoU range

Under matched settings, Hi-Token improves target identification and boundary alignment. Hi-R1 remains compact, using a 3B backbone and 80k grounding samples.

mIoU 84.3 Hi-R1 on RefCOCO

P@0.5 93.1 Hi-R1 on RefCOCO

P@0.95 33.4 Hi-R1 on RefCOCO

Backbones 3 Positive representation gains

Analysis of token supervision density, cross-backbone transfer, and stage-wise IoU distribution — Denser token supervision, transfer across three VLM backbones, and stage-wise changes across the IoU distribution.

RefCOCO family Comparison with specialist VLMs

Method	RefCOCO			RefCOCO+			RefCOCOg
Method	mIoU	P@.5	P@.95	mIoU	P@.5	P@.95	mIoU	P@.5	P@.95
VLM-R1	63.1	69.8	14.4	64.4	71.5	13.6	66.7	73.4	13.7
Rex-Omni	81.9	88.2	31.1	77.5	83.4	33.3	77.3	86.1	36.3
Hi-R1	84.3	93.1	33.4	81.8	89.9	32.9	80.3	86.4	39.4

All values are percentages. The highlighted row is our final Hi-R1 model.

Qualitative grounding

Hierarchical coordinate generation produces tighter boxes in crowded scenes, while Hi-GAR refines boundaries after supervised fine-tuning.

Comparison of Base Model, Rex-Omni, Hi-R1, and ground-truth boxes for a zebra grounding example — Hi-R1 isolates the referred zebra and closely matches the target box.

Examples showing bounding-box refinement from Hi-Token SFT to Hi-R1 — Geometry-aware post-training tightens boundaries for different object types.

What the diagnostics show

The analysis separates representation effects, reward-stage refinement, digit-boundary behavior, and scale-dependent coordinate sensitivity.

Analysis of digit-boundary effects, localization by object scale, and scale-dependent coordinate sensitivity — Hi-Token is stable near tens transitions; localization quality and coordinate sensitivity vary with object scale.

Citation

If you find Hi-Token useful, please cite our paper.

@article{zhu2026hitoken,
  title   = {Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding},
  author  = {Zhu, Xiuyuan and Lu, Ke and Dong, Kun and Jiao, Siwen and Wu, Hao and Du, Zijin and Mao, Shun and Zhang, Dongming and Xue, Jian},
  journal = {arXiv preprint arXiv:2608.03471},
  year    = {2026}
}

Contact

For questions about the paper, code, or model, please contact us by email.

Email zhuxiuyuan22@mails.ucas.edu.cn