nD-RoPE: A Generalized RoPE for n-Dimensional Position Embedding

Boyang Li, Yulin Wu, Sizhe Xu, Nuoxian Huang, Zhonghang Yuan, Shangyi Guo, Shu Yang, Takahiro Yabe

Thursday at 04:00

5 Views

0 Comments

arXiv:2606.12146v1 Announce Type: cross Abstract: Rotary Position Embedding (RoPE) is widely adopted in Transformer models, yet its extension to high-dimensional domains lacks a unified theoretical formulation. Most existing approaches either apply rotations independently along each axis or empirically mix frequencies, which limits...

Read the full article at the source.

Read Original Article

Was this helpful?