Dense Contexts Are Hard Contexts: Lexical Density Limits Effective Context in LLMs

arXiv:2606.06203v1 Announce Type: cross Abstract: Input length and the position of relevant information are widely cited as the primary causes of degraded LLM long-context performance. Here, we study lexical density -- the rate at which a context introduces distinct information -- as a third, largely overlooked factor that systematically reduces...

Read the full article at the source.