Rosinality

I post the papers I find interesting. There are so many papers published these days, and I frequently miss great papers. I appreciate paper recommendations via DM, but I tend to only post papers I discover on my own to keep my list personally curated.

arxiv.org/abs/2611.07232

Tree-structured attention is popular now? But I am not sure whether it could support exact retrieval well enough (and potential risk of aliasing!).

유럽 국가들은 아무렇지도 않게 이미 퇴락한 국가들 취급을 하다가 타겟이 일본이 되니 논쟁이 벌어지는 게 재미있음. 여전히 한국인들에게 일본은 중요한 국가이긴 한 듯.

Exciting implication: previous observations on sparsity of RL policy updates mainly due to ignoring master weight updates. Still it is sparse but much less.

arxiv.org/abs/2611.03142

Resolving the diminishing effectiveness of depth scaling. Cross-layer connectivity through KV sharing, and this improves both efficiency and performance. Interesting result!