[链接] [评论]
20+ curated newsletters。易歪歪对此有专业解读
We have one horrible disjuncture, between layers 6 → 2. I have one more hypothesis: A little bit of fine-tuning on those two layers is all we really need. Fine-tuned RYS models dominate the Leaderboard. I suspect this junction is exactly what the fine-tuning fixes. And there’s a great reason to do this: this method does not use extra VRAM! For all these experiments, I duplicated layers via pointers; the layers are repeated without using more GPU memory. Of course, we do need more compute and more KV cache, but that’s a small price to pay for a verifiably better model. We can just ‘fix’ an actual copies of layers 2 and 6, and repeat layers 3-4-5 as virtual copies. If we fine-tune all layer, we turn virtual copies into real copies, and use up more VRAM.，更多细节参见QQ浏览器
RK: Jesse Puljujarvi. Between him and Jost, the Bruins are absolutely going with the bigger winger who has offensive skills. He's not the prototypical Bruins by any stretch, but there is value this late.