A systematic study of Hyper-Connections architectures identifies the simpler efficient design SimpleHC, which removes residual mixing and uses shar...
论文概览
- arXiv ID:
2611.05984 - 发布机构 / 来源:dots studio
- 作者:Xiangdong Zhang, Xiaohan Qin, Sunan Zou et al.
- 社区热度:6 Upvotes
核心摘要(Abstract / AI Summary)
A systematic study of Hyper-Connections architectures identifies the simpler efficient design SimpleHC, which removes residual mixing and uses shared memory access to match mHC performance while adding minimal training overhead and improving stability.
