Crypto界消息,OpenAI技术人员Clive Chan表示,V4技术报告整体仍然是顶级水平,但给芯片制造商的硬件建议章节「出人意料地平庸甚至有错误」,与V3形成对比。V3硬件章节的问答曾是学术会议ISCA上最热门的讨论环节,建议内容具体到行业正在制定的互连标准,V4则空泛了许多。Chan逐条提出质疑,认为报告关于功耗的建议「恰恰适得其反」,并质疑GPU之间的数据传输方式选择。关于激活函数,Chan认为报告建议用更简单的函数替换SWIGLU以降低计算负担没有必要,因为Sonic Moe已经证明用SWIGLU照样能达到最优性能。

View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pin