Analysis: The open-source content of TileKernels corresponds to the V4 architecture specifications disclosed by Yifan Zhang

robot
Abstract generation in progress

CryptoWorld News reports that the specifications of the V4 architecture disclosed by analyst Yifan Zhang correspond in multiple places with the open-source TileKernels core library from DeepSeek. Zhang states that the residual connections in V4 use manifold-constrained hyperconnections (MHC), which is an improved version of the HC with double random matrix constraints proposed by the ByteSeed team in 2024. By analyzing the TileKernels core code, the V4 architecture is inferred, with three core matches and one mismatch. The model card confirms that V4 uses MHC, which is a match. The model card also confirms that V4 is an MOE model, which is a match. The weights are stored using a hybrid of FP4 and FP8, which is a match. The only mismatch is the conditional memory module (Engram), which the model card also does not mention. The model card reveals a new component not covered by TileKernels: the hybrid attention mechanism (CSA + HCA), which is the key to V4’s significant improvement in long-context efficiency, with inference FLOPS at only 27% of V3 at 1 million tokens, and KV cache at only 10%. Training was switched to the Muon optimizer.

View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pin