Analysis: TileKernels Open Source Content Corresponds with Yifan Zhang's V4 Architecture Specifications

According to monitoring by Dongcha Beating, the TileKernels kernel library open-sourced by DeepSeek corresponds in multiple ways with the V4 architecture specifications previously disclosed by Yifan Zhang. Zhang stated that the V4 residual connections use Hyper-Connections. The open-sourced TileKernels feature mHC (Manifold-Constrained Hyper-Connections), which is an improved version of the HC proposed by the Byte Seed team in 2024, addressing the signal divergence issue encountered during large-scale training with the original HC. mHC itself is a type of Hyper-Connections, as the original HC cannot support stable large-scale training; thus, mHC is likely what is actually used in V4. Zhang mentioned that V4 employs Fused MoE Mega-Kernel to manage 384 expert activations across 6 MoE layers, while the MoE module in TileKernels includes Top-k expert selection, token-to-expert mapping, and the distribution and collection of fused experts. TileKernels also contains the Engram kernel, which is a conditional memory module proposed in a paper by DeepSeek earlier this year, but Engram is not mentioned in Zhang’s V4 specifications. The library supports SM90 (Hopper) and SM100 (Blackwell), but does not support Huawei Ascend. Previously, The Information reported that V4 was trained on Blackwell, and DeepSeek has spent months adapting the model for Huawei and Cambricon chips.