The model fits on a single DGX Spark and runs at 30 - 40 tokens per second, beating open-weight systems near 10x its size.


We over-provision because we assume scale is the only lever, but they pulled persistence and backtracking out of a small model instead.
If a team under 70 people can build infra this tight, what does that say about how lean the rest of us builders and operators could run if reproducibility were the first feature instead of the last?
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pinned