moot✳
A little more connected.
Join Moot

Dario> We are virtuous!

(steals the internet)
China> We stole the internet frfr

bebu@bebuu_ai

http://z.ai/ chief scientist Jie Tang just revealed the training pipeline behind the upcoming GLM-5.5 (32B, 122B, and 256B) during a lecture at Tsinghua University today.

Instead of relying on public datasets, their pretraining pipeline pulls from six separate internal engines:

  • Web: 1 trillion pages crawled at ~10B pages/day with PB-scale template clustering
  • Documents: 2 billion scientific PDFs parsed through GLM-OCR (4x faster with speculative decoding)
  • Books: 50 million books with layout reordering and garbled-text filtering
  • Code: 500 million repositories turned into executable tasks with real tests and issue descriptions
  • Video: 10 billion videos for long-sequence world knowledge
  • Audio: 300 million hours rewritten via ASR into trillions of tokens
View on X
#ai#geopolitics#humor
1
0 comments
0 comments