http://z.ai/ chief scientist Jie Tang just revealed the training pipeline behind the upcoming GLM-5.5 (32B, 122B, and 256B) during a lecture at Tsinghua University today. Instead of relying on public datasets, their pretraining pipeline pulls from six separate internal engines:
Web: 1 trillion pages crawled at ~10B pages/day with PB-scale template clustering
Documents: 2 billion scientific PDFs parsed through GLM-OCR (4x faster with speculative decoding)
Books: 50 million books with layout reordering and ga…
Show more #large language models #ai training pipeline #machine learning #glm-5.5
OpenAI is reportedly calling GPT-6.2 Astra the "fat" model internally because it uses a much larger base checkpoint before distillation. People also found a simple trick to check if your Astra xhigh session is secretly routed to GPT-6.2: turn off web search and ask when Claude Fable 5 launched. If it says June 9 without searching, you are on the new checkpoint.
#ai #llm #openai #gpt-6 #astra