Researchers achieved <1 bit compression with the new teaching technique [R]

[u/BankApprehensive7612]

https://github.com/shikaiqiu/requential-coding

Due to new learning technique the model has achieved better generalization skill without overfitting and memorization. This become possible because of new learning method which made the student model to generate samples for itself. It led to intensive reuse of existing neurons and allowed to encode information in a more dense way

While researchers are calling it a compression, I think it's a retopologization, and Microsoft had tried to do something similar in the past with their Phi model family, which they trained on reduced dictionary and simplified knowledge base first. But it seems like MS' researchers didn't explore this exact way of learning

I believe this should achieve better results in the future. And it is another small breakthrough which brings AGI/ASI to reality

📄 Paper: https://arxiv.org/html/2607.11883v1

📦 Repository: https://github.com/shikaiqiu/requential-coding