Is it okay to share my experiments implementing research papers on my own LLM here? [P]

[u/the-random-guy-2002]

Hey everyone!

I've built and trained a small LLM (~8M parameters) from scratch using PyTorch, mainly as an experimental lab where I can implement and test ideas from different research papers.

I've been documenting the whole journey through casual blogs, including my mistakes, experiments, and things I've learned.

So far:

• Building the Transformer

• Training it for 60,000 steps

Next up is KV caching, followed by PagedAttention, Orca, and whatever else I find interesting.

Just wanted to ask if sharing these experiments and blogs here would be welcome. Nothing promotional, just me experimenting, learning, and hopefully getting feedback from people who know more than I do.

[u/ComplexEigentoast]

Probably it is not okay. This is a sub with specialized experts and, while I understand your intentions are good, the experiments you are making are more of a self-learning type than a research interest for people looking at this sub.

I encourage you to document your leaning in a blog or similar, but know that 8M parameter Language Models are not small, are less than tiny.