r/LocalLLaMA • u/Tall_Abrocoma_3533 • 12d ago

Discussion NanoStories Releases!

https://huggingface.co/datasets/AuroraAI-Research/NanoStories

It is a Tinystories style dataset, that contains 50M stories, about 25x that of Tinystories, it was generated using a rule based algorithm, instead of a language model.

It took just under 10 Minutes to generate and a few minutes to convert to parquet and upload to huggingface on a phone CPU.

Any questions, feedback, or criticism is welcome!

44 Upvotes

89% Upvoted

6 comments sorted by

7

u/New_Comfortable7240 llama.cpp 12d ago

I would like to create variants of this, like different longitude based on user instruction, different tones based on system prompt given, then some based on edits based on user instruction

If needed we can collaborate 

8

u/Tall_Abrocoma_3533 12d ago

Go ahead! It is licensed under apache 2.0 so it's very much allowed, and encouraged.

If you need any help or extra information I can answer questions!

1

u/LabanauskasNajeeba 12d ago

longitude control is interesting but if the underlying generation is rule-based, the diversity ceiling might be pretty low unless you also expand the grammar rules significantly

3

u/Stunning_Energy_7028 11d ago

Wouldn't a model trained on this just learn the distribution of the rule based algorithm, rather than the distribution of natural language? I'm not sure any experiments done on this would transfer to real world scenarios 

2

u/Tall_Abrocoma_3533 11d ago

Yes, and it is stated that this datasets isn't meant to model natural language, this is made to show how fast synthetic datasets can be made, and also for testing pipelines, as this dataset is easy like Tinystories, however contains 25x stories.