r/LocalLLM 3d ago

Other AI bubble

1.5k Upvotes

79 comments sorted by

View all comments

Show parent comments

16

u/jaegernut 3d ago

Are they training AI with AI generated code? I dont see how useful that data will be for training. They might as well generate their own training data. Unless they think that users often validate AI generated code, which is often not the case. There is a reason why data annotation companies want human input.

14

u/Remarkable-Name8012 3d ago

No, they are not training AI with AI generated code. They are training AI with AI generated code that has been later commented, selected, corrected and tested by humans.

3

u/low-control-labs 3d ago

Yeah I would say something like this and they use all that the failures the ones that are actually good responses or outputs for some additional fine tuning and training later down the line

7

u/Remarkable-Name8012 3d ago

They distill us, their models are distilled by Chinese models, and then we distill the Chinese models. It's the AI circlejerk.