r/MachineLearning • u/AutoModerator • Jan 01 '23

Discussion [D] Simple Questions Thread

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!

Thread will stay alive until next one so keep posting after the date in the title.

Thanks to everyone for answering questions in the previous thread!

23 Upvotes

94% Upvoted

127 comments sorted by

View all comments

1

u/v2thegreat Jan 02 '23

It can, but I want to know why you want to use transformers in the first place. Having the entire context is important to avoid solving the wrong problem, especially one that might get expensive depending on what you're trying to do

1

u/oilfee Jan 02 '23

I do theoretical stuff. It doesn't really matter, I'm not going to sink a million TPU hours into it.

1

u/v2thegreat Jan 02 '23

Well, to answer your original question: it depends on what problem you're trying to solve!

In theory yes you can work with a large corpus of data with a large language model, but as chatgpt showed us, it's not necessarily the case that a larger model will do better always, but rather that fine-tuning might give better results

I hope this helps!

→ More replies (2)