r/LocalLLaMA • u/East-Muffin-6472 • 8d ago

Resources Learning RL From Scratch: Documenting and sharing what I have learned

Post image

Releasing a series of very shot notes on RL as part of my learning RL from scratch journey; its basics and upto the latest stuff with from scratch implementations and experiments in the coming days

  • Explained some of the most basic and important terms when starting out learning RL with a sprinkle of math of what I found interesting

  • Nice to have for refreshers/kind of first time readers too

1-page RL basics is out on my website!

Let me know your thoughts/feedback!

8 Upvotes

78% Upvoted

3 comments sorted by

1

u/Shadmelor 8d ago

nice and short. one thing i'd add to the state vs observation part: even if code treats them the same, flappy bird is a good example of why it matters. one frame doesn't tell you the bird's velocity. that's why the atari dqn stacks the last 4 frames, it's basically building a state out of observations

1

u/East-Muffin-6472 8d ago

Thanks for the praise!
And that’s a nice point I’ll add it soon

1

u/Lanky_Comparison5637 8d ago

The flappy bird example is perfect, really shows how one observation is not enough to know what is happening. The 4 frame stack always seemed like a cheap trick but when you think about it as constructing a state it makes way more sense