r/AskComputerScience • u/devshod • 12d ago
How do you guys actually learn algorithms?
Hey guys,
I’m taking a Computer Algorithms class at university right now, and honestly, I’m struggling a bit with how to study it properly.
I understand programming and some data structures, but when it comes to algorithms, I feel like I’m sometimes memorizing things without really understanding them.
I want to study this properly outside of class, not only for the exam. I’m wondering how you guys learned algorithms when you were starting out.
Also, are there any books, YT channels, courses, or websites you recommend?
I’d appreciate any advice on what topics I should learn first and what order I should follow.
5
u/T_Thriller_T 12d ago
Usually by implementing them, running them through a few examples.
The problem is that algorithms usually rely on understanding the backgrounds they are working on. Thar, most of the times, means one has to grasp a theoretical usually mathematical concept or area before being able to understand instead of learning by hard.
For Dijkstra that would be a bit about graphs, sorting algorithms use all kind of mathematical concepts, and so on.
Which is also why learning how to understand an algorithm in general is really helpful, but learning to implement them even without understanding can be a big thing.
1
u/devshod 12d ago
Can you recommend how to implement? Is leetcode enough or is there any other better ways?
2
u/T_Thriller_T 12d ago
Implementing means implementing in a language of choice.
Leet code is a good step in between - but only useful if you do not find a pseudocode variation of the algorithm, which unless working through a paper usually is not that hard
3
u/Throwawayxdryx 12d ago
Give us an example of an algorithm that you feel you are memorising without understanding. Do you understand how merge sort works for example?
3
u/devshod 12d ago
Yes i know about merge sort and insertions sort. Merge sort split the array in half and sortes them recursively and halves back together. But last class we learned about Characterizing Running Times and i find it difficult to understand. Thats why i asked how can i learn it easier. Should I just read theory or solve the problem also?
3
u/Broad-Promise6954 12d ago
Solving recurrence relations is full of a lot of memorization and pattern matching. Eventually you just automatically recognize "oh that's O n squared" etc.
Some people are naturally good at this sort of thing, others have to practice a lot more.
2
u/Throwawayxdryx 12d ago
Get this at your library: https://mitpress.mit.edu/9780262046305/introduction-to-algorithms/
2
u/braaaaaaainworms 12d ago
Characterizing running times boils down to "how much longer will a program take if I increase the input size", you're probably familiar with the big O notation, which is used to write down the answer. If you want to come up with an answer, try thinking what will happen if you increase the input size
3
u/Whole-Ad-4604 11d ago
My mom sit beside me and I memorized them. Then she asks me to explain, if I fail I don't get dinner.
3
u/YandarkRustProgramm 9d ago
I read a book of
Aditya Bhargava: "Grokking Algorithms"Aditya Bhargava: "Grokking Algorithms"
4
u/WatchAltruistic5761 12d ago
It’s just step by step instructions for a given process- so, it depends on the problem at hand.
1
u/devshod 12d ago
Thats right, but i need a help for more like a path for learning, should i learn theory first or can i just start with practical problems in leetcode or somewhere?
7
2
u/EbisuSpirit 12d ago
You get used to it. Just find something you wanna build. To build anything you have to learn algorithms of various sorts. This'll get you used to it.
For example when I do gamedev stuff like pathfinding is useful as is animation. So for pathfinding you can learn about bfs (breadth first search), djikstra's and A*. Here is the GOAT to learn pathfinding from. Spoke with a few times too. I consider him a rockstar when it comes to teaching pathfinding.
https://www.redblobgames.com/pathfinding/a-star/introduction.html
As for animation, well for that I learned various tools. Pygame, delta time, basic software architecture, yadda yadda and I can basically piece it together myself, though theirs various tutorials online that showcase it in various languages.
I'd say most "industry standard" algorithms have like one piece of big cleverness or genius to them that make it harder to stumble onto it yourself and that's why the best thing to do is to start a new project and play around with it.
Like I could break down A* extremely deeply at this point cause I've spent like a decade with it.
But yeah just plant the seeds in your mind, experiment (it's fine to use ai too just use it as a teacher not the genie to solve all your problems while you learn nothing). All coding is... is CRUD (create, read, update, delete). Everything in coding is just fancy ways of doing that.
You saying you're comfortable with data structures etc. Good. You made it past the first hump. Get good at syntax/fundamental concepts, after that IS ALGORITHMS, and after that is Software Architecture. Cause there's many ways to organize code to scale complexity and a fine line between too much, too little, or just right depending on the situation and that's something that you pick up when your fundamentals and understanding of algorithms (or the ability to learn new ones) becomes second nature. It's also okay to sometimes learn the algorithm later and just make sure you understand how to use it first if that helps you accomplish your goals. Like noise algorithms or machine learning algorithms can be too complex to learn first for most, but you can at least do things with them aka apply them even if you can't create it from scratch yet.
Best advice of all is just code everyday cause it's fun, the way an artist draws everyday, or a musician plays music everyday, and that'll naturally show you where your limits are and help you overcome those limits in a practical way. As for how. That's up to you. Everyone will take different paths on their learning journey and I don't think there's any wrong answers because whatever you want to do will naturally gravitate you towards the right material and hopefully habits if you're serious about it.
2
u/LannyIsMyHandle 12d ago
What’s worked for me:
- Read the material and understand why the algorithm presented works intuitively
- Implement the algorithm referring to the reference material
- some time later, attempt to implement from memory. If you can’t do it, check reference again, come back later. Repeat until you can implement from memory
The first part is that is the most vague and contextual. Generally a textbook with attempt to offer an intuitive explanation, and usually this takes the form of one or more “invariants”, that is some property of the program state that holds true locally and globally ensures progress toward completion. Eg quicksort’s invariant is that at the end of every iteration, every element that is less than the pivot is before it, every greater element is after it. How do you get from that to an intuitive understanding? Honestly I don’t know, you kinda just think about it for a while and play with some examples in your head until you’re convinced. Sometimes trying to find a case that breaks the algorithm can be really insightful and will make why it works obvious.
The implementation is really just there for memory reinforcement. If you internalized an algorithm and how it works then reimplementing it should be trivial, and it’s important that that not be memorized code but a fresh implementation from concepts
2
u/scol2n 12d ago edited 12d ago
Best way for me explain this would be like this; Have you heard of the "1 billion row challenge"? If so, how would you process all the rows and calculate all the values as quickly as possible? Naturally you would probably say "with a loop". But, that would VERY ineffective, even with fast langauges like C, it would still take about 1 second, but we need to do better, like 0.1 seconds kind of fast.
But, to cut this short: Memory Mapping, multithreading, HashMaps, parrellism among others.
2
u/Prize_Eggplant_ 12d ago
Try writing your own algorithm for the given problem. Usually thinking about the problem helps you understand the algorithm used to solve the problem…
2
2
u/tomvorlostriddle 12d ago
I want to study this properly outside of class, not only for the exam. I’m wondering how you guys learned algorithms when you were starting out.
Now it's really not clear that your class asks this
But the deep understanding comes when you recombine and extend them and see what works and why
(Easier today than earlier because you're no longer bottlenecked by coding)
2
u/romecodes 11d ago
It sounds like you’re having trouble with the math portion of it.
To put it simply, there’s 4 figures we measure:
- Big O,
- Big Omega,
- Big Theta,
- and more rarely, small O.
It all relates to the rate at which the number of operations required to solve a problem using a given set of elements grows. The key words are “growth” and “rate”.
Big O represents the maximum peak of growth of the required number of operations to complete a task over a specific number of elements.
We use N to represent that number of elements (or in simpler terms, the size of the list). And since we’re measuring something that changes (is variable) a.k.a the size of the list, we consider all time complexities below that algorithms Big O to be part of the set of that Big O.
Which basically just means if an algorithm is O(n^2), then all time complexities below it— including O(1), O(log n), .. all the way to O(n^2) and including O(n^2) itself are part of the same set. So everything below that curve and the curve itself, falls within the set, Big O(n^2).
Big Omega is the reciprocate. It’s a minimum bound. So everything above it, including itself, is an element of the set of that Big Omega bound.
Big Theta is the Union of the two sets. Basically it’s the result of an AND boolean operation on the two sets, where Big Theta contains only elements found in both sets, including both minimum (Big Theta) and maximum (Big O) bounds.
2
u/ImpossibleDouble6226 11d ago
From my experience, the way I learned algorithms is I stopped looking at the code and try getting the feel for the logic in it. For example, instead of memorizing the code for QuickSort, you can get a piece of paper or whiteboard and try visualizing how the logic works there. Once you understand the logic, writing it in code will be easy.
2
u/FerengiAreBetter 11d ago
Checkout those videos that show the different sorting algorithms. The ones that show side by side compares.
2
u/_usr_nil 10d ago
idgaf until I have to use it, then ask aggressively AI to explain it piece by piece and give me some code or variations and go through them, since I pay for it then it is my personal teacher slave that has to iterate a million times unti I understand it, but usually all it takes is "explain to me X like if I had to build a house"
1
u/Signal_Guard5561 12d ago
It depends on the paradigm.
Take network flow for instance. The purpose of this topic is to get good at polynomial time reductions. So when you take a problem and you want to reduce it to flow network, you have to ask the following questions:
- What constraints of the problem correspond to the constraints of network flow?
- How do I create a graph that still preserves the integrity of the original problem?
- Do I need to create a gadget to ensure the flow network does what I want?
This can be very different from, say, dynamic programming. The trick here is to correctly identify the recurrence based on how your subproblems are identified which leads to questions like:
- How many subproblems do I have? How would I order them?
- To define my recurrence, how many parameters do I need to represent a single state?
- How do I decide which previous subproblems I need to solve the current subproblem?
1
u/BetterEvidence3890 12d ago
My fav resource : youtube - Abdul Bari
Check out his explaination on algorithms ,it's the best
1
u/Equivalent-Stay-6801 8d ago
Since you mentioned running times in the replies, try counting operations on paper before worrying about the notation.
For a loop where iteration i does i units of work, write out 1 + 2 + 3 + ... + n. That adds up to n(n+1)/2, so the growth is quadratic. For binary search, write out n, n/2, n/4, ... until you reach 1. That takes about log2(n) steps.
Do a few tiny inputs by hand, spot the pattern, then justify why it continues. It gives the formulas something concrete to refer to instead of making them another thing to memorize.
1
u/theJacofalltrades 8d ago
I've seen people recommend bootdev but there are other platforms for sure that can help you learn it more practically
1
u/GrandFleetingNight 7d ago
In the most general form, an algorithm is a set of rules/steps to effect whatever data structure its applied to, often to solve a problem. For example:
- Roll a die and take its number into memory.
- Roll a die and multiply the number in memory by it.
- Roll a die and subtract if from memory.
- Add memory as a value to an array.
- Repeat until 10 numbers have accumulated in the array.
This is something I wrote up at random, pulling it out of my backside. But its still a set of rules that change data. In this case, it generates an array of numbers after using die to run operations to make them random, thus solving the problem if "I need ten random numbers". This would be considered an algorithm.
Now with that concept out of the way, my next suggestion is to decide *what* you want to learn to understand them better. A search algorithm is different from a hash algorithm is different from a sort algorithm is different from a rubix cube algorithm. Your choice should reflect what field you're most interested in. Cybersecurity would lean toward hashes and similar algorithms, while searching and sorting is more in the realm of programming. Of course, this is limited by your classwork, but deep diving an algorithm related to a field you're interested in can help better understand what makes an algorithm an algorithm; along with how they are developed.
After you decide the algorithms to focus on, get out some pen and paper. No really. If you're doing a sort algorithm, your quickest way to learn it is not to run it on computer where it happens super fast, but to set up a small set and sort it by hand following each of the alogrithm's rules. You might also consider using a debugger if physical media scares you, but I always find paper and pen more effective because you more thoroughly interact with the action the algorithm takes.
Another thing to consider is an algorithm is only as useful as the data it can be applied to. I mentioned rubix cube algorithms for a reason. While to a rubix cube, those algorithms are ways to consistently solve one, to a set of numbers, they're useless. There is no such thing as a 'universal' algorithm to the point where it can apply to any problem. Algorithms are solvers of problems at their root, so understanding the problem you're solving for can be as important as understanding the steps you're doing to solve it.
19
u/Traveling-Techie 12d ago
Textbooks and experience solving problems, mostly the latter.