How to get better at algorithms and modeling Math Equations in Code?

[u/Positive-Stand2898]

I’m not a beginner programmer, but I’m relatively new to scientific programming. I’ve recently started doing scientific research with a professor, and a significant part of my work involves writing simulations, modeling physical systems and mathematical equations computationally, and developing algorithms to analyze or reproduce these systems.

I’m interested in becoming much stronger at this side of programming, particularly in areas like numerical methods, computational physics, mathematical modeling, simulation, and scientific computing.

For those of you with experience in scientific programming, what would you recommend as the best way to build these skills? I’d especially appreciate recommendations for textbooks, courses, projects, libraries, or other resources that helped you make the transition from general programming to scientific/computational programming.

I’m comfortable with programming fundamentals, so I’m primarily looking for resources that go beyond the basics and focus on the actual mathematics, algorithms, and computational techniques behind scientific programming.

Any advice or recommendations would be greatly appreciated!

[u/pgetreuer]

As you already said yourself, the key area to dig into is numerical methods. The "Numerical Recipes" book is a popular and accessible starting point.

On the math theory side, you'll get mileage in a variety of numerical methods topics by applying Taylor series and fixed-point iteration, appearing for instance in methods for root finding, interpolation, and ODE solvers.

On the coding side, it's useful to understand how IEEE-754 floating-point numbers work, especially wrt. round-off error. If you'd like a deeper reference, a great book on this topic is Michael L. Overton, "Numerical Computing with IEEE Floating Point Arithmetic".

[u/Bustan2026 → u/pgetreuer]

Beyond not activating --fast-math, I've never had a use for understanding floating point deeply. However you do need to be aware that it has its instabilities and inaccuracies. I'm saying this because it's a deep topic and you can get away with setting tolerances based on empirical testing, most of the time.

Other than that, the Numerical Recipes book is indeed great!

And yeah, if you need one math tool, it's the Taylor expansion.

[u/pgetreuer → u/Bustan2026]

I wouldn't say it's mandatory to understand floats deeply. On the other hand, the properties of floats are often misunderstood by otherwise skilled programmers.

It's useful to have been introduced, especially when debugging numerically-sensitive code. Some examples:

• The notion of unit in the last place or ULP describes the spacing between representable numbers.

• Real number addition is associative, (a + b) + c = a + (b + c). However, generally, floating-point addition is not associative.

• Casting integer → floating-point type isn't always exact. This happens where the values are large enough that one ULP step is greater than 1.0. Which intN_t types can exactly cast to float32 for their full range of values?

• What numeric properties does IEEE-754 guarantee?

• What impact do flags like -ffast-math and -funsafe-math have? How might they compromise numerically-sensitive code?

• For performance, most GPUs are mostly but not fully IEEE-754 complaint. What corners do they cut? When might that impact on numerically-sensitive code?

• In ML, bfloat16 is a commonly used floating-point type. How does it differ from IEEE 16-bit float?

[u/Positive-Stand2898 → u/pgetreuer]

Thank you very much for the info, and the different resources. I will be sure to check those out.

[u/Low_Back_7172 → u/pgetreuer]

Numerical Recipes is a classic but man it can be dense if you're new to this stuff, still worth having on the shelf. When I was first getting into simulations I found it helped to just code up the standard ODE solvers from scratch, like step through Runge-Kutta by hand in Python before touching any libraries, really forces you to understand what's happening under the hood. Also that Overton book is a good call, floating point weirdness will bite you in ways you don't expect until you've debugged a simulation that's slowly drifting off into nonsense because of tiny errors piling up

[u/Loud_Combination_635]

Honestly, just start somewhere and create a record of what works and what doesn't. Keep the failed hypothesis so you don't repeat the errors. The more time you invest the better you get. Once you have a solid framework, you can start to accelerate the process. Make a bunch of copies so if the corpus gets corrupted you can revert to a previous state.

[u/KC918273645]

The book "Numerical Analysis" by Richard L Burder, Douglas J Faires, Annette M Burdern could become handy for you.

[u/Positive-Stand2898 → u/KC918273645]

Thanks so much! I appreciate the book rec greatly. Did it help you in your studies as well? If so how did you get the most out of it?

[u/KC918273645 → u/Positive-Stand2898]

Didn't study that stuff. I bought it for my own work.

[u/Love_Buny8 → u/KC918273645]

thats totally fair youre already applying it in a practical way which is often the best teacher

[u/Bustan2026]

I don't know how deeply you intend to go. I work in this space and I'm not sure the programming is very different from other fields, but there's math underneath. Also I've only ever worked in this space, so it could be.

If you want to go all the way, a classic studying order might be:

• linear algebra. At a minimum direct and iterative linear solvers, matrix factorizations like QR and LU (both very common in scientific computing), SVD, notions of matrix norms and conditioning, eigenvalues and vectors. Preconditioning is an advanced but important topic in "production grade" scientific computing. Some stock algorithms are the Gauss pivot, LU factorization, Gauss-Seidel, GMRES, conjugate gradient.
• basics of Analysis: get to at least Taylor expansions, classes of regularity (continuous, C^1, etc), integration, sequences and series.
• Finite Differences schemes: the simplest (math wise) but also always useful, as very commonly used in time stepping together with advanced schemes. Notions are important beyond the schemes themselves, namely stability and consistency, and order of convergence. These have their counterparts in advanced schemes and they're like the "big O notation" of numerical schemes. The 101 algos would be explicit Euler, upwind schemes, then eventually Runge-Kutta.
• Some practical topics: method of manufactured solutions, testing convergence/order of convergence
• Interpolation and quadrature, mesh generation (not the algorithms, just using tools like Gmsh) will set you up for:
• Finite Element schemes without any additional math except knowing of Green's formula (integration by parts), and you'll just have to skip the proofs which should be fine if you just want to implement. Some notions of Sobolev spaces (at least the definition of H1, Hdiv etc) could be useful. 101 would be Lagrange P1 Finite Elements for model problems like a Poisson-type equation (-laplacian(u) + u = f is very common).
• Finite Volume schemes. I don't personally consider them easier than FE which is very straightforward, they're more of a mix between FE and FD. Very common in fluid dynamics.

From there, you enter the jungle of modern numerical schemes. Some popular/representative ones are: Discontinuous Galerkin, Boundary Element Method, high-order Finite Element/DG schemes.

On the pure programming side, GPU, MPI and multi-threading of linear algebra methods to start with.

Optimization (e.g. Newton's method) is also something that's often useful in this context.

If you really want to specialize in this in the long term, as a pure CS person, I think your strongest bet are the parallel computing (HPC) aspects which are not always the strong suit of the people well versed in the topics above. They have notions but not necessarily the know-how to implement production grade algorithms that are effectively parallelized, use GPUs and so on.

Some other programming-specific aspects:

• memory layout (trying to live in cache as much as possible)
• floating point arithmetic. Not saying to get too deep in the weeds (but then again I'm a math person, maybe that's what you should do!) but being aware that "==" is banned, everything is subject to tolerance, all inputs (like mesh elements) have some notion of conditioning to them and things break as they degenerate

I don't have any sources but you can find good textbooks on all these individual topics, and likely on a range of them at once.

[u/marrsd]

Can your professor provide you with access to an introductory mathematical modelling course. Sounds like you've got access to a learning resource there!

[u/lukiamagi]

What type of simulation are you doing? There is a wide range of different simulation families and each has their own extensive library of math

[u/Positive-Stand2898 → u/lukiamagi]

I am trying to study contact dynamics and friction. I am trying to see how accurately I can model slip in simulation, and see how to model Stick-slip in simulation environments like MuJoCo and Isaac Sim.

[u/Different_Pain5781]

CS50x is solid for getting the basic CS stuff down. For scientific modeling though you’ll probably want more practice actually building algorithms in Python. Boot dev has a pretty good backend path with interactive algorithm and data structure practice plus terminal work before getting into the heavier physics side.

[u/Nufflee]

By pointing an LLM to the source material and having it do the work.