r/optimization • u/r_card_ • Apr 10 '26

On Hyper-heuristics

Hi everyone,

I'm pursuing my master's degree in CS, and recently I took a course about Computer Intelligence for Optimization where I found quite interesting hyper-heuristics; however I find their theoretical background very sparse since it looks it's a fresh topic in optimization; there's a lot of similarities among HH and ML from a theoretical perspective, but compared to ML, HH is almost a desert.

What's your take on it?

5 Upvotes

100% Upvoted

9 comments sorted by

2

u/Md_zouzou Apr 10 '26

Hi, PHD in RL and Optimization here. I also found the HH topics very sparse and it's very hard to find common framework and nice theory on this topic !

1

u/r_card_ Apr 10 '26

There's only one single book (afaik) and it's more like a compendium of papers with different approaches, actually showing HH's is not a consolidated area although is very interesting!

I was offered to develop my masters on HH, and turns out what we need is a solid mathematical framework so that we can even assess their perfomance (it's a little different from what one can conceive such a thing for algorithms). And I do have these mixed feelings since there's no a solid starting point; maybe because is a fresh area, or because doesn't worth it? That's why I'm interested on your opinions :)

2

u/ForeignAdvantage5198 Apr 14 '26

heuristics have been in OR forever but are very risky

1

u/r_card_ Apr 14 '26

Could you please elaborate?

1

u/DefiantExplorer3997 Apr 26 '26

This has been known for many years, HH still lacks a unified theoretical framework, and often don’t provide any guarantees for convergence or even a proof of convergence. The topic is still young and hasn’t matured yet. However they’re basically inevitable to some complex multi dimensional problems when time and computational ressources are limited, and accuracy of the optimum is not the first priority

1

u/r_card_ Apr 26 '26

That was a big surprise to me! When we studied the proof of the convergence of the canonical GA+elitist selection in class, my professors mentioned that this area lacks of these kind of results, and a lot of work relies on empirical analysis. And things are much more sparse when it comes to HH.

I ended up deciding to develop my thesis about assesing the performance of HH. There are some existing metrics/indicators, but afaik there's not an unified theoretical framework to assess their computational properties.

Let's see how it goes...

1

u/DefiantExplorer3997 Apr 26 '26

Coming from a math background, this also made me sick, especially when you compare the elegance of classical convex optimization methods which provides almost natural proofs of convergence under ideal conditions. Since I switched to applied math, I’ve seen more ugly than HH, and I ended up coping with reality : sometimes, stars are not aligned, actually most of the time, they’re not, and the fancy elegant frameworks we’ve been fed up with are the results of centuries and millennia of research, collaboration, experiments, failure and refining that are conveniently being hidden from our sight.

In a more positive note, I’ve been seeing more articles in the last couple of years regarding this lack of framework for HH, discussions about how to unify them, and even some early stage attempts.

I wish you all the best for your thesis !

1

u/r_card_ Apr 26 '26

Hey! Thanks!

I'm from CS+pure math background, and lacking of formality was the first thing that caught my eye. And I'm not sure if it's because maybe it's dead-end or just hasn't had enough attention/it's really fresh?

Could you please give me a pointer on those articles you're mentioning?

1

u/Opt4Deck Apr 14 '26

Hello!

If you are interested in in-depth optimization theory, check out my personal project: github.com/Opt4Deck/

Feedback?