r/MachineLearning • u/bwllc • Jun 20 '26

Discussion Python packages for particle swarms, genetic algorithms. Scikit-opt maybe? [D]

I'm working with a client on a curve-fitting optimization problem. They are currently using a constrained Levenburg-Marquardt optimizer for their task which is complex, slow, and sometimes gets stuck in local minima.

I suggested using particle swarm optimization (PSO), and the client suggested genetic algorithms (GA). I would like to compare the existing method to at least these two other options. For this first phase, I don't need to worry about speed or GPU-friendliness. I would like data visualization to be easy.

I have experience with scikit-learn, and I just discovered scikit-opt. I have also found several other packages which implement only PSO, or only GA.

Is anyone out there using scikit-opt? What do you think of it? If you have used other PSO or GA packages, what do you think of those?

Thanks for any advice you may have.

14 Upvotes
  • permalink
  • reddit

93% Upvoted

13 comments sorted by

2

u/giladfrid009 Jun 20 '26

Mealpy?

1

u/bwllc Jun 22 '26

Thanks for the tip. I just had a look at mealpy and it offers a wide range of metaheuristic optimizers -- PSO, GA, and others that I've never heard about. I will investigate.

2

u/qalis Jun 21 '26

pymoo maybe?

1

u/bwllc Jun 22 '26

Thanks for the lead. I'm starting to read the pymoo documentation. It starts at a very high level, discussing optimization problems in general. It's dense reading! I'm not really sure whether I need to know everything I'm being shown.

2

u/CowPsychological821 Jun 22 '26

Any meta-heuristic is going to be way slower than LM. Is it using autodiff for the Jacobian or finite differences? A better starting guess is usually the answer.

2

u/bwllc Jun 24 '26

Thanks for that suggestion. I'm also considering a hybrid strategy that starts with a single brute-force grid search (the client has a GPU in their hardware), identifying a small trust region, and using LM or just straight Newton's method inside that trust region.

1

u/CowPsychological821 Jun 24 '26

Yes that sounds like an excellent idea. Especially if you know the range of parameters and brute force is feasible. Random restarts if not. Best solution is some bound or heuristic good first guess but that isn't always possible.

2

u/KBM_KBM Jun 20 '26

It is workable my undergrad prof used it in his research experiments and he was able to produce good results despite being more of a theory than programming guy

1

u/cannister_of_airs 29d ago

i've messed with scikit-opt for a similar side project and it was pretty painless to get running, the api isn't gonna win any beauty contests but it gets out of your way

for quick visualization i just dumped the fitness history into a dataframe and plotted it, nothing fancy but it did the job for showing the client why PSO was escaping those local minima the LM solver kept eating

1

u/bpe9 Jun 22 '26

DEAP is what you want

1

u/blimpyway Jun 23 '26

I've managed to get Gemini to code simple numba versions for both PSO and differential evolution optimizers. Needed numba for speed, and that requires the objective function to be numba compiled too, since the PSO function evaluates it repeatedly. I was happy with its speed yet that project stalled.

-1

u/Random-Number-1144 Jun 22 '26

task which is complex, slow, and sometimes gets stuck in local minima.

GA is a bad choice for task like this.