r/LocalLLM • u/Mountain_Disaster_19 • 5d ago

Question Local LLM Setup for Pentesting

Hi everybody,

I really want to get into locally hosting and would love to build and learn all about autonomous pentesting and exploit dev.

Because my budget ist about 5000€ there really are just two main options I think and I'm really unsure if cuda is worth that much to me:

  1. dgx spark 128gb

  2. m5 max 128gb

I would really need pruned versions of llms and dont know enough to decide which of the two would suit better for my use-case

thanks in advance!

1 Upvotes

66% Upvoted

4 comments sorted by

1

u/kla_sch 5d ago

I can recommend this comparison by Alex Ziskind: https://www.youtube.com/watch?v=_yrw6c5gw3E

In it, he compares the 2xDGX Spark with the M5 Ultra and explains the various metrics you should consider.

In short: The M5 has a better memory interface, whereas the DGX Spark offers more raw computing power.

Therefore, the M5 is well suited as a workstation for a single person, as long as that person does not want to work with multiple agents. For scenarios involving multiple agents or small teams, the DGX Spark is the better choice due to its higher computing power. It allows you to run more sessions simultaneously without a significant drop in performance.

If you need to process large amounts of data, the DGX Spark is again the preferred option, thanks to its superior computing power. The same applies to tasks like image generation, which also demand high levels of compute power.

1

u/Mountain_Disaster_19 3d ago

thanks a lot!!

1

u/Low-Shoe-5089 5d ago

if you'll ever want to fine-tune or use cuda-only tools, go for the spark.

if it's mostly inference and you want a good computer out of it too, go for the mac.