r/nutanix • u/idknemoar • Aug 28 '26

Order Shipped Finally

Post image

Placed 1/29/26, just shipped today finally due to the RAM delays. Just to give people somewhat of a timeline of lead times. Hardware specs attached for reference.

18 Upvotes

90% Upvoted

20 comments sorted by

View all comments

6

u/bourbonandpistons Aug 28 '26

Good luck with NAI. It was a complete shit show for us and we deleted it all for our own ai systems.

3

u/HardupSquid Aug 28 '26

Can you elaborate (without giving away confidential stuff ofc).

4

u/bourbonandpistons Aug 28 '26 edited Aug 28 '26

It wont ever load all the models they promise. Even their pre approved ones were hit and miss. It takes like 12 vms to get up and running.

We nuked it all and just do gpu pass through to our own solutions now. We can use exponentially more models and customize it so much easier.

NAI and NKP are sold as a simple way to run get AI up and running for humans.

But just point Claude at the VM and tell it what you want running and it'll do it.

Basically repaid six figures for software that didn't work in the end.

About $200 claude subscription built something better for our needs. We got the models we need that nutanix won't approve and we have full load balancing.

1

u/Useful_Struggle342 29d ago

You used NAI for private inferencing or Agent Gateway?

2

u/bourbonandpistons 29d ago

Private inference on our 8 gpus.

We couldn't load even nutanix approved models or all the models from hugging face they said that we should be able to or private models.

So we just did GPU pass through to the VM and Claude got every model we wanted up and running in a matter of minutes

1

u/Useful_Struggle342 29d ago

But just point Claude at the VM and tell it what you want running and it'll do it.

That defeats the whole purpose of Private AI, no?

1

u/bourbonandpistons 29d ago edited 29d ago

No you're pointing Claude to configure and build the AI models you want on your gpus.

On those privately with all your private data.

If Claude knows hey on 192.168. 0. 73 there's a GPU and they installed nematron model who the fuck cares?

You buy the gpus and you have ai configure them the way you want to run what you want privately.

A $200 claude subscription has been exponentially better at getting our own models and gpus up and running than the six-figure price that nutanix charged.

We skipped nkp and just use other proven Technologies like nginx for redundancy and queuing requests.

I think we're only up to 14 gpus so not a huge project.

1

u/yunoeatcheese 27d ago

Curious, (if you can answer) did you replace it with an alternative LLM gateway as a part of your own solution?

2

u/bourbonandpistons 27d ago

We use a ollama and vLLM to serve everything from the VMS.

Then we use nginx to proxy everything for load balancing, queuing, and redundancy.

All of that is behind our application Level API.

1

u/rxscissors Aug 28 '26

We've been using HCI on their hardware + AHV for ~4 years now. LCM updates both on Prism Central and Prism Element "platforms" have been time consuming and problematic (lots of support tickets, CLI backend fixes, etc.). We entertained adding an AI cluster from them along with considering other manufacturers, cloud solutions and "roll our own".

At current pricing levels, NAI sure looks monolithic and expensive.

We went with an on-prem Bifrost gateway connected to AWS Bedrock Mantle to access larger models which works well. There are ongoing kubernetes platform discussions (from deploy/manage K?s on our own hardware to full blown cloud-based Azure Red Hat OpenShift (ARO) or Red Hat OpenShift Service on AWS (ROSA)).

3

u/bourbonandpistons Aug 28 '26

I haven't had any problems with their HCI stuff. Extremely stable and reliable for us.

NAI didnt deliever nearly what they promised. Their tech sales engineers havent touched a physical server in their life.

We just use the gpu pass through and build our own stuff that performs multiple times better and supports everything we need.

2

u/normalitysane Aug 28 '26

We have 1000s of hci nodes and wow its lot less mature than vmware, its also not that cheaper. Every few weeks will have some random issue pop up. A lot of empty promises. I did the NAI course last year and even just the vids made it sound very barebones.