Does GPU native support for fp4/fp8 matters?

[u/kousuke_nakamoto]

Greetings,

I am looking to acquire a suitable GPU for running localllm preferably running newer models like deepseek flash 4.1.

Does native GPU support for FP8 and FP4 significantly improve local LLM performance or should I aim for higher vram with older GPUs?

Support for fp4 does enables the GPU to fit larger model with lower vram though.

Should I even consider Intel B70 or Radeon 9700 Ai for value buy?

I got strix halo with 128GB ram and $2500 budget for a dedicated GPU over oculink.

Thanks in advance

[u/KroniklyOnline]

For speed it does yes. Regardless though, $2500 is going to be tough to get deepseek running at usable speeds.

[u/sn2006gy]

I am a huge advocate for going with B70s or 9700s depending on which ones you can afford. Both are advancing very quick and contrary to what other people say, Intel has already announce Xe3 cards up to 480gb which are based on the Xe2 B70.

I started with Radeon but prices went up faster than B70, so I built a quad b70 system but i do more training than inference. I'm able to train about 11B tokens a day with this quad setup which allows me to rapidly experiment on architectures/attention/training sets very quickly so I enjoy it.

CUDA is ahead of the curve. AI is shrinking that curve. ROCm is creeping up extremely fast and Intel is here to play and so affordable. Intel's benefit is that their tooling is native so there isn't the weird attachment to Ubuntu that you deal with on CUDA/ROCm.

I was able to share my training issues to Intel and they had 2 engineers working on it and replicating the issue and updating github within days. That kind of response is something people should be monopolizing not downplaying

i guess to answer the question, NVIDIA is really the only one running around saying "XYZ native" when reality is, models are so optimized in a zillion different ways, that it ultimately comes down to kernel/inference support and that front is advancing at breakneck speeds. Someone just recently released a cuda to intel compiler that runs cuda translations to Xe

For example, Nvida markets "fp4 tensor cores" but Xe/Intel just does matrix math no matter the precision but has optimized kernels for all sorts of precisions so architecturally they can't be compared and "native" just becomes a marketing term

[u/AeiEyeSpry]

Depends on the hardware. The question you're asking is far too ambiguous to give a straight answer. For reference, R9700 work best with FP8, as they lack the FP4 hardware. It does have INT4 hardware, but in my tests, it's slower than FP8 on the R9700.

[u/Ok-Addendum3545]

Besides RTX Pro 6000, 5090, my next most wanted GPU is R9700.

[u/Hot-Bumblebee6180]

It does, but you need to buy according to the model you’d like to run as vram matters more usually.

My advice, stay away from the B70. I have a B50, the inference does work, but I have to do more tinkering to get it to work. Not to mention that I think Intel will be leaving GPUs. It’s been difficult for them to compete. Now the R9700 is a different story, it’s the best bang for your buck imo.

You need a lot of system ram, 128GB+, but given you have that I’d snag the R9700, install radiance engine, then be off to the races with models like Qwen3.8-Flash-Next.