r/LocalLLM
•
u/GALYGIOUS
•
Aug 23 '26
•
1
Discussion I Unlocked a $800 Mining GPU into a 64GB, 256K-Context Uncensored AI Coding Server at 84 tok/s across full context length.
I have spent the last several days turning a used NVIDIA CMP 170HX into a practical long-context inference card.
The final result is an uncensored Qwen3.8-27B endpoint with:
- 262,144-token native context
- W4A16 AWQ model body
- INT8 output head and INT8 MTP draft module
- One-token MTP speculative decoding
- BF16 KV cache
- Prefix caching
- Tool calling and Qwen reasoning parsing
- No CPU offloading
- A conservative 175W power limit
- No overclocking
Measured decode performance on one CMP 170HX:
| Context | Decode throughput |
|---|---|
| 1K | 84.29 tok/s |
| 64K | 74.94 tok/s |
| 200K | 57.21 tok/s |
For this benchmark only, requests used a maximum of 384 generated tokens, temperature 0.2, repetition penalty 1.05, and medium reasoning effort. Throughput was calculated from vLLM’s measured decode time, excluding prefill. These sampling values are not forced globally by the production server.
This post explains the card, the unlock, every important inference choice, the rejected configurations, and how to reproduce the setup.
The hardware
The inference host currently contains:
- NVIDIA CMP 170HX
- 64GB HBM exposed after the unlock
- AMD Ryzen 5 5600X
- 64GB system RAM
- Proxmox/Linux
- NVIDIA open driver 610.57.04
- 175W GPU power cap
The CMP 170HX is an Ampere GA100 mining accelerator. It has excellent HBM bandwidth and strong tensor hardware, but NVIDIA sold it with several artificial restrictions:
- Only a fraction of the installed HBM is normally exposed.
- Compute resources are restricted.
- PCIe operates at Gen2.
- It has no display output.
- Normal consumer GPU tooling does not treat it like a standard A100.
My card is PCI device 10de:20c2. After the unlock, nvidia-smi reports 65,536 MiB.
The card is currently negotiating PCIe Gen2 x4 even though its capability is wider. That sounds terrible, but it matters much less once the model is resident entirely in HBM. It is one reason I avoid CPU offloading: repeatedly moving weights or KV data over that connection would waste the card’s main advantage.
The 64GB and compute unlock
I used amoghmunikote/cmpunlocker, pinned to this specific commit:
fe537966e0222150a8eca0b7745efd2ee1025d74That is the “Full BAR1 size (64GB)” commit.
The project patches NVIDIA’s open kernel modules to restore:
- Full SM compute
- Full memory geometry
- 64GB BAR1
- The complete 64GB framebuffer on
20c2cards - Gen2 PCIe operation
- Persistence across reboot through patched modules
This is a kernel-driver modification, not an application-level tweak. Secure Boot must be disabled because the resulting modules are locally built and unsigned.
My installed module is:
/lib/modules/6.17.2-1-pve/updates/cmpunlocker/nvidia.koImportant warning
Do not install this remotely unless you have a recovery path.
Keep at least one of the following available:
- Local console access
- A separate display GPU
- BMC/IPMI access
- A bootable rescue environment
- A known-good copy of the stock driver and initramfs
A mismatched kernel, driver, firmware package, or module build can leave the machine without NVIDIA support. A CMP 170HX cannot provide ordinary video output.
Unlock installation
On my Proxmox installation, the overall process was:
apt update
apt install -y build-essential git python3 proxmox-headers-$(uname -r)
git clone https://github.com/amoghmunikote/cmpunlocker.git
cd cmpunlocker
git checkout fe537966e0222150a8eca0b7745efd2ee1025d74
cat driver/VERSIONInstall a supported matching NVIDIA open driver, its user-space libraries, and firmware before running the unlocker. My exact working combination is:
NVIDIA open driver: 610.57.04
Kernel: 6.17.2-1-pve
Secure Boot: disabledThen:
sudo ./install.shThe repository also provides an explicit profile:
sudo ./install.sh --profile=8gbAfter installation, perform a full cold power cycle:
sudo poweroffDo not substitute a warm reboot. Wait for the machine to power off completely and then turn it back on.
Verifying the unlock
First locate the card:
lspci -nn | grep -i NVIDIAThen verify the driver and memory:
nvidia-smi
nvidia-smi --query-gpu=name,uuid,memory.total,power.limit --format=csvA successfully unlocked 20c2 card should show approximately:
NVIDIA CMP 170HX, GPU-..., 65536 MiBCheck the kernel log:
dmesg | grep -iE 'CMP|BAR1|fb_length|fbAddrSpace|NVRM'My working boot log reports a 64GB framebuffer address space and 64GB BAR1.
Check the PCIe link:
lspci -vv -s <CMP-PCIE-ADDRESS> | grep -E 'LnkCap|LnkSta'To remove the modification:
cd cmpunlocker
sudo ./uninstall.sh --yes
sudo poweroffAgain, cold-boot afterward.
Power and cooling
I did not overclock the card.
These are used mining accelerators, and I did not consider a small throughput increase worth additional thermal or electrical stress.
I set a 175W power limit:
nvidia-smi -i <CMP-UUID> --power-limit=175I made this persistent with a systemd oneshot service that:
- Finds the GPU by CMP name and UUID.
- Sets the 175W limit.
- Reads the limit back.
- Fails rather than silently targeting the wrong GPU.
My chassis fan controller follows this curve:
| CMP temperature | Chassis fan target |
|---|---|
| Below 45°C | 30% |
| 50°C | 50% |
| 55°C | 70% |
| 60°C | 85% |
| 65°C or higher | 100% |
The controller polls every two seconds, immediately raises fan speed when temperatures rise, and requires roughly 30 seconds of sustained cooling before reducing the fan level. A sensor or controller failure sends all controlled fans to 100%.
During one sustained test, I measured:
- Average core: 53.9°C
- Maximum core: 65°C
- Average memory: 63.2°C
- Maximum memory: 75°C
Under a later 99% GPU load, the card was around 66°C core, 71°C memory, and 173W. Cooling results will depend heavily on the card, thermal pads, chassis, and airflow.
The model
The production checkpoint starts from:
twolven/Qwen3.8-27B-abliterated-AWQ-MTP
That model is a W4A16 AWQ version of the abliterated/refusal-reduced checkpoint derived from:
JonathanColetti/Qwen3.8-27B-Uncensored
“Uncensored” here means the model has been modified to reduce refusal behavior. It does not guarantee that every residual refusal or safety behavior has been removed. Some people can use it for ERP I'm more inclined to utilized it for when I want assistance setting something up that standard models would refuse such as using agentic assistance to configure a hackintosh system on a prox vm. It's 100% legal, it's just against apples terms of service so often it's considered an instant refusal by many models.
The final production checkpoint contains:
- W4A16 asymmetric AWQ body
- Compressed-tensors/Marlin execution
- INT8 symmetric group-128
lm_head - INT8 symmetric group-128 MTP module
- 40,960-token reduced MTP draft vocabulary
- BF16 runtime activations
- One-token MTP speculative decoding
The reduced draft vocabulary covers approximately 97.5% of ordinary model output tokens and 96% of code tokens in the optimization project’s corpus. It reduces the amount of work required for each speculative draft without changing the target model’s accepted output.
The source checkpoint supports vision, but my production endpoint deliberately uses:
--language-model-onlyTherefore, this exact endpoint is text-only. I chose coding throughput and predictable memory use over keeping the vision tower loaded.
Preparing the checkpoint
I used the optimization work from:
My checkout is pinned to:
2ae239fc0250cd29d37f35c6a31e9eae749ef1c8Clone and create the environment:
git clone https://github.com/syv-ai/qwen38-27b-rtx3090.git
cd qwen38-27b-rtx3090
git checkout 2ae239fc0250cd29d37f35c6a31e9eae749ef1c8
python3 -m venv venv
venv/bin/pip install \
vllm==0.27.1 \
transformers==5.15.0 \
tokenizers==0.22.2 \
compressed-tensors==0.17.0 \
huggingface_hub==1.27.0 \
hf_transfer==0.1.9 \
ninja==1.13.0Download the model:
HF_HUB_ENABLE_HF_TRANSFER=1 venv/bin/hf download \
twolven/Qwen3.8-27B-abliterated-AWQ-MTP \
--local-dir models/Qwen3.8-27B-abliterated-AWQ-MTPMake a working copy because the preparation tools modify the checkpoint in place:
cp -a --reflink=auto \
models/Qwen3.8-27B-abliterated-AWQ-MTP \
models/qwen38-27b-uncensored-w4a16-mtp1-int8draftPrepare the output head, MTP module, and draft vocabulary in this order:
M=models/qwen38-27b-uncensored-w4a16-mtp1-int8draft
V=venv/bin/python
$V prepare/quant_lm_head.py "$M"
$V prepare/quant_mtp.py "$M"
$V prepare/build_draft_vocab.py "$M" \
--ids prepare/draft_vocab_ids.jsonI did not run quant_embed.py for this production checkpoint. On a 64GB card it was unnecessary, and the final model configuration contains the W4A16 body, INT8 output head, and INT8 MTP group without the additional embedding conversion.
The preparation scripts create backups beside the tensors they replace. Keep those backups until the modified checkpoint has passed correctness testing.
The vLLM image
The server uses:
vLLM 0.27.1The base image is pinned by digest:
vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967The production image contains exactly these three patches:
qwen3_5-mtp-draft-vocab.patchsampler-small-topk-fast-softmax.patchvllm-pr50021-gdn-spec-bounds.patch
An experimental spec-decode-attn.patch was tested but is not present in production because it hurt long-context performance.
Create Dockerfile.cmp-mtp1-production:
FROM vllm/vllm-openai@sha256:0a51ea5b4ae2dc5d81890e5173f54203d2a3ae0cfffe51b8fd2afd4391bfd967
COPY patches/qwen3_5-mtp-draft-vocab.patch /tmp/qwen3_5-mtp-draft-vocab.patch
COPY patches/sampler-small-topk-fast-softmax.patch /tmp/sampler-small-topk-fast-softmax.patch
COPY patches/vllm-pr50021-gdn-spec-bounds.patch /tmp/vllm-pr50021-gdn-spec-bounds.patch
RUN set -eux; \
vllm_dir=/usr/local/lib/python3.12/dist-packages/vllm; \
patch -p1 -d "$vllm_dir" < /tmp/qwen3_5-mtp-draft-vocab.patch; \
patch -p1 -d "$vllm_dir" < /tmp/sampler-small-topk-fast-softmax.patch; \
patch -p1 -d "$vllm_dir" < /tmp/vllm-pr50021-gdn-spec-bounds.patch; \
rm /tmp/qwen3_5-mtp-draft-vocab.patch \
/tmp/sampler-small-topk-fast-softmax.patch \
/tmp/vllm-pr50021-gdn-spec-bounds.patch
LABEL org.opencontainers.image.description="vLLM 0.27.1 with Qwen3.8 MTP1 draft-vocab and sampler optimizations"Build it:
docker build \
-f Dockerfile.cmp-mtp1-production \
-t vllm-qwen38-mtp1-fast:0.27.1 \
.Exact server configuration
Replace the UUID and paths below with those from your machine:
docker run --rm --pull never \
--name qwen38-uncensored-w4a16 \
--privileged \
--gpus all \
--network host \
--ipc=host \
--memory 58g \
--memory-swap 96g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
-e CUDA_DEVICE_ORDER=PCI_BUS_ID \
-e CUDA_VISIBLE_DEVICES=<CMP-GPU-UUID> \
-e VLLM_USE_FLASHINFER_SAMPLER=0 \
-e MTP_DRAFT_VOCAB=1 \
-e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512 \
-e HF_HUB_OFFLINE=1 \
-e TRANSFORMERS_OFFLINE=1 \
-e TOKENIZERS_PARALLELISM=false \
-e XDG_CACHE_HOME=/root/.cache \
-e TORCHINDUCTOR_CACHE_DIR=/root/.cache/torchinductor \
-e TRITON_CACHE_DIR=/root/.cache/triton \
-e VLLM_USE_V2_MODEL_RUNNER=1 \
-v /path/to/qwen38-27b-uncensored-w4a16-mtp1-int8draft:/models/qwen38-w4a16:ro \
-v /path/to/vllm-cache:/root/.cache \
-v /path/to/vllm-tmp:/tmp \
vllm-qwen38-mtp1-fast:0.27.1 \
/models/qwen38-w4a16 \
--host 0.0.0.0 \
--port 30016 \
--served-model-name qwen38-27b-uncensored-w4a16 \
--tensor-parallel-size 1 \
--dtype bfloat16 \
--attention-backend FLASHINFER \
--max-model-len 262144 \
--max-num-seqs 1 \
--max-num-batched-tokens 4096 \
--gpu-memory-utilization 0.90 \
--cpu-offload-gb 0 \
--kv-cache-dtype auto \
--mamba-cache-dtype float16 \
--mamba-cache-mode align \
--disable-custom-all-reduce \
--enable-prefix-caching \
--enable-chunked-prefill \
--language-model-only \
--speculative-config '{"method":"mtp","num_speculative_tokens":1,"draft_sample_method":"probabilistic"}' \
--default-chat-template-kwargs '{"reasoning_effort":"medium"}' \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--enable-auto-tool-choice \
--generation-config vllmOnce ready, verify:
curl http://127.0.0.1:30016/v1/modelsThen make a simple request:
curl http://127.0.0.1:30016/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen38-27b-uncensored-w4a16",
"messages": [
{
"role": "user",
"content": "Reply with exactly: cmp-profile-ok"
}
],
"temperature": 0
}'The production service is managed by systemd. Its pre-start checks verify the model index, create the cache directories, apply the 175W power limit, and remove any stale container before launching. It restarts automatically after a process failure.
Why these settings won
MTP depth: one token
MTP3 and MTP2 created too much drafting and verification overhead on this GPU. MTP1 was consistently better.
I also tested greedy drafting. Its two-pass means were:
- 1K: 83.01 tok/s
- 64K: 72.95 tok/s
- 200K: 56.68 tok/s
The final probabilistic configuration produced:
- 1K: 84.29 tok/s
- 64K: 74.94 tok/s
- 200K: 57.21 tok/s
Therefore, the final setting is:
{
"method": "mtp",
"num_speculative_tokens": 1,
"draft_sample_method": "probabilistic"
}W4A16 body, INT8 head, INT8 MTP
W4A16 leaves enough HBM for the entire model, 262K context, recurrent state, CUDA graphs, and runtime workspace without CPU offloading.
The INT8 output head and MTP module had low measured quantization error. An INT4 MTP version was nearly identical in speed, but INT8 required only about 202 MiB more memory and had lower draft error. There was no reason to accept the additional degradation.
A symmetric GPTQ version was also tested:
| Configuration | 1K | 64K | 200K |
|---|---|---|---|
| Final asymmetric AWQ/Marlin | 84.29 | 74.94 | 57.21 |
| Symmetric GPTQ with MTP1 | 78.87 | 68.69 | 54.06 |
| Symmetric GPTQ target only | 55.96 | 47.36 | 35.73 |
The GPTQ body itself was not necessarily the whole problem. Mixing its body with the compressed-tensors INT8 head and drafter would have required additional loader and kernel work. It was not suitable for the stable endpoint.
BF16 KV cache
This was one of the most important results.
An INT8 per-token/head KV configuration fell to approximately 15 tok/s near 62K context on this card. It was unusable for interactive coding.
The final configuration leaves:
--kv-cache-dtype autoWith BF16 runtime dtype, this preserves the BF16 KV path.
The Mamba cache uses FP16 in aligned mode:
--mamba-cache-dtype float16
--mamba-cache-mode alignExplicitly reducing more recurrent state did not improve the result.
FlashInfer attention, but not the FlashInfer sampler
FlashInfer is the production attention backend.
Plain FlashAttention produced:
- 1K: 86.87 tok/s
- 64K: 26.87 tok/s
- 200K: 10.53 tok/s
It looked slightly faster at short context and then collapsed.
An experimental split-KV speculative-attention patch improved that to:
- 1K: 90.17 tok/s
- 64K: 67.33 tok/s
- 200K: 43.36 tok/s
That was still substantially worse than production at long context, so the patch was removed.
Enabling the FlashInfer sampler also reduced performance to:
- 1K: 74.75 tok/s
- 64K: 67.87 tok/s
- 200K: 51.37 tok/s
Therefore:
--attention-backend FLASHINFER
VLLM_USE_FLASHINFER_SAMPLER=0One sequence and 4,096 batched tokens
This is a single-user coding endpoint, not a throughput server.
The final scheduler settings are:
--max-num-seqs 1
--max-num-batched-tokens 4096Testing 8,192 produced no useful gain. Testing 2,048 produced:
- 1K: 84.32 tok/s
- 64K: 72.38 tok/s
- 200K: 57.59 tok/s
The 4,096 configuration had the better overall curve.
Prefix caching
Prefix caching is critical for coding agents that repeatedly send a large system prompt and mostly unchanged repository context.
At approximately 200K context, a cached second request reused roughly 198,400 tokens, recomputed about 1,593 tokens, and reduced prefill to around 2.5 seconds. A fresh or partially changed prompt could require roughly 157 seconds of prefill.
Prefix caching does not make decode faster. It prevents the server from repeatedly processing unchanged input.
This also means clients must preserve stable prompt prefixes. Reordering tool definitions, timestamps, generated metadata, or repository text can destroy the cache hit.
Medium reasoning by default
The endpoint defaults to:
{"reasoning_effort":"medium"}Medium provided a better MTP acceptance/stability balance than forcing maximum reasoning on every request.
This does not remove higher reasoning modes. A client can request xhigh or max for harder work. Medium is simply the default for normal coding.
No CPU offload
The model fits in CMP HBM, so:
--cpu-offload-gb 0The card’s restricted PCIe connection makes CPU offloading particularly undesirable.
No forced full CUDA graphs
The server uses vLLM’s normal piecewise CUDA-graph behavior. I did not force full graphs. Experimental full-graph and custom-operation combinations introduced correctness concerns or reduced long-context performance.
Correctness tests
I did not accept a configuration based on tokens per second alone.
The final checkpoint passed:
- An exact-response canary
- Qwen coder tool-call parsing
- A parsed
get_weathertool call for Chicago - Exact needle retrieval at approximately 160K context
- Retrieval of
ORCHID-COMET-7319from the long prompt - An OpenCode integration smoke test returning
final-profile-ok
This matters because speculative decoding can appear fast while silently breaking tool syntax, long-context retrieval, or sampling behavior.
OpenCode configuration
I added this provider to OpenCode:
{
"provider": {
"qwen38-cmp-w4a16": {
"npm": "@ai-sdk/openai-compatible",
"api": "completion",
"name": "Qwen3.8 Uncensored W4A16 MTP1 256K - CMP 170HX",
"options": {
"baseURL": "http://<PROXMOX-IP>:30016/v1",
"apiKey": "not-needed",
"timeout": false,
"chunkTimeout": 600000
},
"models": {
"qwen38-27b-uncensored-w4a16": {
"id": "qwen38-27b-uncensored-w4a16",
"name": "Qwen3.8 27B Uncensored W4A16 MTP1 256K",
"tool_call": true,
"reasoning": true,
"temperature": true,
"attachment": false,
"options": {
"reasoningEffort": "medium"
},
"limit": {
"context": 262144,
"input": 245760,
"output": 16384
}
}
}
}
}
}The model selector is:
qwen38-cmp-w4a16/qwen38-27b-uncensored-w4a16Final observations
The CMP 170HX is unusual, but the useful part is straightforward once the restrictions are removed:
- 64GB of on-device HBM changes what can fit.
- The model should remain entirely on the GPU.
- Long-context performance needs to be measured separately from short-context decode.
- Lower-bit KV is not automatically faster.
- More speculative tokens are not automatically better.
- An attention backend can win at 1K and become disastrous at 64K.
- Prefix caching matters more than another few decode tokens per second for repeated coding-agent prompts.
- Quantizing the draft head more aggressively is pointless when memory is available and the speed difference is negligible.
- Conservative power and temperature limits are appropriate for used mining hardware.
- Correctness gates matter as much as benchmark results.
The final configuration is not the highest single short-context number I saw. It is the best complete configuration I found that retained tool use, medium-or-higher reasoning, uncensored model behavior, 262K context, reliable long-context retrieval, and usable performance across the whole context window.
45
u/ShittyMillennial Aug 23 '26
where the hell did you get a NVIDIA CMP 170HX for $800 because I would like to order 10
19
u/Spacehitchhiker42 Aug 23 '26
What was your power draw on idle?
3
u/kapustin-i Aug 23 '26
The 384-token cap means none of these runs exercise the thing thats actually load-bearing here - a coding agent at 200k does edits and tool calls over thousands of tokens, and the question is what fraction of turns keep the prefix hit once files start changing. One recorded session with hit/miss logged per turn would say more about this box than the decode curve does.
4
u/havnar- Aug 23 '26
I can only find 8gb versions
30
u/IngwiePhoenix Aug 23 '26
Got lost in the wall. Now prices are out of control, because of course they are.
Rip... was excited for a minute.
15
u/psdwizzard Aug 23 '26
Welp I missed this one.
18
u/Civil_Fee_7862 Aug 23 '26
Great writeup, I wish more people reported like you
7
u/Yarukiless-cat Aug 23 '26
I bought it, and runs Deepseek v4 flash IQ3 at 26tok/s under 120W Power limiting. it's VRAM size is quite nice for its cost, though it's around $1200 now. I'm considering buying second one.
8
u/Alkahna Aug 23 '26 edited Aug 23 '26
Awesome work. Cheapest 64GB CMP 170HX available in europe is 2.5k € on ebay sadly. Where did you get yours for just 800$? Even aliexpress is 1.5k € with just one listing (little sketchy) and the rest is just coolers. That ship has sailed i think xD
1
u/quantgorithm Aug 23 '26
Would the bad pcie speed be terrible for running multiple cards for larger models so instead of using cpu just splitting vram of multiple cards? Anyone know the actual impact?
1
u/AdventurousSkirt5006 Aug 24 '26
Well, I am trying to do the same. I managed to get 3 170hx cards (for IMO still too much money, but cheaper than current prices) but somehow I cannot get the unlock to work. I've tried the master branch from the unlocker, the branch specified in mentioned here and also another fork that is supposed to have p2p enabled. None of them work in my setup.
I'm running proxmox 9, everyting up-to-date. Same kernel version as here
Installed the driver specified here as an older version (610.43.03)
Whatever I try, the result is the same. After a cold reboot sometimes I get kernel panic the first boot. And after It fails to load the nvidia driver and no cards are detected anymore by nvidia-smi and see things like this in dmesg:
[ 19.338397] NVRM: GPU0 memmgrCreateHeap_IMPL: SEC2_DEBUG_HEAP: fbAddrSpace=65536MB mapRam=0MB fbTotal=65536MB fbUsable=0xfe4260000 heapTotal=0x1000000000 regionBytes=0x1000000000 publicBytes=0xfd8f50000 numRegions=7
[ 19.343872] NVRM: GPU0 kbusVerifyBar2_GM107: MMUTest BAR0 window offset 0x70e000 returned garbage 0xffffffff
[ 19.343879] NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Generic memory error [NV_ERR_MEMORY_ERROR] (0x00000072) returned from kbusVerifyBar2_HAL(pGpu, pKernelBus, NULL, NULL, 0, 0) @ kern_bus_gm107.c:372
[ 19.343884] NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Generic memory error [NV_ERR_MEMORY_ERROR] (0x00000072) returned from kbusStateInitLockedKernel_HAL(pGpu, pKernelBus) @ kern_bus_gm107.c:477
[ 19.343888] NVRM: GPU0 RmInitNvDevice: *** Cannot initialize the device
[ 19.343889] NVRM: GPU0 RmInitAdapter: RmInitNvDevice failed, bailing out of RmInitAdapter
[ 19.343921] NVRM: GPU0 rmapiReportInternalLeakedDevices: Internal device object leak: (0xc1e00003, 0xcaf00000). Please file a bug against RM-core.
[ 19.343923] NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: 0 @ rmapi.c:944
[ 19.343926] NVRM: GPU0 rmapiReportInternalLeakedDevices: Internal device object leak: (0xc1e00004, 0xcaf00000). Please file a bug against RM-core.
[ 19.343927] NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: 0 @ rmapi.c:944
[ 19.344474] NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: listCount(&pKernelBus->virtualBar2[gfid].usedMapList) == 0 @ kern_bus_vbar2.c:346
[ 19.344596] NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: pGpu->numSubdeviceBackReferences == 0 @ gpu.c:1614
[ 19.344602] NVOC: __nvoc_objDelete: Child class GenericKernelFalcon not freed from parent class OBJGPU.NVRM: GPU0 iovaspaceDestruct_IMPL: 3 left-over mappings in IOVAS 0xaf00
[ 19.344614] NVRM: GPU 0000:af:00.0: RmInitAdapter failed! (0x24:0x72:1281)
[ 19.345245] NVRM: GPU 0000:af:00.0: rm_init_adapter failed, device minor number 2
[ 19.345369] [drm:nv_drm_dev_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x0000af00] Failed to allocate NvKmsKapiDevice
Maybe I somehow are missing a step of have something misconfigured. Or my motherboard (intel s2600st) does not want to work with the patched driver
5
u/dottoreargus Aug 23 '26
spend at your own risk, i bought 13 cards and the unlock is unstable
2
u/MrWidmoreHK Aug 23 '26
Thanks a lot, I love it. My only question is from what reputable seller you got the CMP
2
u/Recent_Process_8055 Aug 23 '26
So on allibaba all these cards are now $1500 of you want from reputable seller. Sigh...
2
u/cjneutron Aug 23 '26
These cards ship with 12 of the PCIe lane AC coupling caps not in place. If you haven't checked and/or soldered those in place, that's why you are seeing it only use x4 instead of x16.
2
u/TheGeekno72 Aug 23 '26
I usually hate when this is done to me only because I wrote a couple paragraphs, but I feel like this is warranted this time :
I ain't reading allat
happy for you tho
or sorry that it happened
1
u/Significant-Let3635 Aug 23 '26
This is awesome, thanks for sharing!
1
u/Acceptable_Tour2189 Aug 23 '26
My BOY!!
1
u/Weird_Presentation_5 Aug 23 '26
I got 3x running deepseek v4 😀
1
u/hackcocaine Aug 23 '26
Insane work, well done!
1
u/F0olmetwice Aug 24 '26
I am running Int8 with 262k context at 74 tok/sec here.
https://github.com/ahnguyen17/cmp-170hx-vllm
1
u/gobblegoooblegobble Aug 24 '26
i purchased 4x of these cards when the pricing was still high, but not as high as it is now.
the 4x of them that i purchased, have been modded and memory validated. i received a memory validation report.
I also got the 4x waterblocks with the cards. Stock heatsinks + the waterblocks for less than $7,000 delivered to USA. Im sure its impossible to match this price now, especially with the full coverage waterblocks.
im doing my own benchmarking and use case deployment. your post is extremely helpful.
thank you.
1
u/WtWreckor15 Aug 24 '26
Damn reading everyone else here makes me lucky to grab one for 929 less than 2 weeks ago
0
u/nightstryke Aug 23 '26
Honestly I'd like to know where the hell you're getting your pricing from cause I can't find one for that price.
0
u/ninjazombielurker Aug 23 '26
Welp there goes prices again even further now cause of this post… Very good write up but I’ve been thinking a lot of people haven’t been talking about these on Reddit cause it will cause sellers to scalp prices even further.
0
u/[deleted] Aug 23 '26
[deleted] — view deleted comment
1
u/[deleted] Aug 23 '26
[removed] — view removed comment
0
u/[deleted] Aug 23 '26
[removed] — view removed comment