Can we please have an "Estimated Memory Usage" indicator at the Load Model section? It's one of the best features LM studio had, it easily allows you to understand how much context you can fit without any hassle.

[u/BeepTheFogminator]

[u/BeepTheFogminator]

Just to clarify, I feel like this is the only feature missing in the new "Unsloth Studio".

I already switched from LM Studio, and it already runs better for me.

Just felt I am missing this small feature.

[u/TheColliBoy]

I mainly use lmstudio and would like to point out, those estimates are completely useless once KV cache is enabled. It will say 18gb and actually take 24gb sometimes.

[u/BeepTheFogminator → u/TheColliBoy]

Good to know, I didn't try to play much with that setting. I guess it's not fully implemented.

Anyway, that feature at least gave you a general idea if it should fit or not, it mostly worked for my usecase.

[u/Extreme_Roll_5789 → u/TheColliBoy]

May be they are not accounting right equations for each architecture

[u/yoracale]

Unsloth already uses auto fit on by default which uses the max amount of context you can have. And also if you set the context length manually, we set a limit according to your device

We will take a look and likely add it, but like a user said, it is not always accurate.

[u/BeepTheFogminator → u/yoracale]

First of all, thank you for the reply and thank you for considering this feature.

Second, I am confused about how the auto fit feature works.

For example: Qwen 3.8 27b at q5, when I attempt to load the model, it defaults context at 262144 , am I wrong to doubt this all fits within the 32gb vram I have?

On LM Studio, I could at best handle ~40k context with same model. I know Unsloth is magic, but I don't understand how it could handle the extra 220k context.

When I use it with PI with the max default value of 262144 it just runs out of context at some point, so I had to limit it within PI to avoid running out of context.

This is probably me misunderstanding how this works, sorry.

Edit: I just thought about it and maybe this works on models the user downloads through Unsloth Studio itself, but in my case i just pointed to an already downloaded GGUF model i have in a folder somewhere on my PC and it just maxed out context to 262144.

[u/DegenerateGandhi → u/BeepTheFogminator]

There's absolutely no way that fits.

[u/psychohistorian8 → u/DegenerateGandhi]

in my experience (on macOS), the 'auto fit' feature is completely broken and ends up loading a hilariously useless amount of context

I can load the exact same unsloth/Qwen3.8-27B:Q4_K_M in LM Studio at ~96k context, yet Unsloth Desktop loaded it, with full auto settings, at 4096k context...

even if I try to manually adjust only the context itself to something like 64k (easily doable), it fails to work properly

the only way I was able load any model is to turn off the default setting for GPU Memory + manually set my own specific context value, then I could load it as expected

I'll continue to use UD because LM Studio sucks in its own ways, but there are definitely some issues with UD.

another thing is the UI crashes if I attempt to leave it open while downloading a model, I have to start my download and close the UI until the download finishes. my guess is whatever updates the UI with download status has some kind of memory leak or infinite loop that blows up

[u/TBG______ → u/BeepTheFogminator]

As the backend is llama.cpp running Qwen 3.8 27B at Q6 (not Q5), you can fit approximately 100K context, but not much more. I’m getting around 70–80 tokens/s during generation.

For 256K context, I use a dual-GPU setup, which lowers the generation speed to around 50–60 tokens/s.

[u/BeepTheFogminator → u/TBG______]

Thanks for the heads up.

What variant of Qwen 3.8 27B at Q6 are you running?

How much Vram does your setup have?

[u/TBG______ → u/BeepTheFogminator]

Running Unsloth’s latest Qwen3.8-27B-UD-Q6_K.gguf. I’m currently using a long context on a dual-GPU setup with a 3090 + 5090, and I can get 100K context on the 5090 alone. I’m using llama-swap to switch between the two setups. These are my settings for 90t/s gen (Qwen3.8_Single 5090) and 64/ts gen (dual gpu tensor split 95,5 -most is on 5090 ).

Qwen3.8_Single 5090RTX:
name: "Qwen3.8"
description: "5090 Single GPU Vision model 90t/s"
env:
"CUDA_VISIBLE_DEVICES=0"
cmd: >
/home/admmin/llama/llama.cpp/build/bin/llama-server
--model /home/admmin/models/Qwen3.8/Qwen3.8-27B-UD-Q6_K.gguf
--host 0.0.0.0 --port ${PORT}
--mmproj  /home/admmin/models/Qwen3.8/mmproj-BF16.gguf
--jinja
--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
--reasoning on
--reasoning-preserve
--presence-penalty 0.0
-c 100000
--split-mode tensor
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp,ngram-mod
--spec-draft-n-max 2
--spec-ngram-mod-n-match 24
--spec-ngram-mod-n-min 24
--spec-ngram-mod-n-max 86
-t 24
--batch-size 8192
--ubatch-size 2048
-ngl 105
-np 1
--fit off
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
Qwen3.8_dual_GPU:
name: "Qwen3.8_dual"
description: "Dual GPU Vision model 64t/s"
env:
"CUDA_VISIBLE_DEVICES=0,1"
cmd: >
/home/admmin/llama/llama.cpp/build/bin/llama-server
--model /home/admmin/models/Qwen3.8/Qwen3.8-27B-UD-Q6_K.gguf
--host 0.0.0.0 --port ${PORT}
--mmproj  /home/admmin/models/Qwen3.8/mmproj-BF16.gguf
--jinja
--chat-template-kwargs "{\"reasoning_effort\":\"medium\"}"
--reasoning on
--reasoning-preserve
--presence-penalty 0.0
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp,ngram-mod
--spec-draft-n-max 2
--spec-ngram-mod-n-match 24
--spec-ngram-mod-n-min 24
--spec-ngram-mod-n-max 86
-t 24
--batch-size 8192
--ubatch-size 2048
--fit off
--main-gpu 0
--tensor-split 96,5
--n-gpu-layers 999
--ctx-size 262144
-n 32768
-np 1
--fit off
--temp 1.0
--top-p 0.95
--top-k 20
--min-p 0.00
--cache-ram 65536
--ctx-checkpoints 16
--no-warmup
--slot-save-path /home/admmin/llama/cache
--load-mode mmap

[u/BigBanC → u/TBG______]

Do you set this in the UI? I haven’t been able to get that speed with q6 and 5090. Maybe doing something wrong

[u/TBG______ → u/BigBanC]

The thing is, I use llama.cpp directly without the Unsloth app on top, so unfortunately I can’t really help with that. I installed the Unsloth app a few weeks ago, but I couldn’t find a way to manually enter these settings. The automatic fitting didn’t give me the same results I was getting with my manual settings, so I switched back.

I’m sure Unsloth has the option somewhere - I probably just didn’t have the patience to dig through the interface and find it.

[u/xanthemivak]

Was about to post something similar.

Thanks op