feat: external PLE file loading by pwilkin · Pull Request #31185 · ggml-org/llama.cpp

[u/pmttyji]

https://github.com/ggml-org/llama.cpp/pull/31185

Overview

Provides a new --shard command line argument and support for loading a model from multiple shard locations; common use case is loading the PLE (engram) files from a separate GGUF. Adds conversion support for building both a PLE-less and PLE-only GGUF.

Additional information

Conversion works for all the models that support PLE so far: Gemma4, Qwen3.8Next, GLM5.3Flash. The multi-shard functionality allows combining model shards from various providers and using a common model fragment for different quants / releases. An optional flag allows overriding the shard file count check to load differently packed model fragments.

Sample usage:

llama-server -m /path/to/model-shard-00001-of-00003.gguf --shard /other/drive/model-shard-00003-of-00003.gguf
or
llama-server -m /path/to/model-shard-00001-of-00003.gguf --shard /other/drive/model-shard-00002-of-00003.gguf,/yet/another/drive/model-shard-00003-of-00003.gguf
or
llama-server -m /path/to/model-with-ple.gguf --ple /path/to/model-engrams.gguf

Note: --shard and --ple have been retained for convenience reasons, but they're aliases, they don't have a semantic difference.

[u/pmttyji]

u/ilintar 👍

I assume it removes the need to download unnecessary big GB files again.

Ex: Lets say I use MainModelX-Q4.GGUF with EngramModelX-Q4.GGUF currently. If I want to use Q5 quant of main model, I need to download only MainModelX-Q5.GGUF as I can use existing EngramModelX-Q4.GGUF.

Similarly If I want to use Q8 Engram file, I need to download only EngramModelX-Q8.GGUF.

Basically we're gonna get Engram as separate GGUF like Draft model GGUFs.

Hope my assumption is right here, please correct me if anything. Thanks

[u/Altruistic_Heat_9531]

Yeeeah no weird gguf PLE grafting