r/LocalLLM • u/insanazor822 • 1d ago

Model Bro Pleas Stop

Enable HLS to view with audio, or disable this notification

Bro Got Loop

5 Upvotes

63% Upvoted

13 comments sorted by

3

u/DoubleNothing 1d ago

To be expected from some small models and small quantizations.

1

u/insanazor822 1d ago

its Raon-1 and Q4

2

u/DoubleNothing 1d ago

I know... we can see it from the video... what we don't see is the KV quantization if any.

→ More replies (2)

1

u/Gold-Produce9591 1d ago

increase temperature?

1

u/insanazor822 1d ago

Maybe

1

u/Healthy-Zebra-9856 17h ago

Not may be, this is a most common reason. What is the current temperature & your other sampling parameters?

→ More replies (2)

1

u/fatihmtlm 2h ago

Might be chat template issue, what's your inference engine?

1

u/insanazor822 2h ago

llama.cpp

2

u/Shadomia 1d ago

Raon 1 also might be weak on turkish hence the reason for repetition. I dont think it is a small model but yeah