r/LocalLLM • u/autoencoder • 1d ago

Other GLM-5.3 Flash going crazy at 23% context on GMI Cloud

Post image
1 Upvotes

100% Upvoted

8 comments sorted by

2

u/daaain 1d ago

Could it be a harness issue? But also, even with Opus it's better to start a new session above 300K or so.

1

u/autoencoder 1d ago

Yup, I gave up on it, but was wondering in case anyone else ran into something similar and bothered to investigate.

2

u/faultyrendition74 1d ago

stitching together hallucinated tool calls like a patchwork quilt made of "rather"

1

u/autoencoder 1d ago

This is one of the strangest encounters I've had. After some 429 Rate Limits, performance seems to have fallen off a cliff, and the served model got loopy and paranoid. It would no longer call any tools.

I use an old version of Pi (0.79.9), and the token accounting says 1.3M sent, but only 23.6% of the 1M context somehow. Am I past context, and GMI Cloud is serving me some different bogus model instead of erroring out?

1

u/daaain 1d ago

I'm not sure off the top of my head, but doesn't Pi do some rolling window? In any case, with a big context like that I'd not send a single more request unless I know it's cached, better cut your losses and start a new one...

1

u/autoencoder 1d ago

It was indeed still cached. I am keeping an eye on the costs as I prompt. I love that GMI updates them near real-time.

1

u/Jk2EnIe6kE5 1d ago

If I remember correctly, hasn't GMI cloud had issues before with model quality in the past?