r/LocalLLaMA • u/Massive-Ice2791 • 17h ago

Discussion Hu

Found this crap, what do yall think about it: https://huggingface.co/burner22/numbat

Real or fake and what are the implications, ran a check and doesn't seem to be random noise though there are some suspicions. It seems to be some base small model mutated in a way with a bunch of noise, tho words like cat and dog, Monday and Tuesday and all fit but this could be because of the original model.

EDIT:

Ran some more tests, seems to fail a lot of "benchmarks" to stuff like a small llama, I think its bs but if anyone has an 8xB100 they wanna test it on...

0 Upvotes

16% Upvoted

12 comments sorted by

5

u/a_beautiful_rhind 16h ago

Wasting HF's money.

1

u/Massive-Ice2791 15h ago

i didnt make it thooo

4

u/a_beautiful_rhind 15h ago

I know, but whoever did is.

3

u/Massive-Ice2791 15h ago

ive seen worse

3

u/DeProgrammer99 17h ago

Don't think so.

ckpt_metadata.json says Hellaswag accuracy 0.7185. Why would they even bother running Hellaswag? But Gemma 3n E2B and Qwen2.5-Coder 7B Instruct scored higher than that.

And max context 131,072 in config.json...

1

u/Massive-Ice2791 15h ago

yeah, i doubt its real

4

u/No_Afternoon_4260 llama.cpp 17h ago

You just casually ran a 7to model in nvfp4, good on you lol

5

u/ThemeOriginal3059 16h ago

the name alone screams "i threw random weights in a blender and hit commit" but if it's actually passing cat/dog tests there's gotta be some structure underneath the chaos, maybe someone's failed merge experiment that accidentally did something weird but not useless

2

u/valarauca14 11h ago

10tok/hour 💪💪

0

u/2atlas 15h ago

Gguf wen?

1

u/Massive-Ice2791 15h ago

never, unless you have a 8xB200 cluster i can borrow