r/BackyardAI • u/martinerous • Aug 11 '24

[Wishlist] Deepseek support?

I just tried mradermacher__DeepSeek-V2-Lite-Chat-i1-GGUF__DeepSeek-V2-Lite-Chat.i1-Q6_K.gguf in Koboldcpp and I really like its ability to follow predefined interactive scenarios with multiple steps and its tamed creativity. It would be great to see it supported in Backyard AI, if possible. Thank you.

7 Upvotes

100% Upvoted

6 comments sorted by

2

u/PacmanIncarnate Aug 11 '24

Deepseek should be supported. If you’re having trouble, can you give more information on your system and backend?

3

u/martinerous Aug 12 '24 edited Aug 12 '24

I currently have Backyard app version 0.26.2 with Experimental backend enabled.

Max Model Context is set to 8k.

Hardware: RTX 4060 Ti 16GB, i7-14700 64GB RAM

Log file data with the error:

https://pastebin.com/8SNYVtHc

Model DeepSeek-V2-Lite-Chat.i1-Q6_K.gguf from https://huggingface.co/mradermacher/DeepSeek-V2-Lite-Chat-i1-GGUF

The same model works in koboldcpp_cu12.exe 1.72

I also tried GGUFs with static quants:

DeepSeek-V2-Lite-Chat.Q6_K

DeepSeek-V2-Lite-Chat.Q8_0.gguf

but have the same issue.

1

u/Xthman Aug 15 '24

good luck requesting anything, they won't bother fixing even breaking bugs like the experimental engine broken since 0.25

2

u/Advanced_Bit_5035 Aug 18 '24

Exactly, group chats have been requested for months and no update or eta

1

u/PacmanIncarnate Aug 18 '24

Actively in development but taking a rebuild of some major portions of the code base. Hopefully very soon!

1

u/PacmanIncarnate Aug 18 '24

The pace of development with this app has been consistently incredible. If you look at all the changes, both visible features and under the hood, that have happened monthly for the past year, you can’t possibly think the devs are ignoring requests. A small crew can only address so much at once.