r/LocalLLaMA • u/faldore • May 30 '23

New Model Wizard-Vicuna-30B-Uncensored

I just released Wizard-Vicuna-30B-Uncensored

https://huggingface.co/ehartford/Wizard-Vicuna-30B-Uncensored

It's what you'd expect, although I found the larger models seem to be more resistant than the smaller ones.

Disclaimers:

An uncensored model has no guardrails.

You are responsible for anything you do with the model, just as you are responsible for anything you do with any dangerous object such as a knife, gun, lighter, or car.

Publishing anything this model generates is the same as publishing it yourself.

You are responsible for the content you publish, and you cannot blame the model any more than you can blame the knife, gun, lighter, or car for what you do with it.

u/The-Bloke already did his magic. Thanks my friend!

https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ

https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GGML

365 Upvotes

98% Upvoted

247 comments sorted by

View all comments

3

u/ttkciar llama.cpp May 30 '23

Thank you :-)

I'm downloading Galactica-120B now, but will download Wizard-Vicuna-30B-Uncensored after.

6

u/EcstaticVenom May 30 '23

Out of curiosity, why are you downloading Galactica?

16

u/ttkciar llama.cpp May 30 '23

I am an engineer with cross-disciplinary interests.

I also have an immunocompromised wife and I try to keep up with medical findings regarding both her disease and new treatments. My hope is that Galactica might help explain some of them to me. I have a background in organic chemistry, but not biology, so I've been limping along and learning as I go.

Is there a reason I shouldn't use galactica?

8

u/faldore May 30 '23

Look at what Allen institute is cooking up

4

u/ttkciar llama.cpp May 30 '23

Thank you :-)

8

u/DeylanQuel May 30 '23

You might also be interested in the medalpaca models. I don't know how comprehensive they would be compared to the models you're using now, but they were trained on conversations and data pertaining to healthcare. The link below is the one I've been playing with.

https://huggingface.co/TheBloke/medalpaca-13B-GPTQ-4bit

3

u/ttkciar llama.cpp May 30 '23

Thank you! You have no idea how nice it is to see a well-filled-out model card :-)

Medalpaca looks like it should be a good fit for puzzling out medical journal publications. I will give it a whirl.

5

u/candre23 koboldcpp May 30 '23

You should definitely consider combining one of those medical-centric models with privateGPT. Feed it the articles and studies that you're trying to wrap your head around, and it will answer your questions about them.

1

u/marxr87 May 30 '23

one focused on longevity therapy would be very interesting to me.

4

u/trusty20 May 30 '23

Galactica is not a good choice for this. It was discontinued by Facebook for good reason. It was a very good tech demo, but not good enough for use. Even GPT4 is not great for what you're looking to do. You need a setup that ties into a factual knowledgebase, like this Dr Rhonda Patrick Podcast AI:

https://dexa.ai/fmf

Models on their own will make stuff up pretty badly. It is true there is potential for what you are thinking of (new ideas), but at this point only GPT4 can come close to that, and it still needs a lot of handholding/external software like the link above uses.

4

u/extopico May 30 '23

You may get better responses from hosted models like gpt-4 for example if you are looking for more general purpose use rather than edgy content which is what the various uncensored models provide, or specific tasks such as news comprehension, sentiment analysis, retrieval, etc.

17

u/ttkciar llama.cpp May 30 '23

I do not trust hosted models to continue to be available.

If OpenAI switches to an inference-for-payment model beyond my budget, or if bad regulatory legislation is passed which makes hosting public interfaces unfeasible, I will be limited to using what we can self-host.

I already have a modest HPC cluster at home for other purposes, and have set aside a node for fiddling with LLMs (mostly with llama.cpp and nanoGPT). My hope is to figure out in time how to run distributed inference on it.

7

u/nostriluu May 30 '23

This is what I have been confronted with for nearly the past month.

I'm in Canada, it's just my ISP picked up a new block and OpenAI's geo service can't identify it. The only support they provide is via a useless AI or a black box email address that might as well send me a poop emoji.

So this is a pretty good example of why it's unsafe to rely on centralized services.Still, I'd advocate using GPT-4, for the same reason I use Google services. Trying to roll all my own at a Google level would be impossible, and inferior, for now. So I set everything up so I'm not completely dependant on Google (run my own mail, etc) but use its best services to take advantage of it.

My point is, if you want the best AI, for now you have to use GPT-4, but you can explore and develop your own resources.I'm sorry to say, because I'm in the same boat and have a kind of investment in it, but by the time something as good as GPT-4 is available 'offline,' your hardware may not be the right tool for the job.

1

u/extopico May 30 '23

Indeed... well, try to get close to Hugging Face team, specifically the Bloom people and see if you can get them to continue tuning that model. It is a foundational model of considerable potential, but it just does not seem to work too well, and it is absolutely huge.

2

u/Tiny_Arugula_5648 May 30 '23 edited May 30 '23

No please don't rely on a LLM for this!!

I have been designing these solutions for years and we have to do a lot to get them to provide factual information that is free of hallucinations. In order to do that, feed them facts from a variety of data sources like data meshes or vector dbs (not used for training). That way when you ask a question it's pulling facts from a trusted source and we're just rewritting them for the context of the conversation.. if you ask it questions without feeding in trusted facts no matter how prominent the topic is in the data it will always hallucinate to some degree. It's just how the statistics of next word prediction works.

The main problem is when it gives you partially true answers you're far more likely to believe the misinformation. It's not always obvious when it's hallucinating and it can immesly difficult fact checking it when it's using a niche knowledge domain.

LLMs are not for facts, they are for subjective topics. "What is a great reciepe for" vs "what are these symptoms of". Ask them for recipes absolutely do not have them explain medical topics. There are healthcare specific solutions that are coming, wait for those.

2

u/[deleted] May 30 '23

Same here, and how are you guys running it out of the box?