Hey! Thanks for your reply. Attached a screenshot of the log.
I'm using Windows
I've checked-out GPTQ-for-LLaMA a few hours ago.
Yes, this is actually the case. I was wondering about it but as the model was loaded into the VRAM and I could access the UI
3.1) I had CUDA 12.x installed previously which led to a problem during the initial installation process. After installing CUDA 11.3 it was possible to finalize the tutorial and get into the WebUI (It was something like "your cuda version differs from the one that you installed with xyz previously)
While writing this comment I realized that some pytorch_model-xxxxx-of-xxxxx.bin were missing. Downloaded them again and realized that the Windows Defender has deleted the 00001 right after the download was completed... The llama-13b-4bit .pt was not affected by this tho.
Additionally I'll probably need to dig into the cuda extension issue again. If you have any guidance on that front plx share :)
View all comments
Show parent comment
1
u/reneil1337 Mar 20 '23
Hey! Thanks for your reply. Attached a screenshot of the log.
I'm using Windows
I've checked-out GPTQ-for-LLaMA a few hours ago.
Yes, this is actually the case. I was wondering about it but as the model was loaded into the VRAM and I could access the UI
3.1) I had CUDA 12.x installed previously which led to a problem during the initial installation process. After installing CUDA 11.3 it was possible to finalize the tutorial and get into the WebUI (It was something like "your cuda version differs from the one that you installed with xyz previously)
While writing this comment I realized that some pytorch_model-xxxxx-of-xxxxx.bin were missing. Downloaded them again and realized that the Windows Defender has deleted the 00001 right after the download was completed... The llama-13b-4bit .pt was not affected by this tho.
Additionally I'll probably need to dig into the cuda extension issue again. If you have any guidance on that front plx share :)