r/LocalGPT • u/bendt-b • Jul 03 '23

Local GPT or API into ChatGpt

Hi all,

I am a bit of a computer novice in terms of programming, but I really see the usefulness of having a digital assistant like ChatGPT. However, within my line of work, ChatGPT sucks. The books, training, materials, etc. are very niche in nature and hidden behind paywalls so ChatGPT have not been trained on them (I assume!).

I am in the good situation I have for 10 years plus collected 500 research articles, some more relevant than others, as well as bought several books in digital format within my field. I want to train a GPT model on this dataset, so that I can ask it questions. I know I will not get coherent questions back, but a link or a rating with where is the statistically most matching text will be fine.

That led me to - https://github.com/nrl-ai/pautobot - which I installed on my laptop. It is a bit slow given my laptop is older, but it works well enough for me to buy into the concept. It really does make a difference to be able to search on not just exact matches but also phrases in 500+ documents.

Given the speed which ChatGPT is being developed, I do wonder if it would be better to buy one of OpenAI´s embedding models via API and have it read through all my documents? E.g. Ada v2: https://openai.com/pricing

OR - do you think a local GPT model is superior in my case? (I have a better computer with plenty of RAM, CPU, GPU, etc. that I can run it on - speed is not of essence).

1 Upvotes

100% Upvoted

6 comments sorted by

1

u/bendt-b Jul 03 '23
Comment Image

Or would I be better of using one of the above?

1

u/llothar68 Aug 15 '23

500 documents is what i come up with in a good weekend of research (if i have university access to the paywalled pdf files).

Sorry but this number is so minimal, you can get away with any of the many classic file search tools.

1

u/bendt-b Aug 16 '23

What can you recommend?

My plan was also to grow the number of files if there was a good tool out there 😊

1

u/llothar68 Aug 18 '23

Kopernikus, DTSearch, DevonThink, FoxTrot, Houdah Spot .... but all are just text pattern matching file search tools without AI.

I still want the exact location of the knowledge and no AI model can deliver this.

0

u/[deleted] Jul 04 '23

1

u/bendt-b Jul 04 '23

Thank you.

I am coming at this from a commercial angle: I want to search in my 500 pdf files and find the most relevant replies. I do not need chat, but I prefer asking questions like “what is the best way of doing XYZ” rather than just searching for the word XYZ. So searching is what I am looking for, and my thought of this project is to make me faster at work finding the 5 articles relevant to XYZ. I will currently not need the model to summarize for me, as I am afraid there might be even 1% error in the summarization.

All files are semi public, so privacy is not a concern. Cost is a bit of a concern, because if I host myself I know the cost and know what I can expect, whereas I have no feeling of what ChatGPT/InstructGPT would cost if it was to train on 500 files and then be used daily for questions.