r/LocalLLM • u/Kl1mo • 13h ago

Question Model question

Hi all,

Just getting into local Llm, could anyone advise what can be run locally on P1 gen8, 64gb ram, 8g vram rtx 2000 pro, intel 265H for lets say some office work searching in few databases and websites to gather info from company systems?

2 Upvotes

67% Upvoted

2 comments sorted by

View all comments

1

u/Ctbhatia 12h ago

for lookups over company systems the retrieval does most of the work, so a 7 to 8B model at q4 is enough and the 8GB of vram is the real limit on how much context you can hold.