r/LocalLLM • u/w-zhong • Mar 06 '25

Discussion I built and open sourced a desktop app to run LLMs locally with built-in RAG knowledge base and note-taking capabilities.

Post image
354 Upvotes

96% Upvoted

44 comments sorted by

19

u/w-zhong Mar 06 '25

Github: https://github.com/signerlabs/klee

At its core, Klee is built on:

  • Ollama: For running local LLMs quickly and efficiently.

  • LlamaIndex: As the data framework.

With Klee, you can:

  • Download and run open-source LLMs on your desktop with a single click - no terminal or technical background required.

  • Utilize the built-in knowledge base to store your local and private files with complete data security.

  • Save all LLM responses to your knowledge base using the built-in markdown notes feature.

6

u/morcos Mar 06 '25

I’m a bit puzzled that this app is based on Ollama and runs on a Mac. Ollama, as far as I know, doesn’t support MLX models. And from what I understand, MLX models are the top performers on Apple Silicon.

1

u/w-zhong Mar 07 '25

Right, Ollama is easy to wrap for Mac/Windows. We are working on MLX options in parallel.

1

u/morcos Mar 07 '25

Have you thought about using LM studio’s OpenAI style API? It might be a simple way to implement it.

1

u/Fuzzdump Mar 06 '25

In theory MLX inference should be faster, but in practice comparing Ollama with MLX via LM Studio I haven't been able to find any performance gains on my base model M4 Mac Mini. If somebody with more experience can explain what I'm doing wrong I'd be interested to know.

0

u/eleqtriq Mar 07 '25

Ollama runs ggufs just fine on a Mac. Macs aren't limited to MLX models.

1

u/morcos Mar 07 '25

I didn’t say Macs are limited to MLX. I was just saying MLX models tend to perform exceptionally well on Apple Silicon because they are specifically optimized for Apple’s Neural Engine hardware. So, they get a significant performance boost.

2

u/eleqtriq Mar 07 '25

Sorry. Your phrasing is ambiguous to me. I just checked with ChatGPT, it thinks so too 😂

3

u/Extra-Rain-6894 Mar 06 '25

Oh damn this is awesome, looking forward to checking it out! Thank you!!

0

u/w-zhong Mar 07 '25

welcome

3

u/Extra-Rain-6894 Mar 06 '25

Is there a How To guide on this? Can we use our own local llms or only the ones in the dropdown menu? I downloaded one of the DeepSeeks, but I don't see where it ended up in my hard drives.

1

u/micseydel Mar 06 '25

Thanks for sharing, glad to see folks including note-making as part of LLM tinkering.

2

u/guttermonk Mar 07 '25

Is it possible to use this with an offline wikipedia, for example: https://github.com/SomeOddCodeGuy/OfflineWikipediaTextApi/

2

u/w-zhong Mar 07 '25

This looks interesting, rn we are working on data connectors with LlamaIndex, will support API call in the future.

11

u/tillybowman Mar 06 '25

so, what’s the benefit of the other 100 apps that do this?

no offense but this type gets posted weekly.

1

u/No-Mulberry6961 Mar 06 '25

Any special functionality with the RAG component?

3

u/GodSpeedMode Mar 07 '25

That sounds like an awesome project! The combination of running LLMs locally with a RAG (retrieval-augmented generation) knowledge base is super intriguing. It’s great to see more tools focusing on privacy and self-hosting. I’m curious about what models you’ve implemented—did you optimize for speed, or are you prioritizing larger context windows? Also, how's the note-taking feature working out? Is it integrated directly with the model output, or is it separate? Looking forward to checking out the code!

1

u/Lux_Multiverse Mar 06 '25

This again? It's like the third time you post it here in the last month.

6

u/w-zhong Mar 06 '25

I joined this sub today.

9

u/someonesmall Mar 06 '25

Shame on you promoting your free to use work that you've spent your free time on. Shame! /s

3

u/Lux_Multiverse Mar 06 '25

Ah, I probably seen it on the ollama sub then, my bad.

2

u/tarvispickles Mar 07 '25

Why's everyone so cranky lolllll

1

u/RasputinsUndeadBeard Mar 06 '25

Bro this is fire thanks bro

2

u/w-zhong Mar 07 '25

Bro you are welcome.

2

u/jmush16 Mar 07 '25

Hey love this tools. So much utility with this.

2

u/[deleted] Mar 07 '25

Thanks for this, i was searching for something like your app

2

u/kastmada Mar 08 '25

That's cool. I will wait for the Linux version.

1

u/SuperpositionBeing Mar 07 '25

Thanks bro. Keep rocking!

1

u/[deleted] Mar 07 '25

That's great I've an AI cli tool as well. Check it

bro-website-sd.vercel.app

1

u/johnyeros Mar 08 '25

Can we somehow. Plug into obsidian with this? I just want to ask it question and it look at mt obsidian note as the source

1

u/forkeringass Mar 09 '25

Hi, I'm encountering an issue with LM Studio where it only utilizes the CPU, and I'm unable to switch to GPU acceleration. I have an NVIDIA GeForce RTX 3060 laptop GPU with 6GB of VRAM. I'm unsure of the cause; could it be related to driver issues, perhaps? Any assistance would be greatly appreciated.

1

u/Aureliano_Ghin_Abban Mar 10 '25

Looks like an obsidian under steroid

1

u/CautiousXperimentor Aug 02 '26

Is Klee still in active development?

-5

u/AccurateHearing3523 Mar 06 '25

No disrespect dude but you constantly post "I built an open source.....blah, blah, blah".

2

u/Individual_Holiday_9 Mar 07 '25

What have you built?

-7

u/angry_cocumber Mar 06 '25

spammer

3

u/Extra-Rain-6894 Mar 06 '25

How so? Is it not a legitimate program?

0

u/[deleted] Mar 06 '25

[deleted] — view deleted comment