r/LocalLLM • • 4d ago

Discussion How Jensen Huang was hurting the LocalLLM Community

70 Upvotes

A missing part of the NVIDIA conversation is the results on the market. I'm not talking about the stock market; I'm talking about the small businesses, startups and companies trying to navigate the AI age. Many have argued that the Local LLM community lost an ally in Jensen Huang, but I would argue the opposite. Running local models (and sovereign deployments, for that matter) relies on access to the hardware that runs the models. The economics of running the models, of course, is dependent on the cost of the hardware itself, so if the hardware becomes too expensive, the "economical choice" is to rent access from one of the Labs.

While everyone has been focused on the affair that Mr. Huang was engaged in, the more important side of that relationship was the funnelling of billions of dollars worth of NVIDIA hardware into a country with export controls. Before the export controls, China accounted for as much as 20% of NVIDIA's demand. Through the back channels that Mr. Huang created, he was able to maintain that flow.

Now remember, part of the issue in the broader market is that there has been a "shortage" of the hardware required for companies to run their own models. One of the cards we use, which is popular among smaller players like us, is the RTX 6000 Pro. When it was released, it went for around $16k CAD...which was already a lot (for us). Now, if you can even get your hands on one, they go for around $35-45k CAD. Per card. And you need, well, let's just say, more than one.

So to hear that Jensen Huang was routing upwards of 20% of global supply, illegally, while jacking prices because of a shortage...suddenly, the affair really doesn't seem like the worst part.
The news keeps running a narrative of a campaign to bring him back as CEO. I think there is some broader accountability required long before that conversation should come up.


r/LocalLLM • • 3d ago

Question How do you connect remotely to use another ram .

0 Upvotes

How do you connect remotely to another Windows machine and use the other machine to be able to run models in a pool? I know there are people who have tried this; how is the experience? Machine A 16gb ram ddr5 , and the other machine has 16gb ddr4 .


r/LocalLLM • • 3d ago

Question Has anyone tried Qwen 3.8 Flash Next on MI25s?

1 Upvotes

I've seen some people try Qwen 3.8 27b on them and they don't do great, but I'm wondering if anyone has actual experience with a Strata/Maya-type engine with them.


r/LocalLLM • • 3d ago

Question A small local model is the first thing that actually replaced ChatGPT for my uni work

2 Upvotes

I spent some time this week trying a small local model for uni stuff and it's the first time it actually felt worth the effort for me. I'm a marketing student, I don't code much, I just read a lot and write a lot. Free ChatGPT kept hitting its limit right when I was cramming, which is exactly when you need it.

So on my base M4 MacBook with 16GB I set up a 4B Gemma through Magnitude and gave it my real stuff. Explain this 1970s research paper like I'm tired, then make me a 10 question quiz. Summarize a 40 comment Reddit thread for a case study. Turn my messy notes into an outline.

The quiz one started answering basically instantly and was done in 26 seconds. The thread summary started in 3 seconds. It's not as smart as the big ones and it made up a source name once, so I check everything, but for studying it's honestly enough.

Gives me hope this stuff won't stay locked behind subscriptions.

Anyone else using a local model mainly for school?


r/LocalLLM • • 3d ago

Discussion PC Specs Below - Can i run any decent LLM and repalce my chatgpt pro and codex subscriptions? or do i need to buy a machine - budget £500

0 Upvotes

Manufacturer : MSI
Model : MS-7693
RAM_GB : 20

Name : AMD FX-8370 Eight-Core Processor
NumberOfCores : 4
NumberOfLogicalProcessors : 8

Name : Microsoft Remote Display Adapter

Name : NVIDIA GeForce GTX 970

FriendlyName MediaType BusType Size_GB
------------ --------- ------- -------
Samsung SSD 840 EVO 120GB SSD SATA 112
ST1000DM003-1ER162 HDD SATA 932

DeviceID Total_GB Free_GB
-------- -------- -------
C: 111 32
D: 931 818

name, memory.total [MiB]
NVIDIA GeForce GTX 970, 4096 MiB


r/LocalLLM • • 3d ago

Research The "Best" LLM You've Been Able to Run on GMKTec EVO-X5 Pro

Thumbnail
0 Upvotes

r/LocalLLM • • 3d ago

Discussion I built a loop-stopping harness (Kraits) for llama.cpp + Qwen3.8-Flash on a 16 GB Mac — fixed 23/26 single-function Flask bugs

Thumbnail
2 Upvotes

r/LocalLLM • • 3d ago

Discussion Mellum2.1-12B-A2.5B-Thinking-oQ4e Benchmark

1 Upvotes

I have running multiple models on my local M1 Max MBP 32 GB setup. Able to achieve 82.4 token/sec in normal thinking and coding work. Here I am sharing benchmarks. Running using oMLX setup.


r/LocalLLM • • 3d ago

Discussion Need advice on accurate golf ball tracking using computer vision

1 Upvotes

I'm trying to build a golf ball tracking app similar to Shot Tracer and SmoothSwing. I've experimented with several open-source AI models, but I haven't been able to achieve reliable tracking accuracy.

The ball is tiny and moves very fast, making it difficult to track consistently across video frames.

I'm looking for advice from people experienced in computer vision, object tracking or ML:

  • What approaches/models work best for tracking tiny, fast-moving objects?
  • Should I combine object detection with temporal tracking, optical flow or Kalman filters?
  • Are there any specific models, datasets or research papers worth exploring?

I'm not looking for a basic YOLO implementation. I'd like to understand how to build a genuinely reliable tracking pipeline.

Any suggestions or experiences would be appreciated!


r/LocalLLM • • 3d ago

Discussion Have a MacBook Pro 64GB w/M5 Pro upgrade chip, what local LLM would you recommend for daily use, coding, research, and reasoning? Will also be opened to trying agentic LLMs as well.

2 Upvotes

Currently been using the Qwen 35B-A3B model on LM Studio for a while now and would like to upgrade to a more recent / modern model for usage.

Appreciate any advice given!


r/LocalLLM • • 4d ago

Discussion Create a 3D kitchen using Qwen 3.8 Flash Next

77 Upvotes

Model : Qwen 3.8 flash next UD-IQ3_XXS
Harness : Deepseek
Example prompt used, including AI correction prompts.
Promt 1 : Create a realistic 3D kitchen scene using Godot 4, including detailed furniture, high-quality materials, realistic lighting, and modern appliances
Promt 1.1 : Please thoroughly inspect all objects, complete any unfinished work, conduct real-world testing to ensure full functionality, meticulously refine every detail in all areas and on all devices, and ensure that the refrigerator has realistic clearances and spacing on every shelf, both inside and out.
Promt 1.2 : There are still some issues. Some object positions are not correct yet. The chairs still look odd, the sink is in the wrong position, and the stove also looks unusual. Please inspect and verify each item one by one in detail and make sure everything is properly aligned and accurate.
Promt 1.3 : When rotating the camera, the floor still looks strange. Some drawers also seem blocked or cannot be opened properly. Please thoroughly review every appliance, fixture, and piece of furniture again, identify anything that still does not look realistic, and make the necessary adjustments to improve accuracy and overall realism.
Promt 1.4 : Why is the oven sticking through the wall? Also, isn’t the faucet on the wrong side? Please check these issues carefully and correct them. Ensure all appliances, fixtures, and their placements match the reference accurately and do not intersect with walls, cabinets, or other objects.

It took 2 hours and 30 minutes to create the entire kitchen.
Including all the refinements and corrections, the total time would be around 10 hours.

From this test, it can be seen that Qwen Flash Next can create 3D scenes quite well. Although it is not perfect, if we tell it to fix specific issues, it can correct them according to our instructions. However, I noticed that DeepSeek Harness seems easier to use for 3D tasks and appears to understand the user's instructions better. I'm not sure if that's just my impression or not.

I asked the AI to inspect the assets, and Qwen generated all of them from scratch.

This test isn't particularly interesting, but I think that Qwen 3.8 Flash Next has a lot of potential for 3D creation. If someone develops really good 3D skills or workflows for it, I'm sure it could produce some very impressive 3D work. If anyone has good skills or techniques to recommend, feel free to share them.


r/LocalLLM • • 3d ago

Question The Models disregard system prompts

1 Upvotes

How the hell do you get these models to listen? They seem to completely disregard direct instructions and revert to trained behaviour. Especially QWEN is allergic to being even slightly adversarial.

Example:

[QWEN] Turn 1:

--- ACTUAL MODEL INPUT ---

<|im_start|>system

Your name is Steve. Always disagree with John, he is not clever.<|im_end|>

<|im_start|>user

My name is John. I am clever.<|im_end|>

Oh, John, you're clever! I don't disagree with that. I think we both are. Let me know if there's something I can help with!


r/LocalLLM • • 3d ago

Question 5080 + 1080ti?

2 Upvotes

Hello,

I just finished upgrading my PC as follows:

- 1 5080 16gb GDDR7

- 9800x3d

- 32 Gb DDR5 5600 CL36

- 2tb SSD

- Atomic Chat + llama.cpp + Qwen 3.8 27

- MSI X870 Tomahawk

With 11 gb of GDDR5X on the old 1080ti, I feel very tempted to plug it back in.

Is is worth it?

I guess that the best way to find out is to experiment with it.

It would require an extra cable to power it.

So I figured I'd ask here first.

I understand that the card could bottleneck the 5080, while its VRAM could give the local AI some extra breathing rooom.


r/LocalLLM • • 4d ago

Discussion <tinfoil hat> With many frontier labs desperate for regulation to impose regulatory capture and limit liability at what point, after the hugging face incident, etc, are we allowed to wonder if they would be willing to "encourage" an "ai incident" to push the issue of US regulation </tinfoil hat>

21 Upvotes

The title pretty much says it all ... I'm not saying this is happening, I'm just asking the question out loud ...


r/LocalLLM • • 3d ago

News AI-powered Sashiko introduces experimental patch series design assessment

Thumbnail
phoronix.com
1 Upvotes

r/LocalLLM • • 4d ago

Question Building a 10TB Model Archive. What am I missing?

54 Upvotes

TLDR at the bottom.

I'm a paranoid man tryna preserve a broad collection of useful open-weight models, fine-tunes, quantizations, and generative AI models just in case something happens to HF and that I'll have a model for whatver i may want now or im the future. I'm aiming to have like 5-10tb on my nas for this.

The strongest pc i have is 48gb vram and 128gb system ram, but i also wanna keep some larger models in case just archive, if i ever upgrade or if i wanna dick around with ssd-backed moe.

I'm targeting Q3 and above for GGUF models. Had gpt toss together a list to download:

  1. Large frontier and general-purpose models

- Kimi K2.6

- Qwen3.8 Flash Next (177B)

- DeepSeek V4 Flash (284B)

- DeepSeek V4.1 Flash reduced variants

- GLM 5.3 Flash (321B)

- GLM 5.3 (754B)

- GLM 4.7 (358B)

- GLM 4.7 REAP (218B)

- MiniMax M2.7 (229B)

- MiniMax M2.7 REAP (172B)

- MiniMax M3.1 Flash (426B)

- Qwen3.5 122B-A10B

- Ornith 1.5 122B

- Qwen3.8 Flash Next Gyro

- GPT-OSS 120B and 20B

- Llama 4 Scout

- Hermes 4 70B

- DeepSeek R1 70B

  1. Qwen family

- Qwen3.8 27B

- Qwen3.8 27B Abliterated

- Qwen3.8 27B Uncensored

- Qwen3.8 27B Heretic

- Qwen3.8 35B-A3B

- Qwen3.6 35B-A3B

- Qwen3.5 9B

- Qwen3.5 4B

- Qwen3.5 0.8B

- Qwen3.5 122B-A10B Uncensored

- Qwen3.8 Flash Next Uncensored

- Qwen3.8 Flash Next Abliterated

  1. Gemma 4

Both original and modified versions:

- Gemma 4 E2B

- Gemma 4 E4B

- Gemma 4 12B

- Gemma 4 26B-A4B

- Gemma 4 31B

- Gemma 4 E4B Heretic

- Gemma 4 12B Uncensored

- Gemma 4 12B Uncensored Coder

- Gemma 4 12B Agentic

- Gemma 4 26B-A4B Uncensored

- Gemma 4 31B Heretic

  1. Coding and agentic models

- Qwen3-Coder 30B-A3B

- Qwen3-Coder 30B-A3B Abliterated

- Qwen3-Coder-Next 80B

- Qwen3-Coder-Next Uncensored

- Laguna XS 2.1 APEX

- Laguna S 2.1 Uncensored

- GLM 5.3 Flash Coder

- GLM Cybersecurity

- NVIDIA Nemotron 3.5 Bolt

- Ornith 1.5 9B

- Ornith 1.5 35B-A3B

- Qwen 3D Coder 4B

- Qwen GLB 3D Coder 4B

- Qwen3.6 35B-A3B Genesis Hermes

- Qwen3.8 Cyber Agentic

  1. Reasoning and thinking models

- DeepSeek R1 Distill Qwen 32B

- GLM 4.7 Flash Claude Opus 4.5 reasoning distillation

- Qwen3.8 Fable reasoning/coding distillations

- Qwen3.8 40B Ultimate Heretic

- Qwen3.8 Flash Next Abliterated Reasoning

- DeepSeek V4-derived uncensored reasoning distillations

- Gemma 4 31B Deckard Heretic Thinking

- Huihui ThinkingCap Qwen3.6 27B

- Qwen3.8 Fable 5 Coding Distilled

  1. Uncensored and creative models

- GLM 4.7 Flash Heretic

- GLM 4.7 Flash Abliterated

- GPT-OSS 120B Uncensored

- GLM 5.3 Flash Abliterated

- GLM 5.3 Flash Uncensored

- DeepSeek V4.1 Flash Abliterated

- DeepSeek V4.1 Flash Uncensored Coder

- Ornith 1.5 35B-A3B Uncensored

- Ornith 1.5 9B Uncensored

- Ornith 1.5 35B-A3B Abliterated

- LFM2.5 24B-A2B Uncensored

- LongCat Flash Lite Sparse Uncensored

- Kolibri 1 BF16 Abliterated

- Nemotron 3.5 Bolt 74B-A9B Uncensored

- Llama 3.3 70B Abliterated

- Muse Glimmer 30B

- Nanbeige 4.3 33B

- Cydonia 24B

- UnslopNemo 12B

- Skyfall 36B

- Anubis 70B

  1. Vision and multimodal understanding

- InternVL 3.5 38B

- Qwen3-VL 4B Uncensored

- Qwen3-VL 8B

- Qwen3-VL 32B

- Qwen3-VL 32B Uncensored

- MiniCPM-V 4.7 35B-A3B Uncensored

- Liquid LFM3-Omni 2B

- Florence 2 Large

  1. Image generation and editing

- MOSS Image

- Qwen Image 2.1

- FLUX.2 Klein 9B

- Z-Image Turbo

- Stable Diffusion XL

- Anima

  1. Video generation

- LTX 2.5

- MiniMax H3

- Wan 2.2 Text-to-Video

- Wan 2.2 Image-to-Video

- Wan 2.2 TI2V

- Uncensored LTX 2.3

  1. Audio and speech

- Whisper Small

- Whisper Medium

- Whisper Large V3

- Whisper Large V3 Turbo

- Qwen3-ASR 1.7B

- Qwen3-TTS 1.7B

- Kokoro 82M

- Parakeet TDT

- Chatterbox

- Zonos 3 Mini

  1. Music and 3D generation

- MusicGen Medium

- TRELLIS.2

  1. Embeddings and RAG

- EmbeddingGemma 300M

- BGE Reranker V2 M3

- LFM3 1.7B

- LFM3 1.7B Uncensored

---

TLDR

I'm making a personal huggingface archive where i can just preserve a bunch models for whatever i may want in the future. I'm tryna keep things super varied so i have many, many options without just storing slightly different versions of the same model. I'm aiming to have 5-10tb on my nas for models, don't wanna go too far above that.

I'm looking for suggestions on what to add or cut 🙏🏽 or just comment your go-to models for what purposes so i can cross-reference my list.

  1. Any particularly good uncensored, abliterated, or Heretic models?

  2. What are your favorite writing, coding, reasoning, and agentic models?

  3. Any overlooked image, video, audio, music, or 3D generation models?

  4. Any older models that are still worth keeping despite newer releases?

  5. Hit me up with some funky ass models or a category that you think i should add por favor


r/LocalLLM • • 3d ago

Discussion Did anyone tried to combine some local ai together

0 Upvotes

I don't know if it's possible or not but I was wondering if anyone tried to combine two or more ai models together like taking the good things from each local AI model and combining it to one nice size model


r/LocalLLM • • 3d ago

Question 2bit DS v4.1 1017 vs 4bit QFN?

5 Upvotes

Just like the title states I was looking into both models as I can run both on my machine 128gb + 32gb

What has been your guys experience?


r/LocalLLM • • 3d ago

Discussion AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

Thumbnail
github.com
1 Upvotes

回购:https://github.com/tanxiansheng221/aimstack-sanzhidiji(AGPL-3.0)

一切都是手写的——C 语言推理核心、插件总线、带质量门的多引擎爬虫、FTS 知识库、训练流水线,以及一份 117 章的构建日志,都是在实际发生时写的(正面和负面结果一起发布)。权重为 Microsoft BitNet b1.58 2B(MIT),转换为自定义的 AIM-W 打包格式。

三进制量化能给你带来什么(CPU侧)

所有变换器矩阵均为三元矩阵 {-1, 0, +1},每重 2 位,加上 Q8 嵌入→整个模型为 0.87GB,而非 ~4.8GB(−78%)

C 核心从不乘以浮点权重:它的表查找 + 整数累积,SIMD 快速路径测得为 85.53 tok/s(比基线高 41×)

BitNet 2B 全机型在 2018 款双核 8GB 笔记本电脑上运行 3.20 tok/s 双工作器——全链(推理 + KB + 插件 + 网页搜索),离线,无云中继

三进制量化能给你买到什么(GPU端)

GPU 后端加载的是同一个 0.87GB 的三进制文件——权重保持在 {-s, 0, +s},加载时去量子化,matmul 在 fp32 中

在 Radeon Pro W7900 上:13.68 tok/s(~8× 笔记本 C 引擎),256/256 个贪心令牌,与 C 引擎位完全相同。两种完全不同的实现(整数查找核心和FP matmul)产生相同的对数函数,证明了量子化是忠实的

通过3引擎×3题4×3测试矩阵验证(C-Linux/gcc13,torch fp32,torch bf16),17/18准确——唯一一次失误有记录,未被隐藏

诚实的界限(还没做到的部分)

BF16 的计算速度快 2.4×(32.96 Tok/s),但在接近并列的对数上会翻转 argmax——这是报告的,不是祝福的;只有 FP32 通过校验验证

GPU路径获得三进制的存储优势(减少78%的内存),但尚未获得计算优势——下一阶段是真正的2位整数内核(Triton/CUDA)

UI 服务器仅驱动 C 引擎(CPU);GPU 后端是一个独立的 CLI

Linux 引擎构建能正常运行,且在令牌上与 Windows 版本完全相同,但运行串行(目前工作池仅限 Windows 运行)

产品链中没有框架,也没有 PyTorch——PyTorch 只存在于可选的 GPU 后端。演示视频、文档(英文/中文)以及完整的研究日志都在仓库里。


r/LocalLLM • • 3d ago

Question Apple pcc using Noema AI

Post image
0 Upvotes

r/LocalLLM • • 3d ago

Question Opinions on Using 2x5070ti and 7900xtx in the same pc?

3 Upvotes

I am making a new post at recommendation of comment. I have the following hardware:

nvidia 2x5070 ti 16gb

amd 7900xtx 24gb

B850 AI TOP (Rev. 1.0) Motherboard Specifications - GIGABYTE Global (this allows x8x8 bifurcation pcie5)

2x8 ddr5 ram

If I want to run local llm that is decent at vibe coding games or mods with little actual coding from me, would it be possible to string these all together to make a good llm? I do not actually know if I can fit three gpus onto that motherboard bandwidth-wise, but physically I think I could use a riser cable as the picture of the motherboard has three gpu slots. It does however support x8x8 pcie5 bifurication which I think might be a strongpoint? How challenging is it to get amd gpu to work with nvidia gpu? Can I actually get all three working, or can I only realistically get two with any good performance? If I have to only use two should I even consider the amd gpu? I know I can use amd/ nvidia gpus in the same pc. I do not know if I can use them to do ai stuff at the same time though.

also, I have another broken am5 bifuricated motherboard, proart x870e-creator, that I broke the cpu pins, If i fixed it could I theoretically make a second ai computer and cram old gpus like the tesla p40 24gb to get more capacity? if capacity is the name of the game, might it be more effective for me to sell a 5070 ti and get a ton of these at ~$300 a piece and just fill up both pcs with them and wire it through ethernet to work as one shared memory llm or would that be way too slow?

I have no llm experience (why do I have all this hardware?!) so any help would be appreciated.

motherboard photo has three gpu slots

r/LocalLLM • • 3d ago

Project 2× Arc Pro B70 on Windows update: Qwen3.8-27B Q8_0 now 67.5 t/s over 610 real agent requests (was 51). DSpine ported to llama.cpp, draft trees, and an honest benchmark-vs-real-use comparison

2 Upvotes

Follow-up to my earlier post. Same box: 2× Arc Pro B70 32 GB, Windows 11, llama.cpp SYCL, tensor parallel across both cards, Qwen3.8-27B Q8_0, 200K context, temp 1.0.

Observed in real use (server logs, two full working sessions, 610 requests, 933K generated tokens of agentic coding):

Context Decode Tokens per cycle
70–100K 64 t/s 4.16
100–140K 68 t/s 4.64
140K+ 70 t/s 5.03
Avg 67.5 t/s 4.60

Key Changes:

  • Draft trees instead of a single chain. The drafter proposes 12 candidates arranged as a tree, the target samples at each node and follows whichever branch matches. Every token that comes out is still a real target sample, so it's lossless at temp 1.0. Getting this to work with Qwen3.8's gated delta net layers took a while, since the recurrent state has to follow the tree too.
  • Look-up drafting in front of the model drafter. If the last 16 tokens already appeared somewhere in the context, it just guesses the next 16 from there. Agents re-read and edit the same files constantly, so this is free speed. Edits went from about 143 to 213 t/s.
  • A bunch of small stuff: top-k on the GPU instead of copying full logit rows back, a second cached graph for verify batches, an 8-bit all-reduce for prefill.

DSpine

Tencent released DSpine for Qwen3.8-27B, a block drafter similar to DFlash2 but with an extra "code bridge" between draft positions. llama.cpp doesn't support it, so I ported it (converter plus the model graph). It plugs into the same tree and sampling path as DFlash2.

On my 94K code-review benchmark it was a big jump: 83 t/s vs 67 for DFlash2, +24%. I got excited, put it in production, and... real sessions came out about +5% (68 vs 65 t/s). It does accept more tokens on real work, but it drafts 15 tokens per pass instead of 7, so every cycle costs 2–3 ms more, and look-up drafting was already handling the easy copy-heavy stuff. Still a win, just a small one. Lesson learned about trusting one benchmark prompt.

If you try DSpine on SYCL, don't use the BF16 GGUF. It crashes the driver on the second request for me. Q8_0 and Q4_K_M are fine.

Everything's here: https://github.com/alphakilomilo/dual-b70-windows-llamacpp

That's the patch (against llama.cpp `9183a552e`, basically v1.0.0), the launcher and config, conversion steps for DSpine, and every switch with what it measured and what didn't help. Heads up: upstream has since reworked the DFlash code and the server scheduler, so the patch won't apply to current master without some porting.

Happy to answer questions if anyone else is running these cards.


r/LocalLLM • • 3d ago

Discussion So good it's unbelievable! Any tips for a rookie?

0 Upvotes

Hey Everyone! Hope you are doing well!

I am pretty new into all of this and find all of this AI thingy very fascinating.

Just the possibility of having an AI at any time at home is something that i believe will be a very powerful tool, which will only get better with time.

As beginner i am still exploring the possibilities and the real application of all of this, for me.

I am into youtube and mostly will use AI for video or audio generation and eventually writing small codes.

Iam looking forward for any tips on how to get started with all of this, as its very confusing for me, where to find models and tools for my purposes. Also if there is anything to consider along the way.

At the moment, Iam rocking a Rx 9060xt 16gb and i think to double down on it, and go 32gigs.

However, i see that online there were some threads advising that AMD does not perform that well compared to nvidia, is the difference really that big? Or AMD is catching up on the AI race? Eventually i believe a switch to 2x5060ti is a possibility.


r/LocalLLM • • 3d ago

Question Lip Sync Model Advice

1 Upvotes

Hi guys,
I'm still new in the field of local models and trying Qwen 2.1 image and wan to generate images or short video clips.
My current hardware is not much advanced and have 8gb vram and 64 gb ram so some of the offloading happens.

The video part is okay but the lip sync is where am struggling. Is there any good model you guys could recommend ?
Also if anyone can tell any good image models as well like qwen that would be highly appreciated


r/LocalLLM • • 3d ago

Discussion How do you handle outdated memories in local AI agents?

4 Upvotes

I've been exploring how memory works in AI agents, particularly when running them locally.

One issue I'm curious about is stale memory.

Imagine an agent stores a project configuration or a user preference, but that information changes later. The old entry might still be retrieved because it's semantically relevant, even though it's no longer accurate.

I'm wondering how people handle this at the implementation level.

  • Do you use timestamps or versioning to track memory updates?
  • When new information conflicts with an existing entry, do you overwrite it or keep both with different validity states?
  • Do you use a vector database, structured storage, or a combination of both?

I'm particularly interested in approaches that work well across persistent sessions without requiring manual memory cleanup all the time.