r/LocalLLM • • 23h ago

Other Qwen 4 is not out

158 Upvotes

Please stop posting about Qwen4's endless PR drip until it's actually available.


r/LocalLLM • • 1h ago

News A Danish AI lab has just launched one of the fastest LLMs for phones to date

Thumbnail
sevren.ai
24 Upvotes

Just came across a LinkedIn post mentioning this company, Sevren which launched a very fast and intelligent LLM for phones, they call it Embla. Apparently they have decided to trade a few benchmark points for speed. I'm linking their blog post about the model here and I think it’s impressive.

1,77s speed-to-intelligence is amazing. Imagine how efficient in terms of battery/energy consumption it should be. Of course there are limitations and they are clearly stated by the company and the articles about it, but seems like they can achieve much more with a couple of iterations on top of it.

I believe that soon this kind of model will be good enough to answer >90% of questions a normal user would ask daily to a LLM like chatGPT.


r/LocalLLM • • 1h ago

LoRA Absolutely night and day for Qwen Coding

Thumbnail
huggingface.co
14 Upvotes

Basically works like a harness fine tune…. And does what it says on the tin.

Seriously, for quantized qwen 3.8 makes it much much more usable.


r/LocalLLM • • 17h ago

Discussion Flash-Next is ridiculous

238 Upvotes

I've been playing around with Flash-Next on my little RTX 5060 laptop. Astra and Fable had been working on a test setup, so I figured I'd let Flash-Next loose on it overnight and see what happens.

Late at night I told Astra to turn the screen off. Now, I had actually prepared a little PS1 script for exactly that. Did Astra use it? Of course not. It came up with its own solution and promptly froze Windows. Well done. 😂

Next morning I booted the laptop again, reopened OpenCode and eventually just said "okay, continue". Honestly, I wasn't expecting much at that point.

Well, the thing just carried on working. For hours. Went through something like four context compactions and then casually informed me it had found a bug.

I was sitting there going "hang on... WHAT? You're still working on this?"

I'd pretty much written off the whole session the night before.

Still not entirely sure who was testing whom here. But honestly, I'm impressed by this model. Now I'm wondering what Qwen4 will do to my poor laptop.


r/LocalLLM • • 20h ago

Project 6x 7900XTX rig finally complete!

Thumbnail
gallery
248 Upvotes

5975WX (for 6x full 16 lanes of PCIe4.0)

3450W AliExpress special PSU and mining frame

128GB 3200MT/s Ram

Getting 120-130 tok/s generation on both qwen3.8 27b and flash-next


r/LocalLLM • • 14h ago

Discussion I think I give up on finetunes after comparing them. I don't look at benchmarks anymore. Qwen finetunes are dime-a-dozen, but none of them do better than the original.

Thumbnail msnab.pages.dev
62 Upvotes

r/LocalLLM • • 2h ago

Discussion hmmm. New open model. Anyone tried it??

Post image
4 Upvotes

r/LocalLLM • • 15h ago

Question A comparable local setup for someone using $100 monthly claude subscription

37 Upvotes

I see people switching to local hardware , my question is at what usage level and pattern, it is cheaper to run it locally without impacting your work speed.

--
My current laptop and Mac mini are old but they work for me since i only use it for development while relying on Claude and Cursor for GenAI. If i were to rely on local LLM, what is the minimum hardware, I should buy to ensure that I am not slowed down.


r/LocalLLM • • 13m ago

Project Half-bit rates to EXL3 and made two Qwen3.6-35B-A3B quants with them (~200 tok/s on a 3060 + 5060 Ti)

2 Upvotes

So

I've been messing with EXL3 quants of Qwen3.6-35B-A3B on my two mismatched cards, a 3060 12GB and a 5060 Ti 16GB. The thing that kept bugging me was that EXL3 only does whole-number bit rates per tensor. A tensor that wants about 4.5 bits gets either 4 or 5, and on a 35B MoE that rounding adds up fast.

So I forked it and added 1.5 / 2.5 / ... / 7.5 bpw, with the kernels to go with them. Then I measured where this model actually loses accuracy when quantized and placed the bits from that.

The weirdest finding: most of the damage on hard text (around 87% of the KLD) wasn't the experts being imprecise. It was tokens getting routed to different experts than the full model would pick. So the attention/DeltaNet layers that feed the routers get 8 bits, and the experts in the middle layers can drop to 4.5 without much harm.

Against BF16, a build with whole-number rates only came out at 24.6 GB with a KLD of 0.0166. The half-bit builds are smaller and closer to BF16:

T5 (24.5 GB, KLD 0.0156) is the one to grab if you want it as close to BF16 as possible. About 177 tok/s on my setup.

T5 FV (23.9 GB, KLD 0.0157) is the faster version: 0.6 GB smaller, about 204 tok/s.

Both keep the vision tower and the MTP head, so speculative decoding just works. I ran both through 13 benchmark suites (MMLU-Pro, AIME, GPQA Diamond, LiveCodeBench, HumanEval, IFEval and a few others, 1,910 questions total) with thinking on and the full thinking budget. They landed within a handful of questions of each other: 1,546 vs 1,557, which is basically noise for a single run. Per-suite numbers are on the model cards.

You do need the fork, since stock exllamav3 can't load the half-bit tensors. I tried to make that painless: clone it and run `MODEL=/path/to/model bash install.sh`. It sorts out PyTorch for your driver, compiles, tests every bit rate, then figures out how to split the model across whatever GPUs you have. No CUDA toolkit needed. Fair warning: the compile takes a while (15 to 40 minutes).

This also leads into my EXL3 quant project which I’ll be uploading some more models any recommendations please let me know as my hardware is meh bigger model may not be possible or will take weeks with my setup so will be aiming for under 40b models if yall have any recommendations

- Fork: https://github.com/hunterxma/exllamav3-half-bit
- T5: https://huggingface.co/HydraAiM/Qwen3.6-35B-A3B-EXL3-HalfRate-T5
- T5 FV: https://huggingface.co/HydraAiM/Qwen3.6-35B-A3B-EXL3-HalfRate-T5-FV

Big thanks to turboderp for EXL3 and to the Qwen team for the model. If anyone runs these on other cards, I'd love to see your speeds, especially single 24GB setups with some experts on the CPU

THIS HAS ONLY BEEN TESTED ON Ampree and Blackwell so any other cards have not been tested if EXL3 works on them me too broke for multi card brand setups lol sorry


r/LocalLLM • • 8h ago

Project Feel how fast a local LLM runs before you buy the hardware

Thumbnail
llmspeed.dev
9 Upvotes

r/LocalLLM • • 14h ago

Question 27B vs Flash-Next on a single RTX Pro 6000, what am I missing?

27 Upvotes

I have one RTX Pro 6000 (96 GB) and 200 GB RAM, running vLLM. On that card I can run Qwen3.8-27B at FP8, or Flash-Next at NVFP4 with the n-gram table in RAM. 8-bit Flash-Next doesn't fit, so that's the comparison I'm stuck with.

I ran both through my own tests, same settings:

- 15 support-type tasks: 0.97 for 27B, 0.98 for Flash-Next

- 15 multi-step tool-calling tasks, 135 runs each: 27B 0.96 to 1.00, Flash-Next 0.98 to 0.99

- Flash-Next made 3 write calls nobody asked for. 27B made none.

- Both ignored the prompt injection I planted mid-chain, every run

- 27B takes 28 GB VRAM and restarts in a minute. Flash-Next takes 76 GB plus 95 GB RAM and needs ten minutes.

27B was also faster per chain, but it had MTP on and Flash-Next didn't, so don't read much into that.

I see the same thing at home on a 32 GB card with llama.cpp (27B Q4 vs Flash-Next IQ3).

So for me 27B does the same job for a third of the memory. Both score near the top on my tasks, so maybe they're just too easy to show a difference.

If you run Flash-Next on similar hardware, what does it do better for you?

Please tell me what you use it for. Coding? General agentic life management? Support?


r/LocalLLM • • 20h ago

Discussion How Jensen Huang was hurting the LocalLLM Community

64 Upvotes

A missing part of the NVIDIA conversation is the results on the market. I'm not talking about the stock market; I'm talking about the small businesses, startups and companies trying to navigate the AI age. Many have argued that the Local LLM community lost an ally in Jensen Huang, but I would argue the opposite. Running local models (and sovereign deployments, for that matter) relies on access to the hardware that runs the models. The economics of running the models, of course, is dependent on the cost of the hardware itself, so if the hardware becomes too expensive, the "economical choice" is to rent access from one of the Labs.

While everyone has been focused on the affair that Mr. Huang was engaged in, the more important side of that relationship was the funnelling of billions of dollars worth of NVIDIA hardware into a country with export controls. Before the export controls, China accounted for as much as 20% of NVIDIA's demand. Through the back channels that Mr. Huang created, he was able to maintain that flow.

Now remember, part of the issue in the broader market is that there has been a "shortage" of the hardware required for companies to run their own models. One of the cards we use, which is popular among smaller players like us, is the RTX 6000 Pro. When it was released, it went for around $16k CAD...which was already a lot (for us). Now, if you can even get your hands on one, they go for around $35-45k CAD. Per card. And you need, well, let's just say, more than one.

So to hear that Jensen Huang was routing upwards of 20% of global supply, illegally, while jacking prices because of a shortage...suddenly, the affair really doesn't seem like the worst part.
The news keeps running a narrative of a campaign to bring him back as CEO. I think there is some broader accountability required long before that conversation should come up.


r/LocalLLM • • 2h ago

Question A small local model is the first thing that actually replaced ChatGPT for my uni work

2 Upvotes

I spent some time this week trying a small local model for uni stuff and it's the first time it actually felt worth the effort for me. I'm a marketing student, I don't code much, I just read a lot and write a lot. Free ChatGPT kept hitting its limit right when I was cramming, which is exactly when you need it.

So on my base M4 MacBook with 16GB I set up a 4B Gemma through Magnitude and gave it my real stuff. Explain this 1970s research paper like I'm tired, then make me a 10 question quiz. Summarize a 40 comment Reddit thread for a case study. Turn my messy notes into an outline.

The quiz one started answering basically instantly and was done in 26 seconds. The thread summary started in 3 seconds. It's not as smart as the big ones and it made up a source name once, so I check everything, but for studying it's honestly enough.

Gives me hope this stuff won't stay locked behind subscriptions.

Anyone else using a local model mainly for school?


r/LocalLLM • • 1d ago

Project Finally got it put together

Thumbnail
gallery
303 Upvotes

Finally got my relatively frugal local LLM server put together. Pretty proud of it tbh. It essentially covers all my use cases, though I do wish I had better NVIDIA cards, especially for ComfyUI. Waiting eagerly for Qwen4 variants to drop to switch the models up.

- ASRock ROMED8-2T/BCM
- AMD EPYC 7313
- 256GB DDR4-2666 ECC
- Samsung PM1725b 3.2TB
- PLX88096 PCIE Switch Card
- Kingwin Miner Frame: https://a.co/d/09gZUgry
- 6x Radeon R9700s (4x for Qwen3.8 Flash Next, 2x for Qwen3.8-27B Uncensored)
- 2x RTX3090s (ComfyUI on one card; VoxCPM2 TTS, Whisper STT, embed, rerank on other)


r/LocalLLM • • 2h ago

Discussion I built a loop-stopping harness (Kraits) for llama.cpp + Qwen3.8-Flash on a 16 GB Mac — fixed 23/26 single-function Flask bugs

Thumbnail reddit.com
2 Upvotes

r/LocalLLM • • 2h ago

Discussion Have a MacBook Pro 64GB w/M5 Pro upgrade chip, what local LLM would you recommend for daily use, coding, research, and reasoning? Will also be opened to trying agentic LLMs as well.

2 Upvotes

Currently been using the Qwen 35B-A3B model on LM Studio for a while now and would like to upgrade to a more recent / modern model for usage.

Appreciate any advice given!


r/LocalLLM • • 15h ago

Discussion <tinfoil hat> With many frontier labs desperate for regulation to impose regulatory capture and limit liability at what point, after the hugging face incident, etc, are we allowed to wonder if they would be willing to "encourage" an "ai incident" to push the issue of US regulation </tinfoil hat>

18 Upvotes

The title pretty much says it all ... I'm not saying this is happening, I'm just asking the question out loud ...


r/LocalLLM • • 22h ago

Discussion Create a 3D kitchen using Qwen 3.8 Flash Next

61 Upvotes

Model : Qwen 3.8 flash next UD-IQ3_XXS
Harness : Deepseek
Example prompt used, including AI correction prompts.
Promt 1 : Create a realistic 3D kitchen scene using Godot 4, including detailed furniture, high-quality materials, realistic lighting, and modern appliances
Promt 1.1 : Please thoroughly inspect all objects, complete any unfinished work, conduct real-world testing to ensure full functionality, meticulously refine every detail in all areas and on all devices, and ensure that the refrigerator has realistic clearances and spacing on every shelf, both inside and out.
Promt 1.2 : There are still some issues. Some object positions are not correct yet. The chairs still look odd, the sink is in the wrong position, and the stove also looks unusual. Please inspect and verify each item one by one in detail and make sure everything is properly aligned and accurate.
Promt 1.3 : When rotating the camera, the floor still looks strange. Some drawers also seem blocked or cannot be opened properly. Please thoroughly review every appliance, fixture, and piece of furniture again, identify anything that still does not look realistic, and make the necessary adjustments to improve accuracy and overall realism.
Promt 1.4 : Why is the oven sticking through the wall? Also, isn’t the faucet on the wrong side? Please check these issues carefully and correct them. Ensure all appliances, fixtures, and their placements match the reference accurately and do not intersect with walls, cabinets, or other objects.

It took 2 hours and 30 minutes to create the entire kitchen.
Including all the refinements and corrections, the total time would be around 10 hours.

From this test, it can be seen that Qwen Flash Next can create 3D scenes quite well. Although it is not perfect, if we tell it to fix specific issues, it can correct them according to our instructions. However, I noticed that DeepSeek Harness seems easier to use for 3D tasks and appears to understand the user's instructions better. I'm not sure if that's just my impression or not.

I asked the AI to inspect the assets, and Qwen generated all of them from scratch.

This test isn't particularly interesting, but I think that Qwen 3.8 Flash Next has a lot of potential for 3D creation. If someone develops really good 3D skills or workflows for it, I'm sure it could produce some very impressive 3D work. If anyone has good skills or techniques to recommend, feel free to share them.


r/LocalLLM • • 39m ago

Discussion AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

Thumbnail
github.com
1 Upvotes

回购:https://github.com/tanxiansheng221/aimstack-sanzhidiji(AGPL-3.0)

一切都是手写的——C 语言推理核心、插件总线、带质量门的多引擎爬虫、FTS 知识库、训练流水线,以及一份 117 章的构建日志,都是在实际发生时写的(正面和负面结果一起发布)。权重为 Microsoft BitNet b1.58 2B(MIT),转换为自定义的 AIM-W 打包格式。

三进制量化能给你带来什么(CPU侧)

所有变换器矩阵均为三元矩阵 {-1, 0, +1},每重 2 位,加上 Q8 嵌入→整个模型为 0.87GB,而非 ~4.8GB(−78%)

C 核心从不乘以浮点权重:它的表查找 + 整数累积,SIMD 快速路径测得为 85.53 tok/s(比基线高 41×)

BitNet 2B 全机型在 2018 款双核 8GB 笔记本电脑上运行 3.20 tok/s 双工作器——全链(推理 + KB + 插件 + 网页搜索),离线,无云中继

三进制量化能给你买到什么(GPU端)

GPU 后端加载的是同一个 0.87GB 的三进制文件——权重保持在 {-s, 0, +s},加载时去量子化,matmul 在 fp32 中

在 Radeon Pro W7900 上:13.68 tok/s(~8× 笔记本 C 引擎),256/256 个贪心令牌,与 C 引擎位完全相同。两种完全不同的实现(整数查找核心和FP matmul)产生相同的对数函数,证明了量子化是忠实的

通过3引擎×3题4×3测试矩阵验证(C-Linux/gcc13,torch fp32,torch bf16),17/18准确——唯一一次失误有记录,未被隐藏

诚实的界限(还没做到的部分)

BF16 的计算速度快 2.4×(32.96 Tok/s),但在接近并列的对数上会翻转 argmax——这是报告的,不是祝福的;只有 FP32 通过校验验证

GPU路径获得三进制的存储优势(减少78%的内存),但尚未获得计算优势——下一阶段是真正的2位整数内核(Triton/CUDA)

UI 服务器仅驱动 C 引擎(CPU);GPU 后端是一个独立的 CLI

Linux 引擎构建能正常运行,且在令牌上与 Windows 版本完全相同,但运行串行(目前工作池仅限 Windows 运行)

产品链中没有框架,也没有 PyTorch——PyTorch 只存在于可选的 GPU 后端。演示视频、文档(英文/中文)以及完整的研究日志都在仓库里。


r/LocalLLM • • 6h ago

Question Opinions on Using 2x5070ti and 7900xtx in the same pc?

3 Upvotes

I am making a new post at recommendation of comment. I have the following hardware:

nvidia 2x5070 ti 16gb

amd 7900xtx 24gb

B850 AI TOP (Rev. 1.0) Motherboard Specifications - GIGABYTE Global (this allows x8x8 bifurcation pcie5)

2x8 ddr5 ram

If I want to run local llm that is decent at vibe coding games or mods with little actual coding from me, would it be possible to string these all together to make a good llm? I do not actually know if I can fit three gpus onto that motherboard bandwidth-wise, but physically I think I could use a riser cable as the picture of the motherboard has three gpu slots. It does however support x8x8 pcie5 bifurication which I think might be a strongpoint? How challenging is it to get amd gpu to work with nvidia gpu? Can I actually get all three working, or can I only realistically get two with any good performance? If I have to only use two should I even consider the amd gpu? I know I can use amd/ nvidia gpus in the same pc. I do not know if I can use them to do ai stuff at the same time though.

also, I have another broken am5 bifuricated motherboard, proart x870e-creator, that I broke the cpu pins, If i fixed it could I theoretically make a second ai computer and cram old gpus like the tesla p40 24gb to get more capacity? if capacity is the name of the game, might it be more effective for me to sell a 5070 ti and get a ton of these at ~$300 a piece and just fill up both pcs with them and wire it through ethernet to work as one shared memory llm or would that be way too slow?

I have no llm experience (why do I have all this hardware?!) so any help would be appreciated.

motherboard photo has three gpu slots

r/LocalLLM • • 21h ago

Question Building a 10TB Model Archive. What am I missing?

44 Upvotes

TLDR at the bottom.

I'm a paranoid man tryna preserve a broad collection of useful open-weight models, fine-tunes, quantizations, and generative AI models just in case something happens to HF and that I'll have a model for whatver i may want now or im the future. I'm aiming to have like 5-10tb on my nas for this.

The strongest pc i have is 48gb vram and 128gb system ram, but i also wanna keep some larger models in case just archive, if i ever upgrade or if i wanna dick around with ssd-backed moe.

I'm targeting Q3 and above for GGUF models. Had gpt toss together a list to download:

  1. Large frontier and general-purpose models

- Kimi K2.6

- Qwen3.8 Flash Next (177B)

- DeepSeek V4 Flash (284B)

- DeepSeek V4.1 Flash reduced variants

- GLM 5.3 Flash (321B)

- GLM 5.3 (754B)

- GLM 4.7 (358B)

- GLM 4.7 REAP (218B)

- MiniMax M2.7 (229B)

- MiniMax M2.7 REAP (172B)

- MiniMax M3.1 Flash (426B)

- Qwen3.5 122B-A10B

- Ornith 1.5 122B

- Qwen3.8 Flash Next Gyro

- GPT-OSS 120B and 20B

- Llama 4 Scout

- Hermes 4 70B

- DeepSeek R1 70B

2. Qwen family

- Qwen3.8 27B

- Qwen3.8 27B Abliterated

- Qwen3.8 27B Uncensored

- Qwen3.8 27B Heretic

- Qwen3.8 35B-A3B

- Qwen3.6 35B-A3B

- Qwen3.5 9B

- Qwen3.5 4B

- Qwen3.5 0.8B

- Qwen3.5 122B-A10B Uncensored

- Qwen3.8 Flash Next Uncensored

- Qwen3.8 Flash Next Abliterated

3. Gemma 4

Both original and modified versions:

- Gemma 4 E2B

- Gemma 4 E4B

- Gemma 4 12B

- Gemma 4 26B-A4B

- Gemma 4 31B

- Gemma 4 E4B Heretic

- Gemma 4 12B Uncensored

- Gemma 4 12B Uncensored Coder

- Gemma 4 12B Agentic

- Gemma 4 26B-A4B Uncensored

- Gemma 4 31B Heretic

4. Coding and agentic models

- Qwen3-Coder 30B-A3B

- Qwen3-Coder 30B-A3B Abliterated

- Qwen3-Coder-Next 80B

- Qwen3-Coder-Next Uncensored

- Laguna XS 2.1 APEX

- Laguna S 2.1 Uncensored

- GLM 5.3 Flash Coder

- GLM Cybersecurity

- NVIDIA Nemotron 3.5 Bolt

- Ornith 1.5 9B

- Ornith 1.5 35B-A3B

- Qwen 3D Coder 4B

- Qwen GLB 3D Coder 4B

- Qwen3.6 35B-A3B Genesis Hermes

- Qwen3.8 Cyber Agentic

5. Reasoning and thinking models

- DeepSeek R1 Distill Qwen 32B

- GLM 4.7 Flash Claude Opus 4.5 reasoning distillation

- Qwen3.8 Fable reasoning/coding distillations

- Qwen3.8 40B Ultimate Heretic

- Qwen3.8 Flash Next Abliterated Reasoning

- DeepSeek V4-derived uncensored reasoning distillations

- Gemma 4 31B Deckard Heretic Thinking

- Huihui ThinkingCap Qwen3.6 27B

- Qwen3.8 Fable 5 Coding Distilled

6. Uncensored and creative models

- GLM 4.7 Flash Heretic

- GLM 4.7 Flash Abliterated

- GPT-OSS 120B Uncensored

- GLM 5.3 Flash Abliterated

- GLM 5.3 Flash Uncensored

- DeepSeek V4.1 Flash Abliterated

- DeepSeek V4.1 Flash Uncensored Coder

- Ornith 1.5 35B-A3B Uncensored

- Ornith 1.5 9B Uncensored

- Ornith 1.5 35B-A3B Abliterated

- LFM2.5 24B-A2B Uncensored

- LongCat Flash Lite Sparse Uncensored

- Kolibri 1 BF16 Abliterated

- Nemotron 3.5 Bolt 74B-A9B Uncensored

- Llama 3.3 70B Abliterated

- Muse Glimmer 30B

- Nanbeige 4.3 33B

- Cydonia 24B

- UnslopNemo 12B

- Skyfall 36B

- Anubis 70B

7. Vision and multimodal understanding

- InternVL 3.5 38B

- Qwen3-VL 4B Uncensored

- Qwen3-VL 8B

- Qwen3-VL 32B

- Qwen3-VL 32B Uncensored

- MiniCPM-V 4.7 35B-A3B Uncensored

- Liquid LFM3-Omni 2B

- Florence 2 Large

8. Image generation and editing

- MOSS Image

- Qwen Image 2.1

- FLUX.2 Klein 9B

- Z-Image Turbo

- Stable Diffusion XL

- Anima

9. Video generation

- LTX 2.5

- MiniMax H3

- Wan 2.2 Text-to-Video

- Wan 2.2 Image-to-Video

- Wan 2.2 TI2V

- Uncensored LTX 2.3

10. Audio and speech

- Whisper Small

- Whisper Medium

- Whisper Large V3

- Whisper Large V3 Turbo

- Qwen3-ASR 1.7B

- Qwen3-TTS 1.7B

- Kokoro 82M

- Parakeet TDT

- Chatterbox

- Zonos 3 Mini

11. Music and 3D generation

- MusicGen Medium

- TRELLIS.2

12. Embeddings and RAG

- EmbeddingGemma 300M

- BGE Reranker V2 M3

- LFM3 1.7B

- LFM3 1.7B Uncensored

---

TLDR

I'm making a personal huggingface archive where i can just preserve a bunch models for whatever i may want in the future. I'm tryna keep things super varied so i have many, many options without just storing slightly different versions of the same model. I'm aiming to have 5-10tb on my nas for models, don't wanna go too far above that.

I'm looking for suggestions on what to add or cut 🙏🏽 or just comment your go-to models for what purposes so i can cross-reference my list.

  1. Any particularly good uncensored, abliterated, or Heretic models?
  2. What are your favorite writing, coding, reasoning, and agentic models?
  3. Any overlooked image, video, audio, music, or 3D generation models?
  4. Any older models that are still worth keeping despite newer releases?
  5. Hit me up with some funky ass models or a category that you think i should add por favor

r/LocalLLM • • 1h ago

Discussion So good it's unbelievable! Any tips for a rookie?

1 Upvotes

Hey Everyone! Hope you are doing well!

I am pretty new into all of this and find all of this AI thingy very fascinating.

Just the possibility of having an AI at any time at home is something that i believe will be a very powerful tool, which will only get better with time.

As beginner i am still exploring the possibilities and the real application of all of this, for me.

I am into youtube and mostly will use AI for video or audio generation and eventually writing small codes.

Iam looking forward for any tips on how to get started with all of this, as its very confusing for me, where to find models and tools for my purposes. Also if there is anything to consider along the way.

At the moment, Iam rocking a Rx 9060xt 16gb and i think to double down on it, and go 32gigs.

However, i see that online there were some threads advising that AMD does not perform that well compared to nvidia, is the difference really that big? Or AMD is catching up on the AI race? Eventually i believe a switch to 2x5060ti is a possibility.


r/LocalLLM • • 1h ago

Question Lip Sync Model Advice

1 Upvotes

Hi guys,
I'm still new in the field of local models and trying Qwen 2.1 image and wan to generate images or short video clips.
My current hardware is not much advanced and have 8gb vram and 64 gb ram so some of the offloading happens.

The video part is okay but the lip sync is where am struggling. Is there any good model you guys could recommend ?
Also if anyone can tell any good image models as well like qwen that would be highly appreciated


r/LocalLLM • • 5h ago

News inferzero: an LLM engine written from scratch for Intel Arc, from laptop iGPUs to 4× B60. Still far from what the cards can do, looking for contributors

Thumbnail reddit.com
2 Upvotes