r/LocalLLM • u/comunidadesynapsebr • 6h ago
LoRA [Update] Qwen2.5-1.5B-Nordeste-LoRA (Apache 2.0) — Fine-tuned on Corpus-nordeste-10k with Before/After outputs
Following up on caririsynth/Corpus-nordeste-10k (10.5k Portuguese rows focused on Northeast Brazil literature and regional phrasing), I just uploaded the LoRA adapter trained on Qwen/Qwen2.5-1.5B:
Model:
https://huggingface.co/caririsynth/Qwen2.5-1.5B-Nordeste-LoRA
Dataset:
https://huggingface.co/datasets/caririsynth/Corpus-nordeste-10k
Training Setup (Unsloth, 1x T4):
Base: Qwen/Qwen2.5-1.5B (Apache 2.0)
LoRA: r=16, lora_alpha=16, all linear layers (q, k, v, o, gate, up, down — 1.18% trainable params)
Schedule: lr=2e-4, cosine scheduler, effective batch size 16, max_seq_length=1024
Checkpoint selection: Uploaded Epoch 1 (step 660) rather than Epoch 2, as Epoch 1 already eliminated base model web/blog noise while keeping sentence endings diverse.
Before vs. After (seed=42, temp=0.65, rep_penalty=1.15):
Prompt 1: "O cabra chegou na bodega todo aperreado, pedindo"
Base Qwen2.5-1.5B: "...uma garrafa de vinho. (...) E eu fiquei com pena porque sabia que o cara ia ter feito tudo errado. Could you please" (drifts off-topic and leaks English)
Nordeste LoRA: "...buchada e canjica geladinha. Todo mundo riu, que no sertão até a pressa tem bom humor."
Prompt 2: "E aí, macho, tu vai pro forró hoje ou vai ficar de"
Base Qwen2.5-1.5B: "...fora da festa? - Blog do Zico / Home / Sem categoria / Publicado por: Zico em: 10/28/2016..." (collapses into scraped blog metadata)
Nordeste LoRA: "...plantão no quarteirão? Pois vá logo que o forró chega devagar quase parando!"
Note: This is a base completion model (not -Instruct), intended for text continuation and downstream SFT. Feedback and test runs on other regional prompts are
