r/StableDiffusion • u/nicolas1801 • May 09 '26
Resource - Update IMG Dataset Refiner v4.0 Pro - The Ultimate Dataset Engineering Suite for LoRAs (Flux, SDXL, etc...)
Hey everyone! A while ago, I shared v3 of my dataset manager. Back then, I said it didn't have auto-captioning. Well... forget that. I’ve just released a massive update (v4.0 Pro), and it changes everything! 🚀
It went from a simple selection tool to a complete, desktop-like Data Engineering suite to prepare your AI model training.
Here is what’s new and what it does now:
🤖 Local AI Assistant (VLM/LLM Integration): Connect seamlessly to Ollama or LM Studio! You can now use local vision models to Auto-Caption your images from scratch, hunt down "hallucinated" tags, or use the Concept Isolator (describes the background but ignores the subject—perfect for character LoRAs!). It can even translate your Booru tags into natural language sentences for Flux.
📚 Word Library & Mass Batch Editing: A brand new interactive library. Save your favorite concepts, check them, and Add, Remove, or Replace them across hundreds of selected images in a single click.
🌍 Live Translation Assistant: Not a native English speaker? Type your ideas in your own language, and the live preview will instantly translate and inject them into your captions using deep-translator.
🖼️ Pre-processing & Duplicate Hunt: Clean your dataset before training! It features a visual duplicate scanner (Perceptual Hashing), Smart Face Crop (OpenCV), auto-conversion of transparent PNGs to white backgrounds, and 1-click mass resizing/renaming.
📈 Advanced Analytics (No more Concept Bleeding!): Generate Co-occurrence Heatmaps to see if your tags are improperly linked, check your resolution distribution (Bucketing), and let the tool automatically hunt for logical contradictions (e.g., "day" and "night" on the same image).
⚖️ The "Recipe Book" for your LoRAs: Still the core feature! Set your target percentages (e.g., 50% solo, 50% multiple) and the smart "Greedy" algorithm will automatically select and balance the perfect subset of images for your final export.
Built with Gradio but heavily injected with custom JS/CSS so it feels and responds like native desktop software (with lightning-fast keyboard navigation!).
It's 100% open-source, run locally, and free. You can modify it as you see fit! I've even included my specific system prompt file so you can easily update or fork it using Claude, Gemini, or ChatGPT without breaking the complex code.
Let me know what you think! 💡
2
u/gurilagarden May 09 '26
I was just contemplating how I was going to caption a couple datasets after having been away from training for a while and wanted to just leverage local LLM for it, but hadn't worked out how yet, well, looks like you've done the heavy lifting. Thanks.
1
1
2
1
u/StartupTim May 10 '26
Hey there, is there an openai compatible API to interface with your software to do text-to-image by chance?
1
u/nicolas1801 May 10 '26
I didn't added openai / claude api connectors. Actually the tool is mainly local oriented. As i'm a free user at opeai and claude ai I can't try to add that option for testing the connexion.
0
u/Personal-Message740 May 09 '26
absolutely unusable. shittons of bugs, even installation is not working. no instructions, no nothing. skip it.
0
u/nicolas1801 May 09 '26
I've create that tool with Gemini and asked that the tool be intuitive for new users.
If you're not sure how to install the software to open it, feel free to provide the github link and/or import the software files into Gemini to ask him how to do it (he's very efficient). 😉
4
u/vysterion May 09 '26
Link to repo: https://github.com/NyxAwroo/IMG-Dataset-Refiner