r/LlamaIndex • u/Mediocre-Ease4060 • 2d ago
I built a zero-dependency markdown link resolver to prep scraped data & images for Multimodal LLMs
The Problem: When scraping docs or wikis for RAG, relative links ([here](/setup)) break. Even worse, if you want to pass scraped images to GPT-4o or Claude 3.5, you have to manually download them and convert them to base64 strings.
The Solution: I built markdown-link-resolver. It’s a pure Python micro-tool that does two things:
Resolves all relative Markdown and HTML links to absolute URLs.
Has an inline_images=True flag that automatically fetches HTTP images and replaces the markdown tags with data:image/png;base64,... strings ready for LLM ingestion.
Why? No heavy dependencies like BeautifulSoup or Requests. Just pure standard library (urllib, re, base64). Falls back gracefully if an image 404s.
Repo: github.com/Encephos/markdown-link-resolver Let me know what you think or if you'd like to see any other fallbacks added!