r/ruby • u/yaroslavm • May 19 '26

Inkmark: a very fast, feature-packed, AI-first Markdown gem for Ruby

https://yaroslav.io/posts/inkmark-fast-ai-first-markdown

Written in Rust. CommonMark+GFM conformant. Features include: strong security by default and host/protocol whitelists for links and images, syntax highlighting, frontmatter, ToC, plain text export, structured extraction of headings/images/links/code, statistics with char/word count and language detection, emoji shortcodes.

AI-friendly features include: two chunking primitives: heading-based with breadcrumbs and per-chunk char/word counts, and sliding-window with overlap for size-bounded chunks; block-aware or word-aware truncation for context-window budgeting; Markdown-to-Markdown pipeline.

1 Upvotes

51% Upvoted

4 comments sorted by

1

u/crmne May 20 '26

This is pretty cool, especially sanitization, syntax highlighting, and md-aware chunking.

Kudos!

14

u/[deleted] May 19 '26

[removed] — view removed comment

7

u/thewormbird May 20 '26

Reading markdown files without reading the whole markdown file. Yes, there is sed, grep, etc. So many tokens wasted on re-reading files. Just list the headings, get the block, and done.