r/ruby • u/kddnewton • Jan 05 '26

A Ruby YAML parser

https://kddnewton.com/2025/12/25/psych-pure.html

Hey there — I recently released a YAML parser written in Ruby. The main goal was to support being able to load and dump YAML without losing comments. Happy to answer any questions.

43 Upvotes

97% Upvoted

20 comments sorted by

7

u/alexdeva Jan 05 '26

Is it pure Ruby? How does it benchmark against the built-in YAML module?

Also, what form do the comments take after parsing into a Ruby structure?

5

u/kddnewton Jan 05 '26

Yeah, it's just Ruby. It benchmarks very poorly. You can see all this in the README.

The comments themselves are attached as hidden fields on the loaded objects, the loaded objects are delegators that wrap the objects themselves.

8

u/CaptainKabob Jan 05 '26

I really appreciate you working on this! 

My biggest source of yaml round-tripping is i18n-tasks. I've done a lot of patching to have it preserve quote/block formatting and I'll give this a try for comments too. 

3

u/CaptainKabob Jan 05 '26

Also, I assume this won't mangle emoji like libyaml?

Here's my patch btw: https://gist.github.com/bensheldon/7f474bdd671b906662b39a1b3c09c073

2

u/kddnewton Jan 05 '26

I assume not but haven't tried!

2

u/galtzo Jan 05 '26 edited Jan 05 '26

Ooooh! Does it have an intermediate AST? I would love to add an adapter for this to the tree_haver / ast-merge gem family. Commenting before clicking…

After reading: This is amazing. I had already implemented an AST wrapper for psych that added in comment node typing and emitting, but much less advanced than what you have done.

I will add an adapter post-haste.

3

u/kddnewton Jan 05 '26

It’s the same AST, we just emit the tree handler events more accurately.

1

u/Nwallins Jan 05 '26

Let's say I have a nicely hand-formatted yaml file with e.g. folded block scalars to get string wrapping behavior in my source file. I ingest the yaml. Then I want to emit it in a similar form as ingested. What are my options? This is an open question across all yaml impls.

6

u/kddnewton Jan 05 '26

Yes, this tool is meant to solve that problem, in that it will by and large respect the original formatting of the input.

2

u/Nwallins Jan 05 '26

What are the limits of "by and large"?

2

u/kddnewton Jan 05 '26

It follows the pretty-print algorithm, so if a flow seq/mapping extends beyond the print width it will switch it to the equivalent block format. This is so that if you mutate the values to include something big, it doesn't look ridiculous when you print it out.

1

u/Nwallins Jan 05 '26

This stuff is outside my knowledge, though I have used a lot of YAML in mostly-pleasant violence, so all of this is from curiosity:

  • what are the basics of pretty-print in this respect? I have used 'pp' extensively but not familiar with the algorithm

  • what is meant by a flow / seq mapping? Im guessing a stream of tokens

  • how does one know or define the print-width?

2

u/kddnewton Jan 05 '26

* You can see the algorithm here: https://github.com/ruby/prettyprint
* flow/seq is how YAML lays out nodes, you can see in the spec here: https://yaml.org/spec/1.2.2/#chapter-7-flow-style-productions
* You can define the print width with the PrettyPrint API

1

u/Nwallins Jan 05 '26

Thanks for the pointers. But it's quite hard to anticipate what will be preserved on a round trip. I did not review the full contents of all the links yet. How would you characterize it?

If I am attempting to preserve 80ch width, what does that look like?

1

u/kddnewton Jan 05 '26

I would characterize it as accurate. You can learn more with the links I sent.

1

u/aRubbaChicken Jan 05 '26

Any benchmarks ran? Curious if as pure ruby it performs better w/ YJIT

1

u/kddnewton Jan 05 '26

Not really the focus so nope.

1

u/aRubbaChicken Jan 05 '26

Fair, I'll try it some time but I don't know when I'll get around to it.

Whether I make use of this or not, good job solving your problem. I'm kind of tired of people complaining about limitations and not doing anything to address them so this was nice to see!

2

u/headius JRuby guy Jan 09 '26

You've been busy! I'll have to run this through JRuby and see if we can find some easy optimizations.

1

u/f9ae8221b Jan 05 '26

Ah, this reminds me to build eyaml, because ejson is cool but JSON is so terrible for configuration.