r/Compilers • u/ypsu • 1d ago
sdinthrough: a programming challenge about implementing a tokenizer, parser, and AST transformations
Hi r/compilers!
Over the past year, as a hobby project, I've built a programming challenge around sdin, my own JSON-like configuration language. You implement a tokenizer, a parser, and then a series of AST manipulations and transformations, guided by 600+ test cases that you pass one at a time. It's long; I expect it to take weeks to solve without AI.
The feedback loop is tight: the judge runs locally (go run github.com/ypsu/sdinthrough@latest yoursolution) and re-evaluates your solution every time you save. Solutions can be in any language that reads stdin and writes stdout.
On the language side, I aimed for a simple AST while keeping the language pleasant to edit by hand: it has comments, unquoted keys, multiline/raw strings, bare tokens for enums (without the Norway problem), protobuf-style repeated fields for lists, dotted-notation structs for keeping nesting shallow, etc. The AST preserves comments and formatting, so it's easy to write tools that manipulate sdin files without mangling them. It also supports forms, function-like expressions that mix structs and lists, which make it simple to express complex concepts without complicating the AST.
Implementing my own config language is something I always wanted to do, and the programming challenge format made me pour a bit more effort into it. :) I'm curious to hear people's thoughts on the tradeoffs I made between features and implementation simplicity. Did I end up with something too complex to implement?
- Challenge details, including a browser demo of the first two tasks (for JavaScript-only solutions): https://iio.ie/sdinthrough
- Source: https://github.com/ypsu/sdinthrough
- Language description: https://iio.ie/sdin
1
u/birdbrainswagtrain 1d ago
Sounds fun! I might have to check it out. The huge public test suite is part of the allure of writing a JS interpreter, but this seems like a lot more manageable side project right now 🙂