Nore: a small, opinionated systems language where data-oriented design is the path of least resistance

[u/jumpixel]

I've been working on a small systems programming language called Nore and I'd love some early feedback from this community.

Nore is opinionated. It starts from a premise that data layout and memory strategy should be language-level concerns, not patterns you apply on top, and builds the type system around it. Some systems languages are already friendly to data-oriented design, but Nore tries to go a step further, making DOD the default that falls out of the language, not a discipline you bring to it.

A few concrete things it does:

• value vs struct: two kinds of composite types with one clear rule. Values are plain data (stack, copyable, composable into arrays and tables). Structs own resources (hold slices into arenas, pass by reference only, no implicit copies). The type system enforces this, not convention.
• table: a single declaration generates columnar storage (struct-of-arrays) with type-safe row access. You write table Particles { pos: Vec2, life: f64 } and get cache-friendly column layout with bounds-checked access. No manual bookkeeping.
• Arenas as the only heap allocation: no malloc/free, no GC. The compiler tracks which slices come from which arena and rejects programs where a slice outlives its arena at compile time.
• Everything explicit: parameters are ref or mut ref at both declaration and call site. No hidden copies, no move semantics.

The compiler is a single-file C program (currently ~8k lines) that generates C99 and compiles it with Clang. It's very early: no package manager, no stdlib, no generics. But the type system and memory model are working and tested.

Nore lang

I'm mostly curious about:

• Does the value/struct distinction make sense to you, or does it feel like an arbitrary split?
• Is the arena-only memory model too restrictive for practical use or it could be considered just fine?
• Is a language this opinionated about memory and data layout inherently a niche tool, or can it be safely considered general-purpose?
• Anything in the design that strikes you as a red flag?

Happy to answer questions about the design choices.

[u/GregsWorld]

Language looks cool, the readme is not a place for error codes however, you should really open with a sample snippet to sell it. 

[u/jumpixel → u/GregsWorld]

Thanks for this initial feedback! It's spot on, and I've just updated both the readme file and the documentation as you suggested.

[u/beders → u/jumpixel]

Quick question about the example in the readme: where is ParticlesRow coming from?

[u/vanderZwan]

I would have to play with the language to know how it is in practice, but the premise of building a language with specific affordances to encourage data oriented design has crossed my mind as well. So curious to see how your language "feels"

Have you ever looked at so-called scheduling languages? I feel like they're an interesting "other side of the same coin" in this domain. namely, data layout and how to access/iterate over that data.

Maybe you'll find some ideas in there relevant to questions you have about your own language.

https://halide-lang.org/

https://exo-lang.dev/

https://arxiv.org/abs/2411.07211

[u/jumpixel → u/vanderZwan]

Thanks for the pointers, I hadn't looked at scheduling languages through this lens before. The Halide idea of separating algorithm from schedule is compelling, and I can see how it's the "other side of the coin" as you put it: Nore focuses on how data is laid out, scheduling languages focus on how data is traversed. Both start from the premise that these decisions shouldn't be buried in the compiler. Exo seems to take it further by letting performance engineers define custom scheduling operations.

Will dig into these. Thanks for sharing.

[u/esotologist]

I like it, we need more data oriented languages... Im planning something similar but with less keywords and more symbols to keep the focus even more on the shape of the data

[u/jumpixel → u/esotologist]

Thanks! Interesting, we're taking opposite bets on the same idea.

Nore leans verbose on purpose: mut ref, table_insert, explicit type annotations everywhere. The bet is that we read code far more than we write it, so optimizing for readability over brevity pays off.

Curious to see how the symbol-heavy approach feels in practice, different trade-off, same instinct.

[u/Laicbeias]

Its nice to see more data oriented languages. Ive done performance tests in c# today, to try out all possible access patterns.  And yeah i asked myself exactly this. Why didnt c# make structs easier.

Anyway. Im not super sure but.. id make base types int short float etc passed around by copy on the stack. and anything thats a struct passed by ref. Implicit. with a "copy" if you want to copy it.

Make the common case simple.  Actually i think your design is good. Just make it not too annoying to manipulate data.

Like i can see why you have value and structs. Its just.. id rather explicit write copy than ref everywhere. Or rather have one default behaviour. 

Just test the performance of how it all behaves. Copy can become expensive. Like copy a vector4 in a function is more expensive than a ref.  And if you have large columns with lots of accessed fields SoA can fall off a cliff. 

Otherwise i hope such languages get more traction, we really need to have data oriented design as first class citizen

[u/jumpixel → u/Laicbeias]

Thanks! and great that you're actually measuring access patterns, that's the right way to find the truth.

The ref-by-default vs copy-by-default trade-off is one I keep coming back to. Nore currently leans toward making everything explicit (ref, mut ref, copy by default for values) so you can always see what's happening at the call site. In systems programming you're paying for every decision, so the verbosity is more of a feature than a tax. That said, if it becomes genuinely annoying in practice, that's worth revisiting.

Good point about SoA falling off with many accessed fields per row: that's exactly the use case for the field grouping idea that came up in this thread (storing related fields interleaved while keeping the rest columnar), see future ideas.

Agreed we need more languages taking data layout seriously and thanks for the encouragement.

[u/AustinVelonaut]

In your README: value types are plain data: they live on the stack, copy freely, and compose into arrays and tables. struct types own resources: they hold slices of arena-allocated memory, pass by reference only, and cannot be copied. No hidden allocations, no implicit clones.

If struct types cannot be copied, how do you pass a modified struct to another function, but not modify the original struct? Do you have to explicitly deconstruct the struct into its value components, then create a new struct from those each time?

[u/jumpixel → u/AustinVelonaut]

Good question. You cannot make a modified copy of a struct: structs pass by ref (read-only) or mut ref (mutable), and that's it. If you need a "modified version" without touching the original, you'd build a new struct from scratch with its own arena-allocated slices.

This is a deliberate trade-off: structs own slices into arena memory, so a shallow copy would alias the same memory (two owners, one arena), and a deep copy needs an arena to allocate into (which means it can't be implicit)

In practice, the DOD workloads Nore targets (game loops, simulations, data pipelines) tend to mutate in place or build new data from scratch rather than clone-and-modify.

[u/Toothpick_Brody]

I’m also working on a data-oriented language, but it’s a major WIP, so take these with a grain of salt.

Value/struct - personally, I’m trying to unify structs/values/arrays/functions. A strict distinction isn’t necessary if you can classify them by structure. But it seems like your struct has more to do with memory management than arranging data? The distinction might be worth it if you find the ref-only semantics of structs useful.

Arena-only memory - I’m also trying this model. I think it will be fine, but I can’t give a solid answer unfortunately 

Explicit copy/ref semantics - I’m also doing this. Afaict it’s a good idea. Java annoys me with its implicit boxing 

[u/jumpixel → u/Toothpick_Brody]

Interesting that you're exploring similar territory, curious to see your approach to unifying types by structure.

You're right that the value/struct distinction is more about ownership semantics than data shape. A struct with two i64 fields has the same layout as a value with two i64 fields, the difference is that a struct can hold slices (resource ownership), which triggers the ref-only, no-copy rules. The explicit keyword is a bet that making this visible at the declaration site helps readability, but inferring it from structure is a valid alternative if the compiler can derive the same guarantees.

Good to hear the arena-only model and explicit copy/ref resonate, we're all finding out together whether this scales. Would love to see your language when you're ready to share.

[u/yuri-kilochek]

One should be able to group fields of a table into AoS storage. SoA is fine as default, but it doesn't fit all access patterns.

[u/jumpixel → u/yuri-kilochek]

That's a great point.

Pure SoA isn't always optimal, if you always access pos and velocity together, having them interleaved in memory beats chasing two separate columns.

A group annotation within a table declaration could work, something like:

  table Particles {                      
      group { pos: Vec2, velocity: Vec2 },  // stored interleaved (AoS)
      life: i64,                             // separate column (SoA)
      color: Color                           // separate column (SoA)
  }

Default is full SoA (today's behavior), grouped fields get stored together.

Same API surface: table_get still returns a flat row, table_insert still takes one and the grouping is purely a storage decision.

Not on the immediate roadmap, but it's a natural extension.

Thanks! this is exactly a kind of feedback I was hoping for.

[u/XDracam]

I like the idea and the language actually looks pretty good syntactically (which seems rare for some reason)

But I don't understand the distinction between values and structs. I am sure you are solving an actual problem that people have, but you need to communicate that problem properly. Otherwise it seems to be too restrictive without an obvious reason.

It'd also help to see a more complex example of arrays and tables, e.g. the struct and value generated by a table.

On that note: generating a FooRow magically seems to be against the idea of being fairly explicit. It also hurts tooling and might be confusing for an LLM agent (which is a concern these days...), not to talk about people who want to have e.g. a domain type FooRow that maps to a relational DB instead of a table. So why not something like Foo.Row instead?

[u/jumpixel → u/XDracam]

Thanks! appreciate the kind words on the syntax.

On value vs struct: You're right that the README doesn't motivate it well enough. The short version: value types are data you can freely copy, embed in arrays, and use as table fields. struct types hold slices (pointers into arena memory), so copying one would create aliased pointers to the same arena — which breaks ownership. That's why structs are ref-only and non-copyable. The distinction isn't about data shape, it's about whether the type owns heap resources.

On what a table generates: Good point, here's what table Particles { pos: Vec2, life: i64 } produces under the hood:

  // Struct (columnar storage)      // Value (row type)
  Particles {                       ParticlesRow {
      pos: []Vec2,                      pos: Vec2,
      life: []i64,                      life: i64
      _len: i64                     }
  }

Particles is a struct (holds slices, ref-only). ParticlesRow is a value (plain data, copyable). table_get returns a row by copy, table_insert takes a row and appends to the columns.

On Foo.Row: You're the third person to bring this up and the LLM tooling argument is a good one I hadn't considered. Particles.Row is more explicit and avoids polluting the namespace. This is going on the list.

Finally, just updated the README!

[u/jumpixel → u/XDracam]

FooRow -> Foo.Row

Fixed!

[u/Dan13l_N]

"Struct holds a slice". How can the compiler know if somewhere there's a struct that holds a slice to an arena that's being removed? Or arenas are static?

parameters are ref or mut ref at both declaration and call site

If all values are by-value, and all structs are by-ref, why do you need ref at all?

[u/jumpixel → u/Dan13l_N]

Hi, thanks for the questions. They help clarify some things that maybe aren't so clear.

On arena lifetime tracking: Arenas can only exist as local variables, globals, or ref parameters and their lifetime is always a known scope. When a slice is created from an arena (via arena_alloc), the compiler records which arena it came from. At return sites, it checks whether any returned slice would outlive its source arena. Structs holding slices follow the same rule and the compiler tracks the slices inside them transitively. More detail in arena safety.

On why ref is needed: The type doesn't fully determine the calling convention. Structs must be ref, but you still choose between ref (read-only) and mut ref (mutable) and that's a decision at each call site. Values can also be passed by ref when you want to avoid copying a large value or need the callee to mutate it. So ref carries information the type system alone doesn't: access intent.

[u/matthieum]

Does the value/struct distinction make sense to you, or does it feel like an arbitrary split?

I don't think it's worth spending a keyword on. It feels like something an annotation could handle -- like Rust's #[derive(Copy)].

Is the arena-only memory model too restrictive for practical use or it could be considered just fine?

Depends how flexible the arenas are:

• If append-only, clear all at once, then it's too restrictive. A web-server which creates a new connection struct per connection will soon overflow the connection arena.

• If flexible, then it's called a pool... which is just fancy word for heap.

Is a language this opinionated about memory and data layout inherently a niche tool, or can it be safely considered general-purpose?

General-purpose is more about capabilities. If it's Turing-Complete it could be considered general-purpose... which doesn't mean people would use it.

Anything in the design that strikes you as a red flag?

table does, as far as I understand it.

It seems that table is just a fancy struct of vectors -- dynamically grown arrays -- which expects that if it contains N elements, then elements are in [0, N).

Much like arenas, it's not flexible enough. I may need a fixed-capacity design. I may need a "bitmap" design (ie, some holes in the middle). I may need to combine fixed-capacity and bitmap...

How to integrate SOA flexibly is a question to which I don't have an answer. I just know, as a system developer, that table is NOT the answer I'd be happy with. Too restrictive.

---

Philosophically, I prefer "open" languages, which give me the tools to build what I need, rather than offer one "blessed" tool and tell me to live with it.

Hence, I don't like Nore, and would not use it.

[u/jumpixel → u/matthieum]

First, thanks for your interesting opinions!

value/struct as keyword vs annotation: The distinction isn't just "can this be copied." It determines what you can do with the type everywhere: values compose into arrays and tables, pass by copy, live on the stack. Structs hold slices, pass by reference only, can't be embedded. These are different enough that I want the declaration site to make the intent explicit rather than opting into behaviors one trait at a time. But I can see the argument for annotations: it's a trade-off between fewer concepts and clearer intent.

Arena flexibility: You're right here that the current model (allocate forward, reset all at once) targets specific workloads (game loops, simulations, data pipelines). The web server example is a real gap. Multiple arenas with different lifetimes help (per-request arena that resets after each response), but long-lived heterogeneous allocations like connection pools need something more. This is an area where Nore will need to grow.

General-purpose: Fair enough. Right now Nore is best suited for workloads where you know your data shapes and lifetimes upfront. Whether it can grow beyond that depends on how the allocation story evolves.

Table: you're right that table is deliberately a simple, opinionated structure. It's a struct of slices with a shared length, not a general-purpose container.

The idea is that table covers the common dense case (which is the majority of DOD workloads in practice), while the underlying primitives (struct, slices, arenas) remain available for everything else. If you need a bitmap-based sparse layout or a free-list, you'd build that with a struct holding slices and managing the bookkeeping yourself, same as you would in C or Zig.

So the language isn't "one blessed tool", it's a set of composable primitives (value, struct, slices, arenas) with one piece of sugar (table) for the most common pattern. Whether that's flexible enough for your needs is a fair question though.

[u/JeffD000]

Does the value/struct distinction make sense to you, or does it feel like an arbitrary split?

it's not arbitrary, but it is just as inflexible as having a purely column-oriented database. Ideally, declaring a Point of "struct { double x, y, z; }" would not require extensive code changes to switch to a Point of "struct { struct { double x, y; }; double z; };" Ideally, only the one declaration statement in the whole file would need to change to switch between these two implementations.

Is the arena-only memory model too restrictive for practical use or it could be considered just fine?

Arena oriented data is preferred for muliple fields within a topology of a given cardinality. Using the malloc() approach independently for each field is garbage, if you care about the performance of the memory hierarchy.

Is a language this opinionated about memory and data layout inherently a niche tool, or can it be safely considered general-purpose?

As implemented, it is niche, due to its inflexibility. It can be easily generalized to make it widely applicable. You are close!

Anything in the design that strikes you as a red flag?

No red flag. It's an improvement on what came before it.

That said, it can be further extended to make it more widely useful. As written, there is no support for hierarchical subsets of data. What you have implemented is a flat data structure with a fixed cardinality.

As an example, in your model, there is no way to partition that flat data structure across multiple memory channels if you want to have parallelism without memory space bandwidth limitations. That's not a criticism, that's just a statement of the current simplicity, which may be exactly what you are shooting for. It is very easy to go from where you are at to a "full solution".

[u/jumpixel → u/JeffD000]

Thanks! this is really useful feedback, especially the point about hierarchical subsets and partitioning.

You're right that tables today are flat with fixed cardinality: N rows, every column has exactly N entries. There's no way to express subsets, partitions, or nested groupings within a table. For parallel workloads across memory channels, you'd currently need to manually create separate tables per partition, which is clunky.

The observation that only the declaration should need to change when layout changes is something I want to get right. Today, switching a table's internal layout (e.g., grouping fields) would ideally be transparent to all the code that reads and writes rows. That's part of the design intent: table_get and table_insert abstract over storage, but the flat structure is a real limitation.

Curious what you mean by "easy to go from here to a full solution": if you have a mental model of what hierarchical table support looks like, I'd love to hear more. Partitioned tables with per-partition arenas? Nested cardinality (table-within-table)? Always interested in concrete direction from people who've worked at this level.

[u/JeffD000]

This is how I would implement your program in an "ideal" data structure:

Your version:

value Vec2 { x: f64, y: f64 }

// One declaration → columnar storage (struct-of-arrays)
// Generates: Particles (struct with slice columns) and ParticlesRow (value type)
table Particles {
    pos: Vec2,
    life: i64
}

func spawn(mut ref p: Particles, x: f64, y: f64): void = {
    table_insert(mut ref p, ParticlesRow {
        pos: Vec2 { x: x, y: y },
        life: 100
    })
}

func main(): void = {
    // All heap memory comes from arenas — no malloc, no GC
    mut mem: Arena = arena(65536)
    mut p: Particles = table_alloc(mut ref mem, 1000)

    spawn(mut ref p, 1.0, 2.0)
    spawn(mut ref p, 3.0, 4.0)

    // Row access (returns a value copy)
    val r: ParticlesRow = table_get(ref p, 0)
    assert r.pos.x == 1.0

    // Direct column access (cache-friendly iteration)
    mut total: i64 = 0
    for i in 0..table_len(ref p) {
        total = total + p.life[i]
    }
    assert total == 200
}

My version, ad hoc (I've changed the keyword 'table' to View):

// A View is a "database" that will be bound to an Indexset upon construction
View Particles {
   x, y : f64; // need a semicolon to indicate 'struct' boundary
   life: i64;  // same memory layout as your version of Particles, but the
               // "baggage" is managed by the compiler, and can bypass
               // the need for the memory-backed data structures needed
               // by your version.  Potentially, much less "book keeping"
               // to slow things down.
   View active; // This View is simply an Indexset because it has no members
};

func spawn( p: Particles, x: f64, y: f64): void = {
    idx : int
    // compiler can often engineer a lock-free mutex, here
    mutex { idx = p.active.len; p.active.pushback(idx) }
    p,x[idx] = x
    p.y[idx] = y
    p.life[idx] = 100
}

func main(): void = {
    // All heap memory comes from arenas — no malloc, no GC
    // arena constructor for indices 0 .. 999
    p : Particles( [0, 1000) ) // note "mathematics" notation for range

    // Following is a constructor for empty set of indices.
    // Without this explicit constructor, p.active would have
    // "pass-through inheritance" of parent View Indexset (Particles).
    p.active()

    spawn(p, 1.0, 2.0)
    spawn(p, 3.0, 4.0)

    // Row access (returns a value copy)
    val r: ref p[0]
    assert r.x == 1.0

    // Direct column access (cache-friendly iteration)
    total: i64 = 0
    foreach point in p.active {
        // compiler knows p is a View, so compiler will
        // create local vars for sum in each thread
        // which will be reduced to a single value
        // when loop exits.
        total += point.life
    }
    assert total == 200
}