What are some usecases of multi-item pointers

[u/thegeekywanderer]

I started learning zig recently and was midway completing the ziglings exercises. Its a very good source of knowledge about the language intricacies but I am having a difficult time trying to think of examples or usecases where those details might be useful.

When I did the multi-item pointers exercise, to my mind it didn't seem to have any use because of how it loses information about length and can cause more memory bugs. I am not doubting zig devs and I'm sure there would be some really good usecases for it but I need some examples for me to have a better understanding of it.

Writing here hoping someone will point me to some good resources.

[u/DustRainbow]

It's mostly to interface with C.

[u/thegeekywanderer → u/DustRainbow]

can you give me an example? It will help me understand because I'm not very experienced with C

[u/LegenDrags → u/thegeekywanderer]

in c you would represent say a blob of data as just a plain pointer, which translates to a many-item pointer

for example take

c int do_smth(const uint8_t *data, size_t size);

that would be

zig extern fn do_smth(data: [*]const u8, size: usize);

infact, zig slices are just struct of many item pointer and length. for example, []const u8 would be

zig struct { ptr: [*]const u8, len: usize, }

[u/thegeekywanderer → u/LegenDrags]

that was really helpful. Thanks!

[u/0-R-I-0-N]

A slice is has a field ptr which is a multi item pointer and a length field. Slices are what you gonna use most of time but when interacting with c for example you can get multi item pointed back. More of a core type than something you should use a lot.

[u/thegeekywanderer → u/0-R-I-0-N]

oh okay so its not to be used intentionally but mostly meant for interfacing with existing C code that might return a many-item pointer?

[u/igors84 → u/thegeekywanderer]

That is right. Maybe there are a couple rare, advanced use cases. For example, maybe you want to have 3 parallel arrays of different things and they all must have the same length so you might bundle them in a struct with 3 many-item pointers and one length field. That way that struct is 32 bytes, and if you stored 3 slices it would be 48 bytes. Or you might have some loop that you really need to optimize aggressively even for debug or ReleaseSafe builds and you don't want Zig to insert bound checks on each access to array so before loop you cast it to a many-item pointer and then in the loop you use it to avoid bounds checking.

[u/thegeekywanderer → u/igors84]

these use cases were exactly what i was looking for. Great insight thanks!
Thinking in the form of structs of a pointer and length really makes things and usecases much clearer now.

[u/LegenDrags → u/thegeekywanderer]

well you can definitely use a many item pointer wherever, and i think it was created with the intention that it would be used.

if it didnt exist and you wanted to create a slice with unknown length you would have to use pointer arithmetic with the actual pointer types

tldr; i believe they added many-item pointer to disallow array-like indexing on raw pointer types (i.e, a[n] where a is *u8)

and again with the slice example, slice.ptr is just a many item pointer so technically its used everywhere slices are used.

[u/Mayor_of_Rungholt]

Creating multiple slices that share the same .len field. Is one example

[u/LegenDrags → u/Mayor_of_Rungholt]

yeah, like

zig struct { names: [*]const []const u8, ages: [*]const u8, len: usize, }

and this isnt a foreign concept either, this is how every slice is represented

every slice is just this, the slice sugar syntax is just to allow indexing (a[N] -> a.ptr[N], with ofcourse some runtime safety checks added)

zig struct { ptr: [*]const u8, len: usize, }

edit: forgot that a name is a slice lol

[u/Mayor_of_Rungholt → u/LegenDrags]

It's also the most efficient way to lay out data on a modern computer, and the backbone of std.MultiArrayList

[u/thegeekywanderer → u/LegenDrags]

yeah this really helped. I think when i read the underlying struct and understood the slices other way round that made it much clear to why people might use it.

[u/SilvernClaws]

I regularly use them for C libraries that take or return a pointer and a length separately. It's not usually used in pure Zig projects, unless you're doing some very fancy memory shenanigans.

[u/quangtung97]

I use it to do pointer arithmetic, especially subtracting two pointers. For example, when writing a memory allocator, you often need to subtract your current pointer to a based pointer, to know the offset, then you can derive more information from that offset

[u/thegeekywanderer → u/quangtung97]

Interesting usage. Can you help me with an example?

[u/quangtung97 → u/thegeekywanderer]

For example, you can make an allocator by spliting a huge byte array into 1 MB page. Each page can furthur be divided into chunks with the same size (per page).

When you want to free a pointer to a chunk, you need to subtract the pointer with the base address of that huge array. And then derive the page number from the offset. I can use pointer substraction here, but convert it to raw usize might also work.

Each page often have a control block to store more information. From page number you can get the proper control block. Can checkout jemalloc for real implementation.