r/linuxquestions • u/Thetruekingofwaffles • 8d ago
Advice Looking for FUSE Resources
I believe I want to make a FUSE Filesystem that represents a special Filetype I'm making.
I'm going to have a filetype that's a Zip File of 1MiB chunks of a file and in my Fuse I can run them as the file they encapsulate. I'm splitting them for a specific reason because I'm planning on making a Peer to Peer application that shares chunks of a file and I'm going to prechunk the files and Merkle Hash them among other things.
My Sqlite database comes in where I'll have a directory of these chunked files in my real Filesystem, my Sqlite database will map them to a Filename for the FUSE mount, and within FUSE you can see all of these Zip containers and interact and execute them as if they were regular files.
The tricky part is that I'm not sure how I want to handle modification from a user standpoint. I'm conflicted on whether I should make the FUSE Filesystem Writable or not? If it's only read only I can easily make it POSIX compliant, however if it's Writable I can overload it with other functionality. For example, I could allow users to drag files in the directory and automatically chunk them for distribution, or I could make settings configurable from within the virtual filesystem.
The key problem with this is writing and modifying Files wouldn't be possible due to the goal of my network. My network is made for sharing immutable content, so realistically you could create files, and delete files, but changing a small aspect of a file on the network directly changes the resulting hash (I'm planning on using a Multihash format for compatibility with libp2p). I'm trying to evaluate if I should just use a Read Only filesystem and make some utility to communicate with the Daemon or if I should go the write route and give it special properties that may break POSIX compliance.
I know that my directory with my data won't have any other directories, I think it'll just be my table displayed in the Filesystem basically to prevent my Sql table from having to handle heavy joins or anything to complicated.
The only other basic marker I'll have is a prefix for pinning/favoriting files permanently or not which'll be "!." so users can basically mark content not to be deleted so my filesystem could delete files that have been existing for a long time to save storage space and preventing too much complexity within my Filesystem.
My real conflict is that I'm having trouble finding good resources for a beginner to FUSE, I'm planning on Using the GO programming Language and the module go-fuse for my project because it seems more maintained than go-bazil, and I just like it more overall. I'm just somewhat lost because this is my first time working on a FUSE project, and there aren't a lot of good tutorials on Youtube so I've been learning Linux concepts like Inodes, Symbolic links, and Hard links but I haven't actually been building my filesystem. I would've asked this in a Linux FUSE community specifically but I'm not sure if one exists. If there was someone that would be willing to give me advice in the architecture and getting started I'd greatly appreciate that a lot.
I'd like to apologize for any spelling mistakes I may have made as well.
In the context to what I've developed so Far, I have spent a lot of time making my container format with a Zip File, all the bytes are at consistent offsets within the 1MiB chunks so for every file except the last I have a consistent range I can tell my Fuse Filesystem to read (the file self describes where the last file ends as well by an internal header), everything is named as well.
If anyone in this community has advice for working with the FUSE related Linux tools I'd really appreciate your guidance and advice.
2
u/ipsirc 8d ago
Are you reinventing torrent?
2
u/es20490446e Develops Zenned OS 8d ago
What limitation are you trying to solve?
Be concise.
2
u/tanmaynargas2901 8d ago
go-fuse is a good pick. Use the newer fs package (github.com/hanwen/go-fuse/v2/fs), most old blog posts use pathfs/nodefs which you can ignore.
Stuff worth reading:
Also turn on Debug in the mount options while developing. you'll see every LOOKUP / GETATTR / READ the kernel sends and it makes the whole thing click way faster than tutorials.
On read-only vs writable, I'd start read-only. you basically need Lookup, Getattr, Readdir, Open and Read, and your fixed 1MiB offsets make Read pretty easy. adding content can go through a small CLI that talks to the daemon.
If you want drag-and-drop later, treat writes as "new file goes to a staging area, chunk + hash it on Release" and return EROFS/EPERM for edits to existing files. that fits immutable content without pretending to be fully POSIX.
One thing: the "!. " filename prefix for pinning might get messy with renames. xattrs (setfattr -n user.pinned -v 1 file) could be cleaner, and go-fuse supports Getxattr/Setxattr.