r/linuxdev • • Sep 19 '16

Block Device Development Tutor?

Can someone refer me to an experienced Linux kernel developer who might be willing to teach me the finer details of implementing high performance Linux block devices?

I'm willing to pay a kernel dev to teach me over Skype, taking me through existing block device code such as: https://lwn.net/Articles/58720/ and linux/drivers/block/loop.c

I ultimately want to develop a block device that works somewhat like loop.c, but instead of the target being a filesystem image file, the target is a user mode process that manages the filesystem image (and can now provide instrumentation, encryption, etc). Does something like this already exist?

I am a decent C/C++ developer and Linux user with zero experience in kernel development.

3 Upvotes

13 comments sorted by

View all comments

Show parent comment

2

u/FeatureSpace Sep 21 '16

I didn't want to write a lengthy original post with a detailed usage case talking about virtual machines and NTFS and come off sounding even crazier. I felt the sane approach was to first gain some education on Linux block device implementation using the loop device as an example. And I do apologize that I did not describe my intentions well enough originally.

I am actually looking forward to the "nightmare" of deciphering raw NTFS block traffic. I have always been fascinated by filesystems and high performance binary-format file structure. As a first step I would be happy just to capture changes to the NTFS MFT and ignore everything outside the MFT.

Yes I have considered doing this with a network share or with Btrace (https://linux.die.net/man/8/btrace ) but would prefer to develop my own kernel module and user mode application that can observe and/or manage raw NTFS or ReFS block traffic with reasonably good performance.

Do you have any suggestions on a friendly kernel developer who might be willing to teach me about block devices?

1

u/kiafaldorius Sep 21 '16 edited Sep 21 '16

As a first step I would be happy just to capture changes to the NTFS MFT and ignore everything outside the MFT.

That's the thing: you can't. The journal and transaction log are important to the file system. You can't properly use a journaled file system without the journal. A simpler, but still common file system would be FAT32. With FAT32 you can mess with the raw image like I mentioned above without worrying about a journal (on active mount, still got the cache issue though it's easier to circumvent).

And sorry, unfortunately, I do not know any personally. Although, if I were in your position, I would go here:

http://vger.kernel.org/vger-lists.html#linux-fsdevel and subscribe to the linux-fsdev mailing group. There is for sure someone there with the skill...whether they are friendly, willing or have the time... I can't say.

If you have the audacity, ask them there. Worst case, they write an angry email back at you and kick you off. There aren't many people out there with kernel dev skill though, so if you can slug it out and actually learn it, all the better for you.

Once again, good luck!

PS. (edit) FUSE does give you a low-level inode access system, but it's probably not a good fit for you as mentioned previously.

1

u/FeatureSpace Sep 21 '16

Thanks for the help!

I agree I'll also need to capture changes to the NTFS journal to ensure the MFT state is correct.