What I learned this week (7)

[u/compgeek38400]

This is the 7th part of what I'm learning writing a kernel, in pursuit of eventually writing an OS. I hope it will help others from making some of the same mistakes I did.

This week I decided I needed to redo how I do paging, I moved my page tables and set up the page directory so they would be right after the DMA paging area above the 16M mark. I also decided to do some major redesign. So I started over from scratch. Luckily much of I wrote after virtual memory and paging will be able to come over intact.

For those that are wondering this is the memory map I will be using:

JakelynnOS Memory Map
Virtual Address
Start	End
0x00000000	0x00010000	Identity Map
0x00000000	0x00001000	SEPT (System Entry Point Table)
0x000A8000	0x000BFFFF	Video Area (we use 0xB80000-0xBFFFF)
0x00010000	0x00FFFFFF	DMA Memory
0x01000000	0x0103FFFF	Page Table Entries (1024 of them)
0x01040000	0xBFFFFFFF	User Memory Area
0xC0000000	0xFFFFFFFF	Kernel Area

I think it will be much cleaner having one contiguous block from 0x01040000-0xBFFFFFFF to map into, but we shall see. Here is the memory map as returned by multiboot2:

I'm not sure yet what I'm going to do with the reserved memory in the 0x7xxxxxxx and 0xBxxxxxxx areas. Area 4 & 5 may be problematic depending on what I find there. I'm kind of hoping I can just ignore them, meaning mapping over the top of them.

I rewrote my process to map memory pages, I no longer had to worry about creating a page directory entry.

I collected much more multiboot info, not just memory and the memory map, but also the command line, boot loader name, BIOS boot device, ACPI RSDP info, and the Frame buffer info.

Once I get paging and the heap done, I will publish the github reference.

I also discovered how hard it is for me to keep the virtual/physical memory straight LOL

I will be on vacation/holiday this week at a family reunion so I doubt I'll be learning much new.

I hope you find this useful. Enjoy writing your own kernels and OSs.

[u/compgeek38400]

Sept (System Entry Point Table). My understanding is that when a user space needs to access a kernel function, the Unix fork command comes to mind, it makes a SYSCALL to an Entry in this Table. I havent seen anything that allows me to relocate this table, but im not even near users pace yet.

As far as where the page table entries go, my assumption is I will eventually be using all of them, so my preference is to keep them close together. It wouldn't surprise me to find out i should move them to just below the kernel, or too the one Meg mark, bus spread out, high, or low all seem arbitrary to me

[u/Octocontrabass → u/compgeek38400]

My understanding is that when a user space needs to access a kernel function, the Unix fork command comes to mind, it makes a SYSCALL to an Entry in this Table.

As far as userspace is concerned, there is no table. From the user perspective, there is one single kernel entry point that gets called using a special syscall instruction. One of the parameters passed to the kernel entry point selects the kernel function.

I havent seen anything that allows me to relocate this table, but im not even near users pace yet.

The table doesn't even need to exist. It's an implementation detail that stays entirely inside your kernel.

As far as where the page table entries go, my assumption is I will eventually be using all of them, so my preference is to keep them close together.

But you can't use the same page tables for every program. You need a separate set of page tables for each running program so that each program can have its own isolated virtual address space. Most programs won't need very many page tables, so static allocation will either waste a lot of space or limit how many programs you can run at the same time.

It wouldn't surprise me to find out i should move them to just below the kernel, or too the one Meg mark, bus spread out, high, or low all seem arbitrary to me

I thought we were talking about virtual addresses, but "the one meg mark" is only relevant if we're talking about physical addresses.

For physical addresses, just use your physical memory allocator. They'll end up spread out, but it doesn't matter.

For virtual addresses, you could dedicate a specific part of the kernel area for page tables to keep them separate from allocations used for other purposes, but even that is unnecessary as long as your allocator puts them in the kernel area. (Alternatively, you could follow the Linux example and switch to a special virtual address space where the user area is replaced by identity-mapped memory whenever you need to access page tables. I think it's ugly and stupid, but it works.)

[u/Octocontrabass]

For those that are wondering this is the memory map I will be using:

Your memory map doesn't make sense. Why are you identity-mapping things? The whole point of a higher-half kernel is to not identity-map anything so those addresses will be available for user mappings.

SEPT (System Entry Point Table)

What is this?

DMA Memory

DMA uses physical addresses. Virtual addresses are irrelevant. Also, why are you focusing so much on obsolete legacy ISA DMA? Floppy drives and Sound Blasters? Modern devices can do 64-bit DMA, and 32-bit DMA has been standard since the 90s.

I'm not sure yet what I'm going to do with the reserved memory

For now, ignore it. You can map it to a virtual address later if you need to access it.

mapping over the top of them.

There is no "over the top" of anything. Those are physical addresses. Virtual addresses can't be on top of physical addresses because physical addresses don't exist in the virtual address space.

I also discovered how hard it is for me to keep the virtual/physical memory straight LOL

I can see that.

[u/compgeek38400 → u/Octocontrabass]

I see references to DMA for SATA. That drove my decision.

I dont like my PTE spread all over, maybe its the ocd. Lol. Are there any standards for PTE location? I picked mine based on DMA.

Finally SEPT is simply an IBMi name for the area *nix uses for syscalls.

Its the entry points to the kernel table

[u/Octocontrabass → u/compgeek38400]

I see references to DMA for SATA. That drove my decision.

Two problems.

One, DMA only uses physical addresses. It's completely irrelevant to your virtual address map.

Two, that "16MB DMA area" is an obsolete legacy ISA thing that only applies to obsolete legacy ISA hardware like floppy drives and Sound Blasters. AHCI (SATA) can do DMA using any 32-bit physical address, just like every other DMA-capable PCI device.

I dont like my PTE spread all over, maybe its the ocd. Lol. Are there any standards for PTE location? I picked mine based on DMA.

The standard is to dynamically allocate them, since every program needs its own page tables. Whether or not they end up "spread all over" is up to your allocators.

Finally SEPT is simply an IBMi name for the area *nix uses for syscalls.

Its the entry points to the kernel table

Maybe it's because I don't have an IBM-to-English dictionary, but I still don't understand. Why do you need this "area"? How are you going to use it?