DLLs for FNK

This sort of outlines my process of design for DLLs when creating the FNK operating system. I'll continue adding onto this post until the system is done, but I will also extrapolate some of the designs that I haven't implemented yet. Basically, the content here is up for change.

Background + Motivation

Its important to understand things like what is a DLL in programming and how processes are even loaded.

The need for dynamic

Nearly all programs will attempt to access some form of data. Whether that be through register displacement: mov ax, [bp + label], relative addressing: jmp short +5 or absolute means: mov ax, [label] your program needs to access data. We can already see a problem when it comes to the absolute means: what happens if our program is put somewhere else? That label marker is relative to literally nothing and is bound by how it is compiled. Once the program is compiled the code will flop down to mov ax, [0x55AA]. Say I place my program in memory at 0x55AA even thought it was expected to be loaded in at 0x3333, I'm now accessing data improperly. With a process system, you want to be able to fit as many programs as possible and give them the room they need. But if I need to specify where processes are loaded, then processes could possibly overlap or request the same memory origin. Personally, I just didn't want to have to make the user manage their own process's memory. Now, you might be thinking "well why don't we just only use relative addressing (and/or displacement to registers like bp)?" Well first of all, someones a bit demanding of their compiler. Still, in a modern context something like that might be possible. But there are still multiple reasons why this wouldn't work (besides the pain and suffering of compiler writers):

C is meant to work on nearly everything. Some processors have limited relative addressing depths (e.g. 6502 is only +-127 bytes). Sometimes your code just wouldn't be able to compile if you forced the compiler to only use relative addressing
Absolute addressing is fast. It makes it so the processor's pipe has to do less calculations. A few instances is fine but if you really want speed (I AM IN AN EMBEDDED ENVIRONMENT HERE I NEED IT), you should opt for absolute addressing when you can

Modern processors have devised nice solutions to this like memory segmentation which makes programs think that they are loaded in at a specific memory location virtually (but physically their at a different address) but I don't want to have to deal with any architecture-specific memory manager, so I opted for a different means.

The need for libraries

Libraries are pieces of code that serve a specific purpose. They are entirely necessary for my project due to the fact that I things like syscalls and util code was necessary. I wasn't going to write every program from scratch and I had to create some non-jank way to communicate to the operating system and drivers. I was going to use sockets of course to communicate to drivers, but I also wanted to be able to get that socket and do other operations via some other way. I had two options here: DLL or statically linked library

DLL (dynamically linked library)
- One instance of the library is created and then the operating system routes the program to it
- A lot of work to implement
- Drivers + kernel know where to look to check on process requests
Static libraries
- Multiple instances of the library (one for each process and it is packaged with the process code)
- Much less work to implement
- All library memory is process memory
- Drivers + kernel don't know where to look to check on process requests (they must have an instance list or something probably)

Because many RTOSs don't have dynamic process loading (I need this because you can insert a chip and that creates a new process) the static solution is good enough. They simply shove everything together so there will only be 1 instance of every library. Programs know where the libraries are because they are linked with knowledge of the library. Regardless, I opted for DLLs because:

It's a fun challenge (of course..)
I needed dynamic process loading and static linking would bloat the process system
The actual resolution of DLLs is quite easy you'll see

The solution

I devised a simple custom format: .fnk. The format allows me to solve all of my problems with very little processing from the operating system.

Relocation tables

Relocation tables are a beautiful thing. They are simply a table that points to all occurrences of absolute addressing. With this information we can just add an offset (where we load in the program) to all entries in the relocation table. Even better, relocation tables are generated by the compiler. You know when you create an object file with -c? That creates a relocatable ELF file. It includes a relocation table for the program. Why? Because when the final linking process happens and the linker may want to move sections (parts of code are put in sections and then absolute addressing usually includes the section + an offset) around in the final binary, it actually uses that relocation table to change the offsets of those sections. To create our program, we can simply resolve all relocation entries (the relocation table contains the absolute address usually so we have to fill that in) and then write the new table with just pointers and sizes to the binary

for (entry = reloctable; entry < reloctable + sizeof(reloctable); entry++) {
    relocentry_resolve(entry, code); // Write the absolute addresses to the code
    fwrite(relocplace, entry->position, entry->size);
}

And when our operating system loads in a program:

for (entry = reloctable; entry < reloctable + sizeof(reloctable); entry++) {
    code[entry->position] += load_offset;
}

Now something like mov ax, [0x55AA] will become mov ax, [0x55AA + 0x3333] (the addition is evaluated I'm just using it as a representation)

DLLs + processes = good but unsafe

With no memory protection utilization I don't care about safe design decisions. So, I decided to make libraries and processes be the same thing. This is through LJDs and PJDs

LJDs

LJDs (library jump descriptor) is a list of the functions you want to expose. You create pointers to all of those functions and then another program can copy it over and reference whatever it wants to.

ljd[] = {
   function_one,
   function_two,
   function_three
};

When the program is loaded this LJD is modified to include the program offset.

PJDs

PJDs (program jump descriptor) are where the LJDs are copied to. The program will look through all libraries listed in the PJDs and then copy the LJDs from the necessary program. Then all the program has to do is call functions from this array.

struct pjd {
    char name[COMMON_FNKCONFIG_NAMELEN];
    uintptr_t functions[COMMON_FNKCONFIG_MAX_LIBFUNCTIONS];
};

struct pjd fnk_pjd {
   .name = "fnk",
}

void (*fnk_socket_init)(
    struct fnk_socket*,
    void*,
    size_t,
    void*,
    size_t
) = fnk_pjd.functions[0];

Loading

Now all we have to do is a memcpy from the LJD to the PJD for the respective library.

The final product

Here is the file format I devised for the .fnk format with all that in consideration

Parameter	Size (bytes)	Value description
MAGIC	6	0xFE 0xED 'F' 'N' 'K' 'Y' - Used to confirm `.fnk` format.
NAME	12	Name of your program. Must be unique.
RELOCATION-ADDR	ARCH_CONFIG_WORDSIZE (config.h)	Address to the relocation table (0 for none)
ENTRY-ADDR	ARCH_CONFIG_WORDSIZE	Address of the entry section (0 for none and therefore unexecutable)
LJD-ADDR	ARCH_CONFIG_WORDSIZE	Address of the LJD section
PJD-ADDR	ARCH_CONFIG_WORDSIZE	Address of the PJD section
BSS-LENGTH	ARCH_CONFIG_WORDSIZE	How long the blank storage section will be

Eric Heflinger