The general rule is that program headers (containing segments) are for runtime, and section headers are for compile time. The ELF loader and RTLD do not even look at the section headers, which is abused often in various ways: most commonly, section headers are just stripped right off, but it's entirely possible to devise a binary with valid-looking section headers that don't match the program headers. Imagine: a malicious binary which points to the malicious code in the program header, but totally innocent code in the section header. Lots of tools - including reverse-engineering tools! - blindly trust the section headers for some reason.
The usual compile-time linker does look at section headers, and in particular will merge the various sections of the input .o files based on section name. Sections have more granularity than segments - for example, there is usually a .data section distinct from the .bss section, but often just a single RW segment that covers both (specifying a longer memory size than file size in order to zero-fill the BSS bits).
I believe the distinction between segments and sections largely arises because the runtime (ELF loader/dynamic linker) have different needs from the compiler (static linker); however, I have not looked deeply enough into the history of the ELF format to know exactly why the distinction was made initially. I know many other formats do not make this distinction as clearly: Mach-O nests sections underneath segments hierarchically, whereas PE just has sections: much of the "metadata" that would normally live in ELF sections is instead in special "directories" pointed to by specific entries in the PE file header.
It's weird how all the tools out there are all about the ELF sections. Apparently the sections are where all the fun stuff gets stored and the program headers are just some kernel wizardry nobody really cares about.
The section/segment dichotomy exists so that kernels need not load needless stuff onto memory. In practice it means things like ELF symbol tables are not available for introspection by the program. If pointers to those symbols are required, the easiest way is to get the ELF interpreter to give them to the program. If it's a static executable then there's no interpreter and things get annoying fast. Most solutions I've seen involve reading the program's own executable into memory and parsing the ELF structures, essentially duplicating the work of the kernel's ELF loader. Alternatively, one may engage in this incredibly fun activity called linker scripting.
I chose to hack together an ELF patching tool in order to get Linux to automatically load arbitrary ELF file segments onto memory. Along the way I requested some linker features and actually got them!
Python one-file-packagers such as PyInstaller typically use the technique of shoving the entire Python standard library into the executable as a zip file; Python can natively import scripts straight out of .zip files (zipimport) so you just point `sys.path` at the executable and voila, self-contained Python in a single binary. (Note that this doesn't immediately solve the problem of using native code dynamic libraries, which usually have to get unpacked anyway or use hacky custom loaders).
One year, for DEF CON CTF, we had to write all of our exploits as single ELF binaries targeting a minimal OS (DARPA CGC) with only seven system calls (exit, read, write, select, malloc, free, getrandom). Since writing exploits in C sucks, I packaged Python for the OS instead, creating a "virtual filesystem" for including Python code (including standard library) as a static chunk of data and implementing open/read/write on top of the VFS. This was really quite successful for us, as we were able to write very high-level Python code for a full, functioning Python interpreter in our exploits.
> so you just point `sys.path` at the executable and voila, self-contained Python in a single binary.
You probably know this, but for context for others, the reason this works well for zip in particular is that zip's top level header is at the end of the file with pointers to headers earlier in the file. It doesn't expect the top-level header to be at offset zero.
Nice. But the beauty of GP's approach is that you don't need to issue another open syscall to open argv[0] in order to access (read or mmap) your code but instead just piggy back on what the kernel just did for your entry point anyway.
Except for the beauty of the approach, I'm not sure what the practical advantages are. Are there cases where a process wouldn't have permissions to access its own executable or argv[0] value is unreliable? Can you exec on a file descriptor of a deleted file?
EDIT: or would /proc/self/exe always point to something the process could open?
I believe the process always holds the binary open as a "txt" file descriptor (check lsof), so opening `/proc/self/exe` always works.
You can even execve a "memfd", an in-memory "file" which is not in the filesystem (distinct from a ramdisk file, which is a file sitting on an in-memory filesystem). /proc/self/exe still works even in that case, even when the original memfd is closed.
Note that argv[0] can never be relied on 100%. The vast majority of programs will set it correctly (especially since many programs will malfunction if provided bogus argv[0]), but a caller has full control over argv[0] and can set it to anything, including a NULL pointer (by simply passing an empty argv array).
> I believe the process always holds the binary open as a "txt" file descriptor
To be clear, the kernel has an association between the process and the "txt" file (because it is mmaped in), but this is not an application file descriptor (like 0, 1, 2, ...). If an application wants to read from it, and it isn't already mapped by a LOAD section, it needs to open() a real file descriptor.
Absolutely. It's certainly not mounted automatically after Linux boots and depending on the system's configuration it might never get mounted at all. Maybe it could even use some other path.
One of my long term goals with the programming language I posted is to boot Linux directly into the interpreter and bring up the entire system from inside it. Not only will /proc not be mounted, my program's gonna be the one that mounts it. So I decided to avoid using tricks like reading /proc/self/exe.
> Are there cases where a process wouldn't have permissions to access its own executable
Yes. Permissions might have changed after execution has begun. The file might even have been removed. This creates a race condition.
> argv[0] value is unreliable?
It is. The program calling execve has complete control over the arguments and environment of the program being spawned. It could set argv[0] to anything, including the null pointer or the empty string.
Last year I sent a patch to GNU coreutils that would let env set the argv[0] of programs. My purpose was to use env to test this exact edge case.
> the symbolic link will contain the string ' (deleted)'
> appended to the original pathname.
It's not 100% clear to me if opening and reading the executable will still succeed in that case. I assume it wouldn't work because the manual says it's just a symbolic link to the executable which will become a dangling link if the file it points to is deleted.
There's more: permissions to read the link can be revoked, the link is invalidated if the main thread ever exits, it has a completely different format in old Linux versions...
The ELF segment approach just ignores everything in this comment by getting Linux to mmap the data in just like the program text and data sections. The data will be ready before the program even runs.
> this incredibly fun activity called linker scripting
Indeed. :-)
> I chose to hack together an ELF patching tool in order to get Linux to automatically load arbitrary ELF file segments onto memory.
Interesting. It's clever. I think a slightly more conventional design would be to have just a LOAD segment for your embedded module (for the loader) with some self-describing application-specific header describing the length of the embedded script (for your interpreter), rather than using the ELF Segments table (LOOS+0xc6f6e65) for that.
A dumber hack with less cool learning about ELF might be to just have a magic number variable in your interpreter that is initialized bogus and then patched to point at an offset by the patching tool.
Very interesting. When I was hand-writing ELFs, it took me a while to grok sections vs. segments. If you ever run across a history of executable formats and the motivations for the particular decisions made with ELF, I'd love to read that.
The main thing that surprised me was how loose of a format ELF is compared to the much more rigid conventions imposed by modern mainstream compilers.
If you want to pull back the curtain, I highly recommend hand-writing your own small ELF binary. In particular, on Linux, the C ELF structures are available via:
#include <elf.h>
Writing C code to generate an ELF makes it apparent that an ELF is just a couple of structs and some assembled code dumped to a file. (I've used Keystone with decent success for assembly.) It's actually pretty easy to build something that works if you follow along with the man page:
man 5 elf
For debugging handmade ELF files, it's handy to explicitly run the system loader under strace:
strace /lib/ld-linux.so.3 ./homemade_elf
You can find the path to the interpreter that will be used via something like:
readelf -a "$(which ls)" | grep -i interpreter
For example, debugging with strace will make it apparent if any memory mappings are failing. The loader also sometimes has its own error messages that are more descriptive than a normal segfault.
Also, don't forget LD_DEBUG for debugging your handmade ELFs:
man 8 ld.so
Your advice is completely on point. I spent some time a while back hand-writing ELF files in GNU Assembler. Statically linked executables are, indeed, quite straightforward. That said, there are definitely some mysteries, like the difference between p_vaddr and p_paddr in the program headers. Also, the difference between segments and sections is never really explained in the main references.
Dynamically linked ELF executables are somewhat of a different beast, though. You need some decent familiarity with assembly and the broad strokes of program loading to make heads or tails of the details around relocation symbols.
When I was playing around with these things, I never succeeded at hand-writing x86-64 dynamically linked ELF to do anything other than immediately segfault somewhere during loading. Maybe I should give it another try.
Not to far back, I spent some time decompiling (small) static ELF binaries by hand. That really hammered in the x86-64 ISA and its subtleties. I also have Levine's "Linkers and Loaders" sitting on my shelf, which I hope to get around to reading sooner than later.
Utterly terrifying. They somehow segfault before executing a single instruction in the program's entry point. Even the likes of gdb are rendered powerless before the might of this uber segfault. I was reduced to posting readelf dumps on stackoverflow. Mercifully people immediately spotted the problem (unsorted PT_LOAD segments).
This was one of my favorite StackOverflow debugging experiences: https://stackoverflow.com/a/12575044/1204143. A user posted that their code, `int main() { return 0; }`, was crashing with SIGFPE. I traced it to the fact that they had compiled the code with a fairly new GCC, but ran it on a machine with a very old libc - their old libc didn't understand `DT_GNU_HASH` and was trying to look symbols up in an empty `DT_HASH` (computing the bucket indices mod 0 - hence SIGFPE).
It's possible to debug these! If it's a segfault (as opposed to the kernel just refusing to load your file), it's usually because ld.so (the dynamic linker) has crashed, and you can debug ld.so explicitly (gdb ld-linux.so.2 ; run ./yourprog). With symbols it's usually feasible to identify the code in the dynamic linker that has crashed.
> Even the likes of gdb are rendered powerless before the might of this uber segfault.
It's quite easy to debug crashes in the dynamic linker if you use a more powerful debugger. For example there is a Graal based AMD64 VM [1] which can record an execution trace of the entire program run, including the dynamic linker, and then you can analyze the execution trace offline and see exactly what happened / what didn't happen or where the linker crashed and how it got there. In case you ever wondered what the kernel roughly does when loading an ELF file: look at the re-implementation in the ElfLoader class of that project.
In my case it was a static freestanding nolibc program, there was no dynamic linker or ELF interpreter. :)
The shell's execve jumps directly to the entry point I provided. The execve itself was segfaulting somehow. I couldn't think of anything to do short of running this entire thing in a virtual machine and tracing the kernel itself to see which branch of the ELF loader I was ending up in.
> like the difference between p_vaddr and p_paddr in the program headers.
I see it being used pretty much exclusively in embedded systems, and it looks like some of them set p_paddr to an address but leave p_vaddr to 0. So, it's only seeming use is to indicate when virtual memory is not expected to be used, but otherwise it's function is identical to p_vaddr. "Put the segment in memory here, please."
Coming from DOS/Windows world, I think the main reason ELF is as weird and complex as it is, is because it tries to be both a dynamic and static object format at the same time, and dynamic linking on Unix-likes is also supposed to be an approximation of static linking, but at runtime. In contrast, PE exports are just a mapping from addresses (more precisely, RVAs) to names, and imports are simply an array of slots that point to the name but get overwritten by the actual address by the loader.
An array of slots that point to a name is actually a description of the ELF PLT (procedure lookup table), which is how it does dynamic relocation. The difference is the PLT usually points to the GOT (global offset table) not the name, because in a running program names are irrelevant and only addresses and offsets have meaning.
Resolving symbols through the PLT/GOT mechanism is much faster than resolving through string lookup at program startup (or library load). That's one of the main reasons why program startup and library loading is measurably faster on an ELF-based system than on a COFF-based system like Microsoft Windows.
It gets a bit more complicated than that as DLLs symbols are namespaced, can be changed to lazy link on demand instead of process/thread start, can represent .NET code instead of native, can contain COM objects with dynamic configuration and lookup rules, have lifetime logic associated to processes/threads, can have a special main entry point and invoked via RunDll or regsvr32.
Ah, custom sections and loaders can also be defined.
Stuck in Windows 3.x days? Most of that isn't needed for quite some time.
Also, COFF isn't Windows only, it originated in the IBM world, and AIX is notorious for being a UNIX that doesn't favour ELF by default, even thought it supports it as well.
The general rule is that program headers (containing segments) are for runtime, and section headers are for compile time. The ELF loader and RTLD do not even look at the section headers, which is abused often in various ways: most commonly, section headers are just stripped right off, but it's entirely possible to devise a binary with valid-looking section headers that don't match the program headers. Imagine: a malicious binary which points to the malicious code in the program header, but totally innocent code in the section header. Lots of tools - including reverse-engineering tools! - blindly trust the section headers for some reason.
The usual compile-time linker does look at section headers, and in particular will merge the various sections of the input .o files based on section name. Sections have more granularity than segments - for example, there is usually a .data section distinct from the .bss section, but often just a single RW segment that covers both (specifying a longer memory size than file size in order to zero-fill the BSS bits).
I believe the distinction between segments and sections largely arises because the runtime (ELF loader/dynamic linker) have different needs from the compiler (static linker); however, I have not looked deeply enough into the history of the ELF format to know exactly why the distinction was made initially. I know many other formats do not make this distinction as clearly: Mach-O nests sections underneath segments hierarchically, whereas PE just has sections: much of the "metadata" that would normally live in ELF sections is instead in special "directories" pointed to by specific entries in the PE file header.