Hey! Today we will look at one of the fundamental file formats that you will be confronted with
when reverse engineering on different platforms:
ELF - or the extensible and linkable format!
ELF was introduced with Unix System Five in the late eighties, and is now the standard
executable file format on Linux, most BSDs, the playstation 4, the Wii, and also a container
format for microcontroller firmwares and many other things.
Lets start by looking at what ELF is mainly used for: The three main uses of ELF are as
a format for executbales, shared libraries and as a format for object files.
It can also be used for a lot of other things, but in this video we will only focus on those
three.
Now the two main terms you will hear in the context of ELF are segments and sections.
Each ELF file can contain zero or more segments, and zero or more sections. Now while
in the english language segments and sections are synonyms for each
other, in ELF they describe two completely different things.
This is super important to remember, and even a lot of articles on ELF get it wrong. The
main thing you have to know is: Segments are basically exclusively used at
runtime, while sections are almost exclusively used at link time!
This means that it's prefectly valid to have an ELF executable that only contains segments,
and it is also completely valid to have an object file that only contains
sections!
Now segments and sections both represent data from the ELF file, , and the way this works
is pretty simple: If you imagine the ELF file as a flat memory
space, a segment will point to an absolute address in that memory, and specify
how many bytes are contained in the segment. Note that there are also some segment types
that do not have any contents, we will take a look at those later.
Sections work basically the same way, and in most cases where you have a file with segments
and sections, they will overlap, so often segments will contain multiple sections.
Now the main difference between sections and segments, is that segments also specify where
and how they should be loaded into virtual or physical memory - so segments tell
the operating system how to map parts of the ELF file into memory!
Lets take a look at an example: Running an ELF executable. A simple, statically compiled
executable contains two main segments that need to be loaded into memory:
The first is the data segment, which contains initialized globals and other initialized
data. It basically just puts a certain byte range from the ELF file into memory at
a certain address. In most cases the mapped data segment in memory will be larger than
the segment in the ELF file: This is used to specify the room
for uninitialized data, such as uninitialized global variables.
The next segment is the code segment, which contains the executable code. It will be loaded
into memory, and finally the operating system will load the address of
the entry-point: The address in memory where the binary should start its execution.
Now this is a very basic, statically compiled binary without features such as address space
layout randomization etc. - lets look at a dynamically linked binary!
In a dynamically linked binary we have an additional segment in the ELF file that specifies
which shared libraries need to beloaded
The operating system then goes and loads those shared libraries, which are also ELF files,
and maps them into memory just like it did with the executable.
There are some more details involved, and so for example the loading is a bit different
for executables with position independent code: Here, the addresses are
not fixed, and so the order and addresses will be different for each launch.
As you probably can imagine, the memory map can get quite complex when you have a binary
with a lot of shared libraries!
Now before we dive into all the different segments and sections, lets start at the beginning
of the ELF file, where ELF header resides:
The ELF header is a structure that contains the metadata of the file, and so lets dig
into it!
At the beginning of the file we have what's called E_IDENT identification indexes - 16
bytes that descirbe how the ELF file needs to be parsed. Lets take a look
at an example!
This is the E_IDENT part of the `ls` executable of Ubuntnu 20.04 running on a 64 bit intel
processor. The first four bytes contain a magic number
- hex 7f followed by the characters E, L, F encoded as ASCII.
These bytes are the same for all ELF files and help you quickly verify whether a file
actually is an ELF file.
The next byte specifies the class of the ELF file: Whether it's a 32 bit ELF file or a
64 bit ELF file. This is important for the ELF parser to know, as certain datatypes
contained in the ELF file will have a different size depending on whether
it's a 32 bit or a 64 bit elf file. In this case it's set to 2, and so we are looking
at a 64 bit ELF file.
The next byte specifies the data encoding used in the file: Whether in multi-byte types
the least significant byte comes first or the most significant byte comes first.
As you can see in this case it's set to LSB - least significant byte first.
Next we have the version byte: This is always set to 1, since the 80s ELF has stayed at
version 1!
The next byte specifies the Operating System Application Binary Interface - the OS ABI.
As you can see, even though this is a Linux executable, the OS ABI is still set
to 0: This is very common, and there have been quiet some discussions around this,
but the essence is: It will often be set to 0, though you will find taht for example statically
compiled binaries will often set this to 3 on Linux.
The next byte, encoding the ABI version, is almost never used, and so we just skip over
it. Finally, we have the last 7 bytes which are
simply used as padding and not used at all.
Lets jump back to the whole header: as you can see i created a pseudo structure, using
the same names for each structure member as in the standard and in
the Linux kernel code. You might also notice the types:
ELF has its own set of types, such as HalfWord, Word, Address and Offset - the size of some
of these types depend on the ELF class - so whether we are looking at a
32 bit or a 64 bit ELF file.
Lets take a quick look at the main types used in ELF, if you want to look at the actual
structure definitions as they are used in the kernel then check the link in the description!
Lets start with HalfWord; Half word is always a simple 16 bit unsigned integer type. Next
we have "Word", which will always be a 32 bit unsigned integer. Then we have
the Address type, used to specify memory addresses: This one can either be a 32 bit
or a 64 bit type, depending on the ELF class. The same is true for the Offset type, which
is mainly used to specify offsets in the ELF file.
There are some more types that you can find in the ELF specification, but this should
give you a good enough overview.
Lets continue take a look at the other members of the ELF header: The first member after
the E_IDENT bytes is called the type, and specifies the type of the ELF file. There
arae currently 5 defined values, the first one being zero or None,
which I've never seen used in the wild. The second one is REL or relocatable file,
which is the ELF type used for object files. It specifies that it can be used for a
relocation respectively linking step. It's the ELF type you get when you for example
compile something in "compile only" mode, here specified by passing the -c parameter
to gcc.
The third, with a value of 2, is the EXEC or Executable file type: This is the ELF type
for executables, but it is important to note that if you see an ELF file with type
EXEC that it will not support ASLR, as the EXEC type does not support
position independent executables. This means that with a modern compiler you have to disable
PIE explicitely, for example by compiling with dash no dash pie.
Next we have DYN or dynamic. This type is mainly used for two purposes: For shared libraries,
and for executables with position independent code!
Then we also have the CORE type, which is used for core dumps, but we will not look
at those in this video.
There are also some processor reserved value ranges, which you won't find too often in
the wild.
The next field is the machine type: There are a couple of standard values for this,
for example hex 03 for x86, hex 28 for ARM, and hex 3E for amd64.
The machine type is followed by the version field, which just like in the E_IDENT bytes
is always set to 1.
Next is the entry point, which is an address type; This specifies the entry point into
our executables, or the constructor address for shared libraries.
If there is no entrypoint it will be set to zero.
The next two members with a type of offset specify where in the ELF file the program
headers, which specify the segments, and the section headers, which specify the
sections, can be found. The offset here is relative to the beginning of the file,
so you can treat it as an absolute address into the file.
The next 32 bit value contains flags for our file: These are heavily architecture & OS
dependent, and often specify things such as the execution mode for the
processor.
This is followed by a member specifying the size of the ELF header itself, which seems
kind of redundant, as we already know its structure from the ELF class, but might
have been introduced to future proof the fileformat.
After this, we have two members specifying the size of each entry in the program header
table, and also the number of entries in the table.
We will take a good look at this once we are done with the ELF header.
Then we have basically the same, but this time specifying the size and number of the
section headers.
Finally, we have what's called the section header string table index, you will see that
this is being used to resolve the names of the sections contained in the
file.
And that's all there is to the elf header! If you ever want to take a look at an ELF
header of a file you have, you can use the "readelf" tool with the -h option.
It will show you all the information contained in the ELF header in a nice
human readable format.
Now we talked earlier about segments and that they are mapped to memory during execution,
now it's time to look at how they are actually defined in the ELF file:
Each segment is definde by a small program header structure. All program headers are
in an array directly behind each other in the ELF file, which is where
our ELF header comes into play again:
The ELF header specifies the location of the program header table in the Program Header
Offset member. It also specifies the size of each program
header entry, and the total number of entries.
Lets take a look inside of a program header!
At the beginning of the program header the type of the segment is specified:
A segment type of NULL is a placeholder, and also provides a simple way to disable a segment
by changing a single value in the ELF file.
Next we have the LOAD segment type: Segments of this type will be loaded into memory. How
exactly it will be loaded is specified by the other members of the program header,
and a LOAD segment can also be used to create zero initialized segments by specifying
a larger size in memory than what's loaded from the file.
We also have a DYNAMIC segment type, which contains information required for dynamically
linked binaries, and is interpreted by the operating system to load shared libraries
related to an executable.
The next type is INTERP, or interpreter. A segment of this stype specifies the program
interpreter required by a dynamic executable. If you remember from earlier, we noted that
both position independent executables and shared libraries have an ELF type
of dynamic. The presence of a program interpreter lets us differentiate between both: Only executables
will have this segment type, while shared libraries will not have a segment
of this type. I've linked an article about the specifics of this in the description,
though I might also do a fundamentals video about it in the future. The content of the
segment is a simple null terminated string, and will point to an absolute path.
We also have segments of type note: Segments of this type can hold auxiliary information
for the binary, for example for a debugger or other tools.
The next type, SHLIB, or shared library, is undefined and should never be used. ELF files
that use this are not conformant to the System Five Elf specification.
Next we have PHDR, or program header. This segment specifies whether and where the program
header table will be loaded into memory.
There's also a special segment type for thread local storage, which contains information
about initialized thread local variables and so on. I've linked some more
info about this in the description.
We also have a range of operating-system specific segment types. For example on GNU Linux there's
a segment that specifies which parts of memory should be marked as
read-only after the dynamic loader has done its job.
Finally, there's also a reserved range for processor specific segment types.
The next field in the structure is the offset: This specifies where in the ELF file the content
of this segment - if it has any - is located.
Then we have the virtual address field: It specifies where the first byte of the segment
will be in memory. The next member is only used in contexts where
physical memory is relevant, for example in firmware files, and specifies
the physical address where the segment will be loaded.
Next the size of the segment in the file is specified: If this is 0, the segment is defined
exclusively by the program header and has no furhter content in the file.
Then we have the size of the segment in memory - if this is larger than the size in the file,
the leftover bytes will be initialized as zero.
The flags allow specifying the permissions of the segment, so whether it's readable,
writable and or executable. For example the code segment will in most cases be set
to Executable & Readable, but not writable.
Finally we have a field specifying the alignment requirements for the segment, which allows
specification of whether a segment needs to be aligned to lets say
a 4 byte or an 8 byte boundary. And that's all there really is!
One more interesting thing though is that the order of the program hedaer members, for
reasons I couldn't find, is different between 32 bit and 64 bit ELF
files...
If you ever want to investigate the segments contained in a binary, you can just run readelf
--segments, and it will list all the segments in the binary, including
all their addresses and flags.
Here for example we can see both the segment that contains the data, identifyable by the
fact that this segment is Only readable and writable, and also the code segment,
which we can identify because it's executable and readable.
We can also see the segment containing the program interpreter, indicating that this
is a dynamic executable.
Now, lets talk about sections! Just like segments, sections are defined by a header, the section
header. And also just as before, the section headers are in an array
in memory, called the section hedaer table.
This section header table can again be found by checking the file offset in the ELF header,
and just like with the program headers,
the ELF header again specifies the size of a section header, and also how many section
headers there are.
Lets dive into the section header and check it out.
Now we will only take a relatively qucik look at sections on a high level, as theyu are
only used during linking and by tools such as debuggers. During execution, and so
also mostly when reverse engineering, only the segments count! You can actually
test this yourself: You can use the tool sstrip to remove all
sections from an executable, and use readelf to verify whether that was successful.
As you can see, even though our file contains no sections, it still runs just fine!
The first member of the section header is the name of the section: Now this member is
not, as you might expect, a string, instead it's an offset into what's called the section
header string table! If you remember, the last field in the ELF header was called
section header string table index: This index points to the section number that contains
a string table containing the name of the sections!
A string table is basically an array of zero terminated strings, for example here we have
the Word example at offset 0, terminated by a null byte.
At offset 8, we have the word table, also terminated with a null byte. And that's exactly
how a string table in ELF works:
It's a large table of strings, in this case defining our section names, and the name field
just specifies an offset into that string table.
Next, we have the section type. Just like segments, sections have quite some different
types, so lets go through them quickly.
We start with NULL, which is just an inactive section and be ignored.
Next we have PROGBITS: A progbits section contains data for the program, whether it's
cpde or variable data or really anything, it doesn't have a defined format
and can really contain anything.
Then we have SYMTAB and DYNSYM: As the name, SYMTAB, already suggests, these are symbol
tables. Symbol tables contain list of symbol table structures, each defining
a symbol. We will not go into too much detail on these in this video,
but basically this is used to define things such as local and global symbols
and their addresses as they are required during linking.
The next type is a string table section, and you just saw a minute ago how those work.
Next we have the RELA, or relocation section type. This section type contains information
on how to modify different sections when assembling an executable, you could probably
do an entire video just on the process of relocation, so I've linked
some further information in the description.
Next we have the HASH type, which contains a hash table, and afterwards the DYNAMIC type,
which contains information required for dynamic linking and generally has the
same contents as the PT_DYNAMIC segment type.
The note type, just like the NOTE segment type, can contain auxiliary information and
notes for use by different tooling.
NOBITS defines a section that does not take up any space in the file, but specifies required
space for things such as unintialized variables.
SHLIB, just as in the segment types, is again reserved but unused.
Finally we again have some ranges for processor specific and user specific sections, and that's
all the section types there are!
The next member in the section structure contains flags. There are 3 flags as part of the specification:
WRITE, which indicates that this section should be writable if a segment is created from it,
ALLOC, which specifies that this section should be part of the segments that are loaded into
memory and EXECINSTR which specifies that the section
contains executable code.
There's also a whole range of flags that are reserved for processor-specific semantics.
The next member specifies the address at which a section will appear in the memory image
of a process, if it is contained for example in a loaded segment.
Offset specifies the lcoation of the section data in the ELF file, and size specifies how
large the section data is in the ELF file.
Next we have the link member: This allows a section to be linked to another section,
and so a section might use the link field to indicate that it contains auxiliary information
for another section.
Info then can contain some extra information depending on the section type, and addralign
specifies whether the section has any alignment constraints.
Finally we have entsize, which for sections that contain fixed size entries contain the
size of each entry. If you divide the size specified in size with the value
of entsize you then get the number of entries in the section.
As always, if you want to introspect an ELF file for yourself, you can use the readelf
tool: With --sections it will list all the different sections,
which would then be used by a linker to create an ELF file with the segments required to
run the code contained in the sections.
Now this was a lot of stuff, but I hope it gave you a good overview in the structure
and contents of an ELF file. In a future video we might take a deeper
look at some of the involved parts, such as the program interpreter, the relocation, and
so on. I hope you enjoyed this video, and to see
you on this channel again soon! Bye!