Why does GCC create a shared object instead of an executable binary according to file?
What am I doing wrong?
Nothing.
It sounds like your GCC is configured to build -pie
binaries by default. These binaries really are shared libraries (of type ET_DYN
), except they run just like a normal executable would.
So your should just run your binary, and (if it works) not worry about it.
Or you could link your binary with gcc -no-pie ...
and that should produce a non-PIE
executable of type ET_EXEC
, for which file
will say ELF 64-bit LSB executable
.
file
5.36 says it clearly
file
5.36 actually prints it clearly if the executable is PIE or not as shown at: https://unix.stackexchange.com/questions/89211/how-to-test-whether-a-linux-binary-was-compiled-as-position-independent-code/435038#435038
For example, a PIE executable shows as:
main.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, not stripped
and a non-PIE one as:
main.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
The feature was introduced in 5.33 but it did just a simple chmod +x
check. Before that it just printed shared object
for PIE.
In 5.34, it was meant to start checking the more specialized DF_1_PIE
ELF metadata, but due to a bug in the implementation at commit 9109a696f3289ba00eaa222fd432755ec4287e28 it actually broke things and showed GCC PIE executables as shared objects
.
The bug was fixed in 5.36 at commit 03084b161cf888b5286dbbcd964c31ccad4f64d9.
The bug is present in particular in Ubuntu 18.10 which has file
5.34.
It does not manifest itself when linking assembly code with ld -pie
because of a coincidence.
Source code breakdown is shown in the "file
5.36 source code analysis" section of this answer.
The Linux kernel 5.0 determines if ASLR can be used based on ET_DYN
The root cause of the file
"confusion", is that both PIE executables and shared libraries are position independent and can be placed in randomized memory locations.
At fs/binfmt_elf.c the kernel only accepts those two types of ELF files:
/* First of all, some simple consistency checks */
if (interp_elf_ex->e_type != ET_EXEC &&
interp_elf_ex->e_type != ET_DYN)
goto out;
Then, only for ET_DYN
does it set the load_bias
to something that is not zero. The load_bias
is then what determines the ELF offset: How is the address of the text section of a PIE executable determined in Linux?
/*
* If we are loading ET_EXEC or we have already performed
* the ET_DYN load_addr calculations, proceed normally.
*/
if (loc->elf_ex.e_type == ET_EXEC || load_addr_set) {
elf_flags |= elf_fixed;
} else if (loc->elf_ex.e_type == ET_DYN) {
/*
* This logic is run once for the first LOAD Program
* Header for ET_DYN binaries to calculate the
* randomization (load_bias) for all the LOAD
* Program Headers, and to calculate the entire
* size of the ELF mapping (total_size). (Note that
* load_addr_set is set to true later once the
* initial mapping is performed.)
*
* There are effectively two types of ET_DYN
* binaries: programs (i.e. PIE: ET_DYN with INTERP)
* and loaders (ET_DYN without INTERP, since they
* _are_ the ELF interpreter). The loaders must
* be loaded away from programs since the program
* may otherwise collide with the loader (especially
* for ET_EXEC which does not have a randomized
* position). For example to handle invocations of
* "./ld.so someprog" to test out a new version of
* the loader, the subsequent program that the
* loader loads must avoid the loader itself, so
* they cannot share the same load range. Sufficient
* room for the brk must be allocated with the
* loader as well, since brk must be available with
* the loader.
*
* Therefore, programs are loaded offset from
* ELF_ET_DYN_BASE and loaders are loaded into the
* independently randomized mmap region (0 load_bias
* without MAP_FIXED).
*/
if (elf_interpreter) {
load_bias = ELF_ET_DYN_BASE;
if (current->flags & PF_RANDOMIZE)
load_bias += arch_mmap_rnd();
elf_flags |= elf_fixed;
} else
load_bias = 0;
I confirm this experimentally at: What is the -fPIE option for position-independent executables in gcc and ld?
file
5.36 behaviour breakdown
After studying how file
works from its source. We will conclude that:
- if
Elf32_Ehdr.e_type == ET_EXEC
- print
executable
- print
- else if
Elf32_Ehdr.e_type == ET_DYN
- if
DT_FLAGS_1
dynamic section entry is present- if
DF_1_PIE
is set inDT_FLAGS_1
:- print
pie executable
- print
- else
- print
shared object
- print
- if
- else
- if file is executable by user, group or others
- print
pie executable
- print
- else
- print
shared object
- print
- if file is executable by user, group or others
- if
And here are some experiments that confirm that:
Executable generation ELF type DT_FLAGS_1 DF_1_PIE chdmod +x file 5.36
--------------------------- -------- ---------- -------- -------------- --------------
gcc -fpie -pie ET_DYN y y y pie executable
gcc -fno-pie -no-pie ET_EXEC n n y executable
gcc -shared ET_DYN n n y pie executable
gcc -shared ET_DYN n n n shared object
ld ET_EXEC n n y executable
ld -pie --dynamic-linker ET_DYN y y y pie executable
ld -pie --no-dynamic-linker ET_DYN y y y pie executable
Tested in Ubuntu 18.10, GCC 8.2.0, Binutils 2.31.1.
The full test example for for each type of experiment is described at:
gcc -pie
andgcc -no-pie
: What is the -fPIE option for position-independent executables in gcc and ld?Keep in mind that
-pie
is set on by default since Ubuntu 17.10, related: 32-bit absolute addresses no longer allowed in x86-64 Linux?gcc -shared
(.so
shared library): https://github.com/cirosantilli/cpp-cheat/tree/b80ccb4a842db52d719a16d3716b02b684ebbf11/shared_library/basicld
experiments: How to create a statically linked position independent executable ELF in Linux?
ELF type
and DF_1_PIE
are determined respectively with:
readelf --file-header main.out | grep Type
readelf --dynamic main.out | grep FLAGS_1
file
5.36 source code analysis
The key file to analyse is magic/Magdir/elf.
This magic format determines file types depending only on the values of bytes at fixed positions.
The format itself is documented at:
man 5 magic
So at this point you will want to have the following documents handy:
- http://www.sco.com/developers/devspecs/gabi41.pdf ELF standard at the ELF header section
- http://www.cirosantilli.com/elf-hello-world/#elf-header my ELF file format introduction and breakdown
Towards the end of the file, we see:
0 string \177ELF ELF
!:strength *2
>4 byte 0 invalid class
>4 byte 1 32-bit
>4 byte 2 64-bit
>5 byte 0 invalid byte order
>5 byte 1 LSB
>>0 use elf-le
>5 byte 2 MSB
>>0 use \^elf-le
\177ELF
are the 4 magic bytes at the start of every ELF file. \177
is the octal for 0x7F
.
Then by comparing with the Elf32_Ehdr
struct from the standard, we see that byte 4 (the 5th byte, the first one after the magic identifier), determines the ELF class:
e_ident[EI_CLASSELFCLASS]
and some of its possible values are:
ELFCLASS32 1
ELFCLASS64 2
In file
source then, we have:
1 32-bit
2 64-bit
and 32-bit
and 64-bit
are the strings that file
outputs to stdout!
So now we search for shared object
in that file, and we are led to:
0 name elf-le
>16 leshort 0 no file type,
!:mime application/octet-stream
>16 leshort 1 relocatable,
!:mime application/x-object
>16 leshort 2 executable,
!:mime application/x-executable
>16 leshort 3 ${x?pie executable:shared object},
So this elf-le
is some kind of identifier that gets included on the previous part of the code.
Byte 16 is exactly the ELF type:
Elf32_Ehdr.e_type
and some of its values are:
ET_EXEC 2
ET_DYN 3
Therefore, ET_EXEC
always gets printed as executable
.
ET_DYN
however has two possibilities depending on ${x
:
pie executable
shared object
${x
asks: are the file executable or not by either user, group or other? If yes, show pie executable
, else shared object
.
This expansion is done in the varexpand
function in src/softmagic.c
:
static int
varexpand(struct magic_set *ms, char *buf, size_t len, const char *str)
{
[...]
case 'x':
if (ms->mode & 0111) {
ptr = t;
l = et - t;
} else {
ptr = e;
l = ee - e;
}
break;
There is, however, one more hack! In src/readelf.c
function dodynamic
, if the DT_FLAGS_1
flags entry of the dynamic section (PT_DYNAMIC
) is present, then the permissions in st->mode
are overridden by the presence or absence of the DF_1_PIE
flag:
case DT_FLAGS_1:
if (xdh_val & DF_1_PIE)
ms->mode |= 0111;
else
ms->mode &= ~0111;
break;
The bug in 5.34 is that the initial code was written as:
if (xdh_val == DF_1_PIE)
which means that if another flag was set, which GCC does by default due to DF_1_NOW
, the executable showed as shared object
.
The DT_FLAGS_1
flags entry is not described in the ELF standard so it must be a Binutils extension.
That flag has no uses in the Linux kernel 5.0 or glibc 2.27, so I seems to be purely informative to indicate that a file is PIE or not.