X86 assembly - Handling the IDIV instruction

The first part of Mysticials answer is correct, idiv does a 128/64 bit division, so the value of rdx, which holds the upper 64 bit from the dividend must not contain a random value. But a zero extension is the wrong way to go.

As you have signed variables, you need to sign extend rax to rdx:rax. There is a specific instruction for this, cqto (convert quad to oct) in AT&T and cqo in Intel syntax. AFAIK newer versions of gas accept both names.

movq    %rdx, %rbx
cqto                  # sign extend rax to rdx:rax
idivq   %rbx

The idivq instruction divides a 128-bit integer (rdx:rax) by the given source operand.

• rax holds the lower 64-bits of the dividend.
• rdx holds the upper 64-bits of the dividend.

When the quotient doesn't fit into 64-bits, idiv will fault (#DE exception, which the OS handles by delivering a SIGFPE signal as required by POSIX for arithmetic exceptions).

Since you're compiling code that uses signed int, you also need to sign extend rax to rdx:rax, that means copying the rax sign bit to every bit of rdx and is accomplished with cqo alias cqto:

movq    %rdx, %rbx        # or load into RBX or RCX in the first place
cqo
idivq   %rbx              # signed division of RDX:RAX / RBX

---

If you'd been doing unsigned division, you'd zero RDX to zero-extend RAX into RDX:RAX:

movq    %rdx, %rbx
xor     %edx, %edx      # zero "rdx"
divq    %rbx            # unsigned division of RDX:RAX / RBX

---

Also note that in the x86-64 System V ABI, int is a 32-bit signed type, not 64-bit. Widening it to 64-bit is legal in this case (because the result is the same) but makes your code slower, especially for division.