How can Android encryption be so fast?

Encryption happens in memory, not on the disk.

You are misunderstanding how disk encryption works. It does not read the entire disk and replace it with a decrypted version. Rather, when a file or sector of encrypted data is accessed, it is read into memory and decrypted in memory. Likewise, when writing to the disk, data is encrypted in memory before being saved to persistent storage. Operating systems keep copies of data that have been read or is to be written in memory (the filesystem buffer) as a performance optimization. It is in this memory that the encryption and decryption take place. This allows data to be read from the disk and decrypted once, but subsequently accessed many times from memory. Incidentally, using memory to store frequently-accessed files is why so many people mistakenly think Linux eats too much RAM.

I also want to point out that the bottleneck is often I/O, not encryption. You say that encrypting gigabytes of data takes a long time, but on most modern machines (including mobile devices), encrypting gigabytes of data can take only a few seconds (especially with hardware acceleration, encryption is really, really fast). However the solid-state drive in most modern Android devices is unable to read or write data at nearly those speeds. So no matter how fast you are trying to read or write data to the disk, the bottleneck will generally always be I/O, not encryption.

Older hardware often did suffer reduced performance when encryption was in use. This was because, at that time, storage speeds were improving faster than processor speeds. The lack of dedicated hardware acceleration for cryptography and the inefficient algorithms often caused a noticeable slowdown when accessing the disk. On modern systems, this is reversed. The processor is so fast that the storage device struggles to keep up. Any overhead is negligible.

It doesn't decrypt the full disk at startup. Instead, it continously decrypts data as it is read from the disk when it is read from the disk, leaving the actual content on the disk untouched and encrypted. Since decryption is fast in comparison to disk reads, it shouldn't affect the reading speed significantly. Of course, it adds (marginally) to overall load, though.

There's no reason to decrypt gigabytes of files at startup. Android only needs to decrypt exact files it needs to load in memory on startup.

FDE in Android doesn't encrypt the kernel, only the user data partition. You can read more on the official site.

Modern hardware-accelerated encryption is quite fast and doesn't slow the system down too much. Especially when it's only user files that are encrypted with FDE, and they aren't really speed-critical. There are other drawbacks, actually listed on the linked page, but speed is not the issue.

Android phones couldn't decrypt the whole partition on startup even if they wanted to - there's nowhere to store it.