How can WhatsApp restore local or Google Drive Backups?
First assumption: The backup key is saved on WhatsApp servers too. Otherwise a local phone to phone backup would not work?
TL;DR: Yes, after some investigation, this seems to be the case.
Secondary devices
The protocol is quite complicated and not limited to WhatsApp, but works generally like this; The phone that uses chat app is called the first device. This device is the almighty device storing all sensitive data like private identity keys, connection passwords, local encryption keys, attachment encryption keys etc. All other devices are called second devices, this includes WhatsApp web and desktop clients. These clients are essentially dumb.
Whenever a new device appears it needs to be authenticated via the first device first. WhatsApp does this using an initial QR-code (other apps use SMS tokens), and then via a set of proofs. Eventually the first device decides if a secondary device is allowed control over the sensitive data. If so, the second device receives these keys on request. With these keys the second device can download the contact file and decrypt it using the requested key. Same works for all media messages. A note for WhatsApp specific: these key requests live very short, and thus are secondary clients required to regularly ask the first device for a new key to access the data. This causes the annoying 'Phone not connected' alert in WhatsApp web.
Whenever you install WhatsApp on a new phone, this entire process will take place and the first device will authorize the new phone and immediately locks out one of the two active instances. You then have the choice to choose one of the phones. If you choose the new phone, ownership of all secrets will be transferred to the second device, which then becomes the first device which makes the circle complete.
Backups
After some investigation, decompiling of the app and running test scenarios, we can conduct the following:
The app is using several local (SQLite) databases. Messages are stored in these databases unencrypted, and the databases itself are not encrypted either. You can check this yourself by downloading the .db files from the data/ folder. This is the default storage location for WhatsApp in running mode, and is unencrypted for performance reasons most likely. Normal apps should not be able to access the .db files in the data folder, but there are quite a few adb workarounds.
Database files including settings and messages are backuped in Google Drive (if you choose to do so). The app can request a lock on the folder in Google Drive to prevent users from downloading or accessing the backup. The message databases send to the remote backup location are the msgstore.db.crypt[0-12] files, where the last number denotes the protocol version. The databases are encrypted with a key stored in the data folder. This key is in fact stored on the WhatsApp server.
Second assumption: So the worst part is that if you backup on Google Drive, WhatsApp has theoretically access (?) to your (hopefully encrypted) backup and also access to the en-/decryption key on their servers. Or is there at least a separation between Google Drive not readable by WhatsApp itself? Does somebody has more details?
The moment you install a new device and setup your Google Account, the files can be requested by the app and configured on the local device. This includes the axolotl database containing the identity key which is necessary in order to prove your identity to others. The decryption key is retrieved after the proving ownership of the phone number (username) to the newly installed WhatsApp instance. In theory, WhatsApp should not be able to access those files, but only you and Google. Ofcourse, WhatsApp could download the files and send them to another location. But at some point we need to put trust in the app, especially if it's not opensource.
WhatsApp is making local backups as well, usually twice a day in sdcard/whatsapp. These local backups also contain the identity key and the message storage databases, and are in fact encrypted. Once again the encryption key is stored on the remote server together with your WhatsApp profile. This explains why you can move the entire WhatsApp backup folder from one device to another. Without a verified phone number you cannot read the backup files, or use the identity key, however any rooted device can give easy access to the original, unencrypted, database files.
One last word about end2end encryption protocol: It seems it is useless (not against normal hackers, but I think against US surveillance) when at least one of your friend will do a Google Drive backup of their chat (history of chats are retrievable).
The story does not get any better from this point, as we know that Google and other companies such as Facebook and WhatsApp supply files on request under the FISA act. There is no need to strike at the end-to-end communication as there is already a 'backdoor'. E2E only protects against active adversaries on the communication channel, who do not possess the power to demand backup files from party one and the decryption key from the other.
Window of Attack
Suppose we look at the situation from a non-government or 'normal' attacker with limited resources, then the device is the obvious weakness. Any app with root privileges can also access databases, can copy keys and so forth. The default Android ROMs contains many apps running under the system user, but also the vendor's apps shipped with the ROM (and updates) are protected from user intervention, and thus run as system.
Malicious apps are not without risk either. With the correct permissions they have full control over the sdcard storage, and can access the encrypted backups. When the verification SMS is intercepted at the correct moment (hooking in on the SMS receive call) and the phone number is copied, it should be possible to activate a self-controlled WhatsApp instance and to receive the database decryption key. The attack becomes even more plausible if the adversary has control over mobile communications (which governments often do).
The adb attack is even worse since it doesn't require root permissions. Basically a downgrade attack is possible where an older version of WhatsApp is installed via the bridge interface. This so called legacy WhatsApp can be tricked into a full application backup, resulting in a tar archive. The tarball is pulled to the adb server side and extracted. When properly prepared it would take an USB cable and a matter of seconds.
The AES-GCM-256 key is generated by WhatsApp server at registration time and is sent to the client. Client stores this key in /data/data/com.whatsapp/files/key. Daily chat backups are encrypted with this key. The 256-bit IV and 256-bit Id of the key is stored in msgstore.db.crypt12 itself. The Id is used to identify the key that encrypts the backup.
When a user sign in to new device, it retrieves the key from the server and decrypts the backup. That key is then reused again to encrypt daily chat backups. WhatsApp service might rotate the key for the client after some period of time. If the user doesn't want to restore the backup, then the new key is generated by the server. If you delete the key, new key is generated and sent to the client when you reopen the app.
Here's the filtered logs of whatsapp.log file when the client decrypts the backup. Information about each log is in the comments.
# Look for local and cloud backups
[main] registername/check-for-local-and-remote-backups
# Query Google Drive for backups if your device is logged in with Google account
[WhatsApp Worker] gdrive-api-v2/auth-request asking GoogleAuthUtil for auth token: [email protected]
[WhatsApp Worker] gdrive-api-v2/auth-request/received-auth-token
# Response from Google Drive: No backup exists
[WhatsApp Worker] gdrive-activity/one-time-setup/account-with-no-backup/[email protected]
# Found local backup, get last backup file
[WhatsApp Worker] msgstore/lastbackupfile/file msgstore.db.crypt12 size=10485760
# Number of local backup files can be max 9 in your local storage, the older ones are deleted
[main] gdrive-activity/one-time-setup/num-of-local-backup-files/2
[main] gdrive-activity/one-time-setup no google drive backups found but local backup exists.
# Check if the backup belongs to your account by comparing
# last 2 digits in the footer with the last 2 digits of your phone number
# If it mismatches, raise exception, jid = jabber Id
[WhatsApp Worker] BackupFooter/has-jid-user-mismatch/expected-jid-user-ends-with: 20 actual-jid-user: YourPhoneNumberAlongWithCountryCode
[main] gdrive-activity/show-restore-for-local-backup
[main] gdrive-activity/show-msgstore-downloading-view
[main] gdrive-activity/msgstore-download/not performed since we are using local, success: true, now, restoring it.
# Get keys from WhatsApp server based on number of backups
# This key is same for multiple backups
# But sometimes when a user starts from fresh that is without restoring the backup,
# the key is changed, see in next logs
# WhatsApp server might also rotate keys for the client after some period of time
# The key is AES-GCM-256-bit key whose IV is stored in msgstore.db.crypt12
backupencryption/getkeys/size 2 (backups=2)
# Get cipher key from the key file with its Id
# When msgstore.db.crypt12 is generated,
# it stores Id of key in its header
# This Id is also stored in the key file
# to retrieve the key later for decryption
# 2 function calls because of 2 backup files
# Key can be same for both
[WriterThread] xmpp/writer/write/get-cipher-key
[WriterThread] xmpp/writer/write/get-cipher-key
# Only the latest backup is restored
[WhatsApp Worker] msgstore/restore/backupfiles msgstore.db.crypt12 (10485760)
[WhatsApp Worker] BackupFile/verifyIntegrity/CRYPT12
# Initial digest is md5 hash of an empty string
[WhatsApp Worker] BackupFile/getFileDigestWithoutFooter/initial digest = d41d8cd98f00b204e9800998ecf8427e
# Get file digest without footer, the last 20 bytes of backup is the footer
[WhatsApp Worker] msgstore-integrity-checker/verify-integrity/actual-digest/ b90a162a06ce60baf7dd89301d2149d6
[WhatsApp Worker] BackupFooter/verify-integrity/actual-digest/ b90a162a06ce60baf7dd89301d2149d6
Expected digest is the md5 hash stored in footer
[WhatsApp Worker] BackupFooter/verify-integrity/expected-digest/b90a162a06ce60baf7dd89301d2149d6
[WhatsApp Worker] BackupFooter/verify-integrity/digest-matches/success
[WhatsApp Worker] msgstore/restore/file-integrity-check/success
[WhatsApp Worker] msgstore/restore/key CRYPT12
[WhatsApp Worker] msgstore/restore/jid-mismatch/false
[main] gdrive-activity/msg-restore-progress/100%
# If md5 hash mismatches
[WhatsApp Worker] BackupFooter/verify-integrity/actual-digest/ f1c5e3afc42867f10f4e9107faf15cd3
[WhatsApp Worker] BackupFooter/verify-integrity/expected-digest/b90a162a06ce60baf7dd89301d2149d6
[WhatsApp Worker] BackupFooter/verify-integrity/failed expected-digest:b90a162a06ce60baf7dd89301d2149d6 actual-digest:f1c5e3afc42867f10f4e9107faf15cd3
[WhatsApp Worker] msgstore/restore/file-integrity-check/failed
[WhatsApp Worker] msgstore/restore/key CRYPT12
[WhatsApp Worker] msgstore/restore/error; exception=java.util.zip.ZipException: incorrect header check
[main] gdrive-activity/after-msgstore-verified/failed/unrestorable-local-backup
# If you start from fresh without restoring your backup, or
# If you delete the key file from /data/data/com.whatsapp/files/key
# Client requests new key from the server
[WriterThread] xmpp/writer/write/create-cipher-key
# No internet connection
# If you delete the key file and try to make backup without going online
[WhatsApp Worker] localbackupmanager/sendCreateBackupKeyIfNeeded/started
[WhatsApp Worker] sendmethods/sendcreatecipherkey
[WhatsApp Worker] localbackupmanager/backup/waiting-for-the-key
[WhatsApp Worker] localbackupmanager/backup/backup-key-not-received
[WhatsApp Worker] backupkey/getinfo/does-not-exist
WhatsApp service doesn't have access to the backup that is saved on Google drive as it requires logged in Google account to request Oauth token. If WhatsApp really wants to, it can simply install a backdoor to pull unencrypted magstore.db from com.whatsapp/msgstore.db. However, law enforcement can order Google to grab the backup and ask WhatsApp to decrypt it for them.
Storing the encryption key on server is a poor security decision as it weakens the security by design offered by E2EE and shifts the attack vector on security by promise. Signal encrypts the backup with AES-CTR-256 derived from the randomly generated pasword with 250,000 rounds of SHA-512 which is a much better approach.