Frequent HTTP 500 Internal Errors with Google Drive drive.files.get API

That's approximately the same error rate I see. Just do an exponential backoff and retry.

I used tenacity to retry the request. First install it with

pip install tenacity

then

from tenacity import retry, retry_if_exception_type, wait_exponential, stop_after_attempt
from googleapiclient.errors import HttpError

@retry(reraise=True, retry=retry_if_exception_type(HttpError),
       wait=wait_exponential(),
       stop=stop_after_attempt(5))
def function_that_calls_the_google_drive_api():
    pass

Because Google infrastructure is complex, large scale and distributed it is close to impossible to have a 0% error rate - servers or hard disks dying during the request, unexpected timeouts between servers internally, datacenter outage or increased load, tentative DOS attacks, misbehaving applications... - all of which might raise the 500's error rate - so as a general good practice, implementing an exponential backoff and retry strategy on your end is good when you deal with Web APIs and actually it is almost mandatory if you want to offer a reliable service, also on your end it might smooth out temporary network glitch etc...

Now 0.5% is a bit high, I believe the global error rate is lower in average but I'm going to bring it up to the Drive team so they investigate and try to reduce this (sometimes it's just about increasing a timeout to one of our server dependencies). We're always taking passes at reducing the error rate but sometimes we have to spend time building new features especially when the products are rather new :)