AWS Error Message: InvalidInstanceID.NotFound

We run a service (Qubole) that frequently spawns and then tags (and in some cases terminates) AWS instances immediately.

We have found that Amazon will, every once in a while, claim an instanceid as invalid - even though it has just created it. Retrying a few times with some sleep time thrown in usually solves the problem. Even a total retry interval of 15s proved insufficient in rare cases.

This experience comes from the useast region. We do not make api calls to different regions - so that is not an explanation. More likely - this is the infamous eventual consistency at work - where AWS is unable to provide read-after-write consistency for these api calls.

AWS Error Message: The instance ID 'i-2b97ac2f' does not exist

You'll have to take the AWS response for granted here, i.e. the instance does not exist ;)

But seriously: Presumably you have already verified that you are actually running an instance with this ID in your account? Then this is most likely caused by targeting the wrong API endpoint, insofar an instance ID is only valid within a specific region (if not specified, the region defaults to 'us-east-1', see below).

In this case you need to specify the actual instance region via the setEndpoint() method of the AmazonEC2Client object within the apparently global ec2 variable before calling startInstances().

There are some examples regarding Using Regions with the AWS SDKs and all currently available AWS regional endpoint URLs are listed in Regions and Endpoints, specifically the Amazon Elastic Compute Cloud (EC2) defaults to 'us-east-1':

If you just specify the general endpoint (ec2.amazonaws.com), Amazon EC2 directs your request to the us-east-1 endpoint.