How do I enable Google Cloud Storage on my local development server?

For those trying to get Google Cloud Storage to work at all from their local Java development app server, I thought one more answer would be helpful. I managed to get my local dev app server working with non-local Google Cloud Storage, but only be digging through the code and figuring out what was needed - there doesn't appear to be documentation on this.

The goal will be to get this block of code to work locally, which reads a file from GCS:

    GcsService gcsService =
       GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
    int fileSize = (int) gcsService.getMetadata(gcsFilename).getLength();
    ByteBuffer byteBuffer = ByteBuffer.allocate(fileSize);
    GcsInputChannel inputChannel = gcsService.openReadChannel(gcsFilename, 0);
    int readResult = inputChannel.read(byteBuffer);
    byte[] fileBytes = byteBuffer.array();

If you try to do this locally, you won't find any files you've upload to GCS, because it will be trying to use a fake local GCS. Unfortunately, I haven't found a good way to upload to this local GCS, so that isn't very useful (there is no file explorer for it as there is in the cloud version, and gsutil doesn't work for it). So instead, we're going to get it to work with non-local (cloud) GCS when running in the local dev app server.

To do that, note that GcsService is created in com.google.appengine.tools.cloudstorage.GcsServiceFactory by this block of code:

if (location == SystemProperty.Environment.Value.Production || hasCustomAccessTokenProvider()) {
  rawGcsService = OauthRawGcsServiceFactory.createOauthRawGcsService(builder.build());
} else if (location == SystemProperty.Environment.Value.Development) {
  rawGcsService = LocalRawGcsServiceFactory.createLocalRawGcsService();

The above says that you need to specify a custom access token provider to get the non-local service, which you do by defining a system property. For an app engine app, you can do that in appengine-web.xml like this:

<system-properties>
    <property name="gcs_access_token_provider" value="com.mypackage.MyAccessTokenProvider" />
</system-properties>

This value of that property is a class you define that implements com.google.appengine.tools.cloudstorage.oauth.AccessTokenProvider, which provides the access token for your app. That class needs to create a GoogleCredential, which can be used to fetch an access token, using the instructions for "Other" for a GoogleCredential on https://developers.google.com/identity/protocols/OAuth2ServiceAccount#authorizingrequests.

Now it will create an OAuth GcsService that talks to the cloud, and you don't need to use the fake local storage.

Turns out you don't need to perform any setup at all. I just assumed there was one with particular name when uploading using the blobstore and one was created for me automatically.

Incidentally, it does not seem to be documented anywhere how you can browse files in the storage of the development server. You can do it by selecting the __GsFileInfo__ entity in the Datastore Viewer admin access to your local dev server.

You need to download and integrate the Google Cloud Storage Client Library for App Engine.

This library provides you the GcsService which is similar to the BlobstoreService, so you can write a file, read a file, delete a file and other functions provided from Cloud Storage

When you use the code in the development environment, the uploaded files are stored in the appengine-generated folder and an __GsFileInfo__ entity is created in the local datastore which preserves the metadata attached to the file

This library works online too, so your code will work for both development and production environments.

Here you can find the Getting Started guide and the full JavaDoc reference