Google Datastore Emulator using Java (Not using GAE)

I assume that you want to test against the Datastore Emulator. In that case it is not necessary to start the Datastore Emulator from the shell. There is a LocalDatastoreHelper in the gcloud library that allows you to create, start, reset and stop a local Datastore Emulator with ease.

I didn't found any documentation on that, so I created this testcases for you:

import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.KeyFactory;
import com.google.cloud.datastore.testing.LocalDatastoreHelper;
import org.junit.*;

import java.io.IOException;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertNull;

/**
 * This testcase demonstrate the use of the datastore emulator in JUnit test cases.
 *
 * from @link https://www.kontaktlinsen-preisvergleich.de
 */
public class DatastoreEmulatorTest {

    protected static LocalDatastoreHelper localDatastoreHelper;

    protected Datastore datastore;

    protected KeyFactory keyFactory;

    @BeforeClass
    public static void setUpClass() throws InterruptedException, IOException {
        // create and start a local datastore emulator on a random free port
        // this also means that you probably can run tests like this concurrently.
        System.out.println("[Datastore-Emulator] start");
        localDatastoreHelper = LocalDatastoreHelper.create();
        localDatastoreHelper.start();
        System.out.println("[Datastore-Emulator] listening on port: " + localDatastoreHelper.getPort());

        // set the system property to tell the gcloud lib to use the datastore emulator
        System.setProperty("DATASTORE_EMULATOR_HOST","localhost:" + localDatastoreHelper.getPort());
    }

    @Before
    public void setUp() {
        // create the datastore instance
        // because of the system property set it in setUpClass() this
        // datastore will be connected with the datastore emulator.
        datastore = DatastoreOptions.getDefaultInstance().getService();
        keyFactory = datastore.newKeyFactory().setKind("TestEntity");
    }

    @After
    public void tearDown() throws IOException {
        System.out.println("[Datastore-Emulator] reset");
        // this resets the datastore after every test
        localDatastoreHelper.reset();
    }

    @AfterClass
    public static void tearDownClass() throws InterruptedException, IOException {
        System.out.println("[Datastore-Emulator] stop");
        // this stops the datastore emulator after all tests are done
        localDatastoreHelper.stop();
    }

    @Test
    public void test1() {
        // stores an entity in the datastore and retrieves it later

        // create an Entity "TestEntity"
        Entity.Builder builder = Entity.newBuilder(keyFactory.newKey(42));
        builder.set("name", "Test1");

        // store it in datastore
        datastore.put(builder.build());

        // retrieve entity by key
        Entity entity = datastore.get(keyFactory.newKey(42));
        assertNotNull(entity);
        assertEquals("Test1", entity.getString("name"));
    }

    @Test
    public void test2() {
        // try to access the entity created in test1, shouldn't work because
        // of calling reset in tearDown() after each test.

        // try to retrieve entity by key
        Entity entity = datastore.get(keyFactory.newKey(42));
        assertNull(entity);
    }
}

The LocalDatastoreHelper creates a Datastore Emulator instance on a free port and doesn't store to disk - when you stop the testcase with the debugger and look for processes, you'll find something like this:

$ ps ax | grep CloudDatastore
2614   ??  R      0:01.39 /usr/bin/java -cp /Users/marco/google-cloud-sdk/platform/cloud-datastore-emulator/CloudDatastore.jar com.google.cloud.datastore.emulator.CloudDatastore /Users/marco/google-cloud-sdk/platform/cloud-datastore-emulator/cloud_datastore_emulator start --host=localhost --port=57640 --store_on_disk=False --consistency=0.9 --allow_remote_shutdown /var/folders/ky/c126qk_161159ltyrbpdxv8w0000gn/T/gcd2141205756617995044

That also means you should also be able to run tests in parallel, too.

The line below always connects to the remote datastore. Uses the default options (e.g. project, auth credentials) from gcloud settings.

Datastore datastore = DatastoreOptions.defaultInstance().service();

To connect to the local datastore, try the below:

@Test
public void test1() throws IOException, InterruptedException {
    Datastore ds = DatastoreOptions.builder().host("http://localhost:9999").projectId("my-project").build().service();
    com.google.cloud.datastore.Key key = ds.newKeyFactory().kind("MyEntity").newKey("mykey");
    com.google.cloud.datastore.Entity entity = com.google.cloud.datastore.Entity.builder(key).set("p1", "Hello World!").build();
    entity = ds.put(entity);
    entity = ds.get(key);
    System.out.println(entity);
}

I started my Datastore Emulator on localhost:9999. Set that as the host when building the DatastoreOptions.

I've confirmed that the Emulator console shows requests are received and entities are persisted. I've also checked the data file (local_db.bin) and it shows the data (of course it is not a plain text file).

The one thing I don't know is - if there is a way to manage the local datastore using a browser interface. I could not find much documentation on how to administer the local datastore just like how we do the remote one from Cloud Console. Perhaps someone else can help with this.