How to use ExecutorService to poll until a result arrives

I think CompletableFutures are a fine way to do this:

ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();

private void run() {
    final Object jobResult = pollForCompletion("jobId1")
            .thenApply(jobId -> remoteServer.getJobResult(jobId))
            .get();

}

private CompletableFuture<String> pollForCompletion(final String jobId) {
    CompletableFuture<String> completionFuture = new CompletableFuture<>();
    final ScheduledFuture<Void> checkFuture = executor.scheduleAtFixedRate(() -> {
        if (remoteServer.isJobDone(jobId)) {
            completionFuture.complete(jobId);
        }
    }, 0, 10, TimeUnit.SECONDS);
    completionFuture.whenComplete((result, thrown) -> {
        checkFuture.cancel(true);
    });
    return completionFuture;
}

it seems to me you are more worried by some stylistic problems than any others. in java 8, CompletableFuture has 2 roles: one is the traditional future, which gives an asynchronous source for task execution and status query; the other is what we usually call a promise. a promise, if you don't know yet, can be considered a builder of future and its completion source. so in this case, intuitively a promise is required, which is the exact case you are using here. the examples you are worrying about is something that introduces you the first usage, but not the promise way.

accepting this, it should be easier for you to start dealing with your actual problem. i think the promise is supposed to have 2 roles, one is to notify your task completion of polling, and the other is to cancel your scheduled task on completion. here should be the final solution:

public CompletableFuture<Object> pollTask(int jobId) {
    CompletableFuture<Object> fut = new CompletableFuture<>();
    ScheduledFuture<?> sfuture = executor.scheduleWithFixedDelay(() -> _poll(jobId, fut), 0, 10, TimeUnit.SECONDS);
    fut.thenAccept(ignore -> sfuture.cancel(false));
    return fut;
}

private void _poll(int jobId, CompletableFuture<Object> fut) {
    // whatever polls
    if (isDone) {
        fut.complete(yourResult);
    }
}