How to query (price) information for multiple in-app products at once?

It is possible now with Billing API v3. You can get information with getSkuDetails() method. Example is here.

ArrayList skuList = new ArrayList();
skuList.add("premiumUpgrade"); 
skuList.add("gas");
Bundle querySkus = new Bundle();
querySkus.putStringArrayList("ITEM_ID_LIST", skuList);

Bundle skuDetails = mService.getSkuDetails(3, getPackageName(), "inapp", querySkus);

int response = skuDetails.getInt("RESPONSE_CODE");
if (response == 0) {
    ArrayList responseList = skuDetails.getStringArrayList("DETAILS_LIST");

    for (String thisResponse : responseList) {
        JSONObject object = new JSONObject(thisResponse);
        String sku = object.getString("productId");
        String price = object.getString("price");
        if (sku.equals("premiumUpgrade")) {
            mPremiumUpgradePrice = price;
        } else if (sku.equals("gas")) { 
            mGasPrice = price;
        }
    }
}