How to get a list of all the blobs in a container in Azure?

There is a sample of how to list all of the blobs in a container at https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/#list-the-blobs-in-a-container:

// Retrieve the connection string for use with the application. The storage
// connection string is stored in an environment variable on the machine
// running the application called AZURE_STORAGE_CONNECTION_STRING. If the
// environment variable is created after the application is launched in a
// console or with Visual Studio, the shell or application needs to be closed
// and reloaded to take the environment variable into account.
string connectionString = Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING");

// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);

// Get the container client object
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("yourContainerName");

// List all blobs in the container
await foreach (BlobItem blobItem in containerClient.GetBlobsAsync())
{
    Console.WriteLine("\t" + blobItem.Name);
}    

Using the new package Azure.Storage.Blobs

BlobServiceClient blobServiceClient = new BlobServiceClient("YourStorageConnectionString");
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("YourContainerName");
var blobs = containerClient.GetBlobs();

foreach (var item in blobs){
    Console.WriteLine(item.Name);
}

Since you container name is $logs, so i think your blob type is append blob. Here's a method to get all blobs and return IEnumerable:

    private static CloudBlobClient _blobClient = CloudStorageAccount.Parse("connectionstring").CreateCloudBlobClient();
    public IEnumerable<CloudAppendBlob> GetBlobs()
    {
        var container = _blobClient.GetContainerReference("$logs");
        BlobContinuationToken continuationToken = null;

        do
        {
            var response = container.ListBlobsSegmented(string.Empty, true, BlobListingDetails.None, new int?(), continuationToken, null, null);
            continuationToken = response.ContinuationToken;
            foreach (var blob in response.Results.OfType<CloudAppendBlob>())
            {
                yield return blob;
            }
        } while (continuationToken != null);
    }

The method can be asynchronous, just use ListBlobsSegmentedAsync. One thing you need to note is the argument useFlatBlobListing need to be true which means that ListBlobs will return a flat list of files as opposed to a hierarchical list.


Here's the updated API call for WindowsAzure.Storage v9.0:

private static CloudBlobClient _blobClient = CloudStorageAccount.Parse("connectionstring").CreateCloudBlobClient();

public async Task<IEnumerable<CloudAppendBlob>> GetBlobs()
{
    var container = _blobClient.GetContainerReference("$logs");
    BlobContinuationToken continuationToken = null;

    //Use maxResultsPerQuery to limit the number of results per query as desired. `null` will have the query return the entire contents of the blob container
    int? maxResultsPerQuery = null;

    do
    {
        var response = await container.ListBlobsSegmentedAsync(string.Empty, true, BlobListingDetails.None, maxResultsPerQuery, continuationToken, null, null);
        continuationToken = response.ContinuationToken;

        foreach (var blob in response.Results.OfType<CloudAppendBlob>())
        {
            yield return blob;
        }
    } while (continuationToken != null);
}

Update for IAsyncEnumerable

IAsyncEnumerable is now available in .NET Standard 2.1 and .NET Core 3.0

private static CloudBlobClient _blobClient = CloudStorageAccount.Parse("connectionstring").CreateCloudBlobClient();

public async IAsyncEnumerable<CloudAppendBlob> GetBlobs()
{
    var container = _blobClient.GetContainerReference("$logs");
    BlobContinuationToken continuationToken = null;

    //Use maxResultsPerQuery to limit the number of results per query as desired. `null` will have the query return the entire contents of the blob container
    int? maxResultsPerQuery = null;

    do
    {
        var response = await container.ListBlobsSegmentedAsync(string.Empty, true, BlobListingDetails.None, maxResultsPerQuery, continuationToken, null, null);
        continuationToken = response.ContinuationToken;

        foreach (var blob in response.Results.OfType<CloudAppendBlob>())
        {
            yield return blob;
        }
    } while (continuationToken != null);
}

Tags:

C#

Cloud

Azure