Azure CosmosDB - Partition key reached maximum size of 10 GB
UPDATE - 11 May, 2020
Microsoft has recently increased the capacity of a logical partition from 10 GB to 20 GB. Please see this for more details: https://docs.microsoft.com/en-us/azure/cosmos-db/concepts-limits
The reason you're getting this error is because even though unlimited collection (a.k.a. partitioned collection) does not have size restriction a partition in that collection have and that is currently 10 GB. Since you have reached that limit for your partition, you're getting this error. From this link (Question 6):
It is important to choose a partition key property that has a number of distinct values, and lets you distribute your workload evenly across these values. As a natural artifact of partitioning, requests involving the same partition key are limited by the maximum throughput of a single partition. Additionally, the storage size for documents belonging to the same partition key is limited to 10GB. An ideal partition key is one that appears frequently as a filter in your queries and has sufficient cardinality to ensure your solution is scalable.
Only solution that I could think of is to recreate the collection and choose a partition key that you know will not exceed this 10 GB limit. You will need to transfer data from your old to new collection as well.
You may find this post useful in choosing a partition key for your collection: https://docs.microsoft.com/en-us/azure/cosmos-db/partition-data#design-for-partitioning.
Furthermore, per this blog post, the minimum RU/s for an unlimited collection is now 1000 instead of 2500.
UPDATE - 11 May, 2020
Microsoft has recently increased the capacity of a logical partition from 10 GB to 20 GB. Please see this for more details: https://docs.microsoft.com/en-us/azure/cosmos-db/concepts-limits
I emailed Aravind Krishna, who is an engineer on the Azure Cosmos DB team and asked for clarification on this point. This is a summary of his answer:
In Cosmos DB, there are physical and logical partitions. Within a Collection, all documents that share the same value for the partition key will live within the same logical partition. One or more logical partitions occupy a physical partition. As developers, the physical partitioning is irrelevant; we only have control over what belongs in a logical partition.
Regardless of whether a Collection is Fixed (10GB) or Unlimited, the 10GB limit applies to a logical partition. Period.
So Sarva, you will need to either rethink your partition key or implement rolling logs to ensure that data within your debug log partition doesn't exceed the 10GB partition limit.
Data in CosmosDB is stored in collections which are partitioned using a partition key that you choose. Each individual partition is limited to 10GB of data.
Unlimited collections are unlimited in the amount of partitions, but a fixed collection limits you to a single partition. That's the only difference. That's why a fixed collection doesn't need a partition key set (because everything goes to the single partition) and why the limits for a fixed collection and a partition are the same size of 10GB.
How these partitions are actually stored on the CosmosDB servers is infrastructure that you do not have control over, and it's automatically managed for replication and performance. You can store as much data as you want in a collection as long as you can split it amongst enough partitions. It is better to have more partitions than fewer and even picking the ID field (to put a single record in every partition) will work fine. CosmosDB can automatically group partitions together to fill a server but it cannot split a partition apart, so be wary of that in your design.