Sorting and limiting MongoDB queries by insertion order in a multi-process environment
Using ObjectIds or a date field to sort on may not give you the results you are looking for. ObjectIds and dates in inserted documents are generated client-side, so if you are running with connections from multiple machines you will run into ordering inconsistencies unless the timing between your machines is perfect.
Can you provide more details on what you are trying to do? There are a few different ways to get the behavior you want from MongoDB, depending on why you need a list of documents inserted after a specific document.
If, for instance, you are trying to use that ordered list of documents as a kind of queue, you could instead use a findAndModify command to fetch an unread document and atomically update a “read” field to guarantee you don’t read it twice. Each call to findAndModify would find the most recent document in the collection without a read field set to true, atomically set that field to true, and return your document to the client for processing.
On the other hand, if your use case really does require a list of documents in inserted order, you can exploit the natural ordering of inserted documents. In MongoDB, documents are written to disk in order of insertion unless changes in document size or deletions require things to move around. By using a capped collection, which is guaranteed to maintain natural ordering, you could feasibly get your list of documents by exploiting this. Please note there are several major restrictions that come with using capped collections, which you can find spelled out in the documentation.
To ensure insertion order one need an unique auto-incrementing sequence field. There are two recommended ways to implement this:
- Counters Collection
- Optimistic Loop
One thing to note which the documentation does not state: Only the Optimistic Loop does really ensure that insertion order equals sequence order. Still it's very likely for the Counters Collection approach, but it is not guaranteed by theory since the incrementation and the document insertion are two separate operations.