How does the Sort Order of Products in Subcategories work?
Take a look at Mage_Catalog_Model_Resource_Category_Indexer_Product::_refreshAnchorRelations
specially these lines:
$position = 'MIN('.
$adapter->getCheckSql(
'cp.category_id = ce.entity_id',
'cp.position',
'(cc.position + 1) * ('.$adapter->quoteIdentifier('cc.level').' + 1) * 10000 + cp.position'
)
.')';
cc
and ce
are the same catalog_category_entity
table and cp
is catalog_category_product
table.
A product can be in more than one child categories so the product position is a minimum between multiple values.
So basically the product position in the parent category is the minimum between the positions relative to each categories from the table catalog_category_product
following this formula:
(child category position + 1) * (child category level + 1) * 10000 + ( product position in catalog_category_product + 1)
[EDIT]
Conclusion: products in a category with a position higher in the tree will be shown before the ones below.