Drupal - Migrate failure: on clause is ambiguous

Figured it out!

When defining the field in the MigrateSQLMap I can set a table alias for the field:

$this->map = new MigrateSQLMap($this->machineName,
    array(
        'labelId' => array(
            'type' => 'int',
            'not null' => TRUE,
            'alias' => 'l', // it's the letter small case "L", not the digit 1
        )
    ),
    MigrateDestinationTerm::getKeySchema()
);

The 'alias' => 'l' makes the labelId become l.labelId in the queries.