Drupal - Rendering arbitrary field values without a parent entity in Drupal 8

Yeah, you can't get around having an entity object, but it doesn't matter what it is, the only place where it's needed in the following code is FormatterBase::view(), to set #entity_type, which will later be used for the class name.

<?php

use Drupal\Core\Field\FieldDefinition;
use Drupal\node\Entity\Node;

// Load the node, you can also create a temporary object or have your own
// dummy object that implements EntityInterface. (Might need ContentEntityInterface, not sure).
$node = Node::load(1);

// Create the field definition, some might need more settings, it currently
// doesn't load in the field type defaults. https://drupal.org/node/2116341
// Field name is only set to avoid broken CSS classes.
$definition = FieldDefinition::create('entity_reference')
  ->setName('whatever')
  ->setSetting('target_type', 'node');

/* @var \Drupal\Core\Field\FieldItemListInterface $items $items */
// Create a field item list object, 1 is the value, array('target_id' => 1)
// would work too, or multiple values. 1 is passed down from the list to the
// field item, which knows that an integer is the ID.
$items = \Drupal::typedDataManager()->create($definition, 1, $definition->getName(), $node);

/* @var \Drupal\Core\Field\FormatterInterface $formatter */
// Create the formatter plugin. Will use the default formatter for that field
// type if none is passed.
$formatter = \Drupal::service('plugin.manager.field.formatter')->getInstance(array(
  'field_definition' => $definition,
  'view_mode' => 'default',
  'configuration' => array(
    'label' => 'hidden',
    'type' => 'entityreference_label',
    'settings' => array('link' => TRUE),
  )
));

// Prepare, expects an array of items, keyed by parent entity ID, not sure if
// actually used, just array($items) worked too.
$formatter->prepareView(array($node->id() => $items));
print \Drupal::service('renderer')->renderRoot($formatter->view($items));

Yes, finding this out is not trivial, but I disagree with you that the 7.x code is relatively easy, it's just that you know the 7.x field API and internal magic callbacks very well. The main difference is that instead of building arbitrary array structures and passing them around, you need to find the right object/class/service and call methods on there. Once you found them, it's not that hard, FormatterPluginManager::getInstance() for example documents very well what it expects.