Drupal - How can I programmatically display a block?

There are two types of blocks, and the method for rendering the two is a bit different:

Content Blocks

Content blocks are blocks that you create in the interface. They are much like nodes configurable data structures, with fields etc. If you want to render one of these, you can do what you would normally do with entities, load them and render them with the view builder:

$bid = ??? // Get the block id through config, SQL or some other means
$block = \Drupal\block_content\Entity\BlockContent::load($bid);
$render = \Drupal::entityTypeManager()->
  getViewBuilder('block_content')->view($block);
return $render;

Plugin blocks

Blocks can also be plugin, defined in various modules. An example could be the breadcrumb block. If you want to render these, you will need to use the block plugin manager.

$block_manager = \Drupal::service('plugin.manager.block');
// You can hard code configuration or you load from settings.
$config = [];
$plugin_block = $block_manager->createInstance('system_breadcrumb_block', $config);
// Some blocks might implement access check.
$access_result = $plugin_block->access(\Drupal::currentUser());
// Return empty render array if user doesn't have access.
// $access_result can be boolean or an AccessResult class
if (is_object($access_result) && $access_result->isForbidden() || is_bool($access_result) && !$access_result) {
  // You might need to add some cache tags/contexts.
  return [];
}
$render = $plugin_block->build();
// In some cases, you need to add the cache tags/context depending on
// the block implemention. As it's possible to add the cache tags and
// contexts in the render method and in ::getCacheTags and 
// ::getCacheContexts methods.
return $render;

Config entities

Shared for the two types are blocks, are that once you insert them into a region, you will create a config entity that has all of the settings for the block. In some cases it will be more useful handling config entities. Since the same block can be place in multiple regions with and with different configuration, it can get more tricky using the block config entities. The nice thing is that you might want to render a block with specific configuration, the bad thing is that config ids can change by messing with the interface, so the code might end up not working after letting users use the block interface.

$block = \Drupal\block\Entity\Block::load('config.id');
$render = \Drupal::entityTypeManager()
  ->getViewBuilder('block')
  ->view($block);
return $render;

For display only your block in your templates with preprocess the best way is

$block = \Drupal\block\Entity\Block::load('my_block_id');
$variables['My_region'] = \Drupal::entityManager()
          ->getViewBuilder('block')
          ->view($block);

And in your page.html.twig or node.html.twig or xxx.html.twig use your variable My_region like this :

{% if page.My_region %}
    {{ page.My_region }}
{% endif %}

And in renderable array (custom module) by example into an controller custom in content() :

public function content() {
    $block = \Drupal\block\Entity\Block::load('my_block_id');
    $block_content = \Drupal::entityManager()
      ->getViewBuilder('block')
      ->view($block);

          return array(
        '#type' => 'container',
        '#attributes' => array(
          'class' => array("Myclass"),
        ),
        "element-content" => $block_content,
        '#weight' => 0,
      );
}

Using drupal_render is not useful as Drupal already assume the render in D8 and this is deprecated. You should use \Drupal::service('renderer')->renderRoot() instead.

It's a bit heavy, it is better to use the maximum area system and does not add load block from the preprocess. In the case of using a controller in your modules this seems a justified use.

I needed to get the HTML of a custom block and got it using:

$con = \Drupal\block\BlockViewBuilder::lazyBuilder('bartik_search', 'full');
$d   = \Drupal::service('renderer')->renderPlain($con);

print $d->__toString();