Drupal - Expose custom entity to views

You need to implement hook_views_data() just like in Drupal 7. See comment.views.inc.

This is now possible via a class annotation; per the documentation for the hook mentioned in the other answer:

To provide views data for an entity, instead of implementing this hook, create a class implementing \Drupal\views\EntityViewsDataInterface and reference this in the "views" annotation in the entity class. The return value of the getViewsData() method on the interface is the same as this hook.

You should extend EntityViewsData and you'll have the integration of your entity.

Example:

use Drupal\views\EntityViewsData;

class CustomEntityViewsData extends EntityViewsData {
  public function getViewsData() {
    $data = parent::getViewsData();
    return $data;
  }
}