The following modules couldn't be hot updated: (Full reload needed)

The problem, as mentioned by commenters, was missing in my loader - I'm not sure if this had anything to do with it, but I also switched to using babel after typescript - and having typescript compile to ES6. New config below:

var path = require('path')
var webpack = require('webpack')
var HtmlWebpackPlugin = require('html-webpack-plugin');

module.exports = {
    devtool: 'cheap-module-eval-source-map',
    entry: [
        'webpack-hot-middleware/client',
        path.resolve('./src/index.ts'), 
    ],
    output: {
        path: path.join(__dirname, 'dist'),
        filename: 'bundle.js'
    },
    plugins: [
        new webpack.optimize.OccurrenceOrderPlugin(),
        new webpack.HotModuleReplacementPlugin(),
        new HtmlWebpackPlugin({ template: path.resolve('./src/index.html') })
    ],
    module: {
        loaders: [
            {   test: /\.tsx?$/, 
                loaders: [
                    'react-hot',
                    'babel?presets[]=es2015',                  
                    'ts-loader'
                ] 
            },
            {   test: /\.json$/, 
                loader: 'json'
            }
        ]
    },
    resolve: {
        extensions: ['', '.ts', '.tsx', '.js', '.json']
    },
}

if someone still struggles with this see the readme: https://github.com/webpack-contrib/webpack-hot-middleware/blob/master/README.md

This module is only concerned with the mechanisms to connect a browser client to a webpack server & receive updates. It will subscribe to changes from the server and execute those changes using webpack's HMR API. Actually making your application capable of using hot reloading to make seamless changes is out of scope, and usually handled by another library.

webpack-hot-middleware doesn't handle hot reload, you'd need to use react-hot-loader for example