Using a Convolutional Neural Network for time series classification

Here's the code. What I mentioned in the Q&A session is using ReshapeLayer to turn the input vector into a 1-channel, flat tensor that ConvolutionLayer can operate on, not to actually use images, per se.

I've limited things here just to the original industries, but you can try more if you want. Downloading takes a while, so I have an Export there you can use to reload the data later, after you quit your kernel.

However, there is a big problem with this whole idea, as you'll see, which is overfitting -- there just don't seem to be very good patterns for the net to pick up on. The more powerful you make the net, the more easily it can simply memorize particular time series rather than finding 'deeper' patterns. This will often happen with timeseries classification, because each timeseries has a lot of data, and you typically don't have that many timeseries. The only way to overcome this is to use extreme quantities of data, say 10x or 100x time as much data as we have here.

When running NetTrain, you'll see the loss on the validation set plateaus after just 30 seconds or so. At that point further training is just pointless, so you can click Stop - NetTrain will always return the net with the lowest validation loss.

And if you had tried this without a validation set, and left this net to run for 5 minutes, you'd get an apparent 95% accuracy on your training data. Which would be completely misleading. The actual accuracy maxes out around 70%. Fiddling around with this net isn't going to do much: the overfitting is so severe, the net is in some sense already too complicated.

(*industries = {"Basic Industries", "Capital Goods", 
   "Consumer Durables", "Consumer Non-Durables", "Consumer Services", 
   "Energy", "Finance", "Health Care", "Miscellaneous", 
   "Public Utilities", "Technology", "Transportation"};*)    
In[1]:= industries = {"Finance", "Technology"};

In[2]:= importIndustryStocks[name_] := 
  Import["http://www.nasdaq.com/screening/companies-by-industry.aspx?\
industry=" <> StringReplace[name, " " -> "+"] <> "&render=download", 
    "CSV"][[2 ;;, 1]];

stocksByIndustry = AssociationMap[importIndustryStocks, industries];
Length /@ stocksByIndustry

Out[6]= <|"Finance" -> 558, "Technology" -> 373|>

In[4]:= stocks = Catenate[stocksByIndustry];
Length[stocks]

Out[5]= 931

In[8]:= stock2industry = 
  Association @ Reverse[Flatten[Thread /@ Normal[stocksByIndustry]], 2];

stockData = ParallelMap[
   Replace[FinancialData[#, "Jan. 1, 2007"], 
     Except[_List] -> $Failed] &, stocks]; (* this takes a while *)

In[9]:= Export["industry_stocks.mx", {industries, stocks, stock2industry, stockData}];
FileByteCount["industry_stocks.mx"]

Out[10]= 39651493

In[11]:= toList[ts_List] := Rescale @ ArrayResample[ts[[All, 2]], 256];
toList[_] := $Failed;
rules = DeleteCases[$Failed -> _] @ Thread[Map[toList, stockData] -> Lookup[stock2industry, stocks]];
Length[rules]

Out[14]= 880

In[18]:= TestTrainSplit[rules_, frac_] := Module[{grouped},
   grouped = GroupBy[RandomSample[rules], Last, split[frac]];
   Map[Catenate, Transpose @ Values[grouped]]];
split[frac_][list_] := Block[{n, tn},
   n = Length[list]; tn = Ceiling[n * frac];
   TakeDrop[list, tn]]; 
{testData, trainingData} = TestTrainSplit[rules, 0.2];

module[n_, sz_] := 
  Sequence[ConvolutionLayer[n, {1, sz}, "Dilation" -> 4], 
   PoolingLayer[{1, 16}], Ramp];
net = NetChain[{ReshapeLayer[{1, 1, 256}], module[4, 8], (* can add more modules here *) 
   FlattenLayer[], DotPlusLayer[], SoftmaxLayer[]}, "Input" -> 256, 
  "Output" -> NetDecoder[{"Class", industries}]]

trained = 
  NetTrain[net, trainingData, MaxTrainingRounds -> 100, 
   Method -> "ADAM", ValidationSet -> testData]; (* overfitting happens VERY quickly *)

In[25]:= ListLinePlot@NetExtract[trained, {2, "Weights"}][[All, 1, 1]] (* plot the kernels, they don't look particularly nice though *)

In[26]:= measure = ClassifierMeasurements[trained, testData]

In[27]:= measure["Accuracy"]

Out[27]= 0.711864

tech = {"GE", "AAPL", "MSFT", "MRK", "IBM"};
banks = {"DB", "C", "BAC", "HSBC", "BCS"};

financialDataToList[stock_] := 
 FinancialData[stock, "Jan. 1, 2007"] // Transpose // Last // 
   Rescale // ArrayResample[#, 100] &

toImage[stock_] := 
 financialDataToList@stock // List // Replace[#, a_ :> (1 - a)] & // 
  Image

trainingdata = 
  Join[Thread[(toImage /@ tech) -> "tech"], 
   Thread[(toImage /@ banks) -> "banks"]];

width = 100;

height = 1;

net = NetChain[
  {
   ConvolutionLayer[16(*number of filters*), {height, 3(*kernel size*)}],
   PoolingLayer[{height, 2(*pooling size*)}],
   FlattenLayer[],
   DotPlusLayer[10],
   DotPlusLayer[2(*number of classes*)],
   SoftmaxLayer["Output" -> NetDecoder[{"Class", {"tech", "banks"}}]]
   },
  "Input" -> NetEncoder[{"Image", {width, height}, ColorSpace -> "Grayscale"}]
  ]

net = NetTrain[net, trainingdata, MaxTrainingRounds -> 5]

ClassifierMeasurements[net, trainingdata, "Accuracy"]

1.