What do the BILOU tags mean in Named Entity Recognition?

I would like to add some experience comparing BIO and BILOU schemes. My experiment was on one dataset only and may not be representative.

My dataset contains around 35 thousand short utterances (2-10 tokens) and are annotated using 11 different tags. In other words, there are 11 named entities.

The features used include the word, left and right 2-grams, 1-5 character ngrams (except middle ones), shape features and so on. Few entities are gazetteer backed as well.

I shuffled the dataset and split it into 80/20 parts: training and testing. This process was repeated 5 times and for each entity I recorded Precision, Recall and F1-measure. The performance was measured at entity level, not at token level as in Ratinov & Roth, 2009 paper.

The software I used to train a model is CRFSuite. I used L-BFGS solver with c1=0 and c2=1.

First of all, the test results compared for the 5 folds are very similar. This means there is little of variability from run to run, which is good. Second, BIO scheme performed very similarly as BILOU scheme. If there is any significant difference, perhaps it is at the third or fourth digit after period in Precision, Recall and F1-measures.

Conclusion: In my experiment BILOU scheme is not better (but also not worse) than the BIO scheme.

B = Beginning
I/M = Inside / Middle
L/E = Last / End
O = Outside
U/W/S = Unit-length / Whole / Singleton

So BILOU is the same with IOBES and BMEWO.

Cho et al. compares performance of different IO, IB, IE, IOB, IOBES, etc. annotation variants. https://www.academia.edu/12852833/Named_entity_recognition_with_multiple_segment_representations

There is also BMEWO+, which put more information about surrounding word class to Outside tokens (thus "O plus"). See details here https://lingpipe-blog.com/2009/10/14/coding-chunkers-as-taggers-io-bio-bmewo-and-bmewo/

Based on an issue and a patch in Clear TK, it seems like BILOU stands for "Beginning, Inside and Last tokens of multi-token chunks, Unit-length chunks and Outside" (emphasis added). For instance, the chunking denoted by brackets

(foo foo foo) (bar) no no no (bar bar)

can be encoded with BILOU as

B-foo, I-foo, L-foo, U-bar, O, O, O, B-bar, L-bar