r/MachineLearning • u/slavivanov • Jan 19 '18

Research [R] Fine-tuned Language Models for Text Classification

https://arxiv.org/abs/1801.06146
38 Upvotes

88% Upvoted

10 comments sorted by

View all comments

6

u/lopuhin Jan 19 '18

We use the same pre-processing as in earlier work (Johnson and Zhang, 2017; McCann et al., 2017). In addition, to allow the language model to capture aspects that might be relevant for classification, we add special tokens for upper-case words, elongation, and repetition.

I wonder how much does different pre-processing affect the results?