SLATE: A Super-Lightweight Annotation Tool for Experts

A terminal-based text annotation tool in Python.

Sequence Effects in Crowdsourced Annotations (Mathur et al., 2017)

Annotator sequence bias, where the label for one item affects the label for the next, occurs across a range of datasets. Avoid it by separately randomise the order of items for each annotator.

Detecting annotation noise in automatically labelled data (Rehbein and Ruppenhofer, ACL 2017)

When labeling a dataset automatically there are going to be errors, but we can use a generative model and active learning to guide effort to checking the examples most likely to be incorrect.