Annotation & Text-Processing Tools

Text Coding/(Manual) Annotation Programs/Text-analysis Tools & Search Engines

Please note that some of these programs produce XML files in standoff format, which separates the text into different linked levels. The advantage of this type of annotation is that it is possible to link various types of annotation to the same set of data, but the disadvantage is that it’s usually not possible to ‘interact’ directly with that data unless this is done through the interface it’s been created with. In other words, creating standoff annotations usually ties one into specific programs and the functionality the provide for merging views.

DART (Dialogue Annotation & Research Tool

An annotation & analysis tool designed for the semi-automatic annotation of spoken (transcribed) dialogues on the levels of syntax, pragmatics (speech acts), (surface) polarity, semantics (topics), & semantico-pragmatics (modes, ‘IFIDs’).

DART produces annotations in what I refer to as ‘Simple XML’, a highly readable format that still allows the corpus user to ‘interact’ with the data easily to perform corrections, add additinal annotations, etc.

As a research, tool, DART also offers facilities for the creation of dialogue corpora & and their associated analysis resources, a built-in concordancer, n-gram analysis, as well as speech-act statistics.

For a detailed description, see my recent article in Corpus Linguistics and Linguistic Theory about version 1.

Version 2, as well as DART’s ‘big brother’, the Text Analysis & Research Tool (TART), are currently under development.

Embedding Viewer viewer for language modeling based on SketchEngine (no registration).

EXMARaLDA
(Extensible Markup Language for Discourse Annotation)

A system of concepts, data formats and tools for the computer assisted transcription and annotation of spoken language; XML-based data formats; Java-based tools; interoperable with software like Praat, ELAN or the TASX Annotator; based on the annotation graph framework (Bird/Liberman 2001); supports several important transcription systems (HIAT, DIDA, GAT, CHAT) through a number of parameterised functions.

EXMARaLDA includes a facility for concordancing (SQUIRREL("Search and Query Instrument for EXMARaLDA") and also ZECKE ("Ziemlich einfaches Konkordanzwerkzeug für EXMARaLDA") for searching transcribed and annotated phenomena in an EXMARaLDA corpus), but I’ve not yet used these myself, so can’t comment.

NooJ
(by Max Silberztein)

A new website is labelled as currently being under construction when last accessed on 2026-Oct-08 10:33:46.

A free corpus-processing tool and linguistic engineering development platform/environment. Can be used as: corpus processor, information extraction system, terminological extractor, Machine Translation development tool, tool for teaching linguistics & computational linguistics.

Allows linguists to formalize several levels of linguistic phenomena: orthography and spelling, lexicons for simple words, multiword units and frozen expressions, inflectional, derivational and productive morphology, local, structural syntax and transformational syntax. As a corpus processing tool, NooJ allows users to apply sophisticated linguistic queries to large corpora in order to build indices and concordances, annotate texts automatically, perform statistical analyses, etc. Linguistic modules can already be freely downloaded for many languages.

Characteristics: (1) can process texts in over 100+ file formats, including HTML, PDF, MS-OFFICE, all variants of UNICODE. (2) can import information from, and export its annotations back to XML documents. (3) annotation system that allows all levels of grammars to be applied to texts without modifying them; this allows linguists to formalize various phenomena independently, and to apply the corresponding grammars in cascade. For instance, by combining inflection, derivation and syntactic data, NooJ can perform Harris-type transformations.

OneClick Terms Simple term extractor interface giving easy access to terminology extraction functionality. Powered by SketchEngine technology (no registration, but some limitations).

RSTTool

or the related Systemic Coder

RSTTool is a graphical interface for marking up the structure of text. While primarily intended to be used for Rhetorical Structure (cf. Rhetorical Structure Theory (RST): Mann & Thompson 1988), the tool also allows the mark-up of constituency-style analysis, as in the Generic Structure Potential (GSP - cf. Hasan 1984; Halliday & Hasan 1985). Windows, Macintosh, UNIX and LINUX operating systems (requires the pre-installation of Tcl/Tk, a scripting language engine). The Tool consists of four interfaces: Text Segmentation: for marking the boundaries between text segments; Text Structuring: for marking the structural relations between these segments; Relation Editor: for maintaining the set of discourse relations, and schemas. Statistics: for deriving simple descriptive statistics based on your analysis.

Systemic Coder is a tool that facilitates the linguistic coding of corpus material, through the prompting of the user for relevant categories. Linguistic features are organised in terms of a systemic network – an inheritance hierarchy – to reduce the amount of coding effort. You first define your feature hierarchy, and then prompted to code the segements of the text according to the hierarchy. These codings can then be statistically analysed, either using the built-in comparative statistics programs, or by exporting the codings in a form readable by statistical packages.

[My comment: Both these tools do not seem to create output or accept input in an exportable format such as XML. If you know otherwise, please let me know.]

Text Feature Analyser

A tool for investigating textual features that may help in identifying & measuring issues related to text complexity. The basic design and usage are described in my original article on the tool, which can be downloaded from my publications page.

UAM Corpus Tool

Annotation & corpus analysis tool using stand-off XML markup.  Features: (1) Annotation of multiple texts using the same annotation schemes, of your design. (2) Annotation of each text at multiple levels (e.g., NP, Clause, Sentence, whole document) (3) Searching for instances across levels, e.g., finite-clause containing company-np, or future-clause in introduction. (4) Comparative statistics across subsets, e.g., contrasting conversational patterns used by male and female speakers. (5) All annotation is in stored in stand-off XML files, meaning that your annotations can more easily be shared with other applications and allows for multiple overlapping analyses of the same text.


Tools & Resources for Transcribing, Annotating or Analysing texts (inc. speech or audio-visual)

AGTK (Annotation Graph Toolkit)

Work pioneered by Steven Bird; 'annotation graphs' are a formal framework for representing linguistic annotations of time series data. Application included in this toolkit are: MultiTrans: transcribing multi-party conversation; TableTrans: observational coding of audio; TreeTrans: syntactic annotation; InterTrans: interlinear text transcription

ATLAS (Architecture and Tools for Linguistic Analysis Systems)

An architecture targeted at facilitating the development of linguistic applications. The principal goal of ATLAS is to provide an abstraction over the diversity of linguistic annotations. The abstraction, which expands on Bird and Liberman’s Annotation Graphs, is able to represent complex annotations on signals of arbitrary dimensionality. Last build from February 2003.

CLaRK

An XML-based system for corpora development and it includes an Unicode XML Editor, XPath language for navigation in XML documents, XSLT engine for tranformation of XML documents, Cascaded Regular Grammars, Constraints over XML documents, Tokenizers, Concordance tool, Extract, Remove and other tools. The system is implemented in JAVA.

ELAN (EUDICO Linguistic Annotator)

an annotation tool that allows you to create, edit, visualize and search annotations for video and audio data. It was developed at the Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands, with the aim to provide a sound technological basis for the annotation and exploitation of multimedia recordings. ELAN is specifically designed for the analysis of languages, sign languages, and gesture, but it can be used by everybody who works with media corpora, i.e., with video and/or audio data, for purposes of annotation, analysis and documentation.

Emu Speech Database System

A system for managing collections of speech data which supports hierarchical labelling of utterances. Emu is freely available and supports a range of file formats.

GATE (General Architecture for Text Engineering)

is an architecture, framework and development environment for language engineering which can be also used to annotate texts. GATE is a domain-specific software architecure and development environment (SDK) that supports researchers in Natural Language Processing and Computational Linguistics and developers who are producing and delivering Language Engineering systems. It supports the full lifecycle of language processing components, from corpus collection and annotation through system evaluation.

NITE XML Toolkit

Aimed at software developers, to allow them to build the more specialized displays, interfaces, and analyses that are required by end users when working with highly structured or cross-annotated XML data and multimedia data.

Latest download from November 2010,

PRAAT

Free, comprehensive speech analysis, synthesis, and manipulation package; includes general numerical and statistical stuff, is built on a general-purpose GUI (graphical user interface) shell for handling objects, and produces publication-quality graphics. Runs on virtually all platforms (Windows, Macintosh, Unix/Linux, etc.).

SIL tools

Lots of software relevant to speech data (& field linguistics), including Speech Analyzer (recording & editing speech, pitch tracking and spectrograms).

Transana

Open source, but not free (!) program (Windows & Macintosh) for the transcription and analysis of video data. It provides a way to view video, create a transcript, and link places in the transcript to frames in the video. It provides tools for identifying and organizing analytically interesting portions of videos, as well as for attaching keywords to those video clips. It also features database and file manipulation tools that facilitate the organization and storage of large collections of digitized video. Features: import and view MPEG-1 video and MP3 and WAV format audio files; automatically highlights the relevant portion of the transcript while the video plays; a multi-user version, Transana-MU, allows users to share their data and analyses with other research team members via a LAN.

wavesurfer

Free, Open Source tool for sound visualization and manipulation. Runs on virtually all platforms (Windows, Macintosh, Unix/Linux, etc.).

Latest version from 2019.

Winpitch

Windows software. Speech analysis and annotation tool, with fundamental frequency and spectrographic display. Prosodic morphing capability through re-synthesis of natural speech.