two subsets of the Enron data, annotated with person names prepared by Einat Minkov.

Description

Due to privacy issues, it is very hard to get hold of large and realistic email corpora. Here you can find
a couple of email datasets, as well as a dataset of news groups text - annotated with personal names spans.

The full description of these datasets, including relevant statistics and references, is available in:

Einat Minkov, Richard C. Wang & William W. Cohen, Extracting Personal Names from Emails:
Applying Named Entity Recognition to Informal Text
, in HLT/EMNLP 2005 (PDF)

Some fast details:

  • The email corpora given here were extracted from the Enron corpus, made public by the Federal
    Agency Regulatory commission. A version of this data was later purchased by the CALO project,
    and made available for research purposes.
  • The first dataset, 'Enron-Meetings', consists of all messages located in folders named "meetings"
    or "calendar" (excluding a few very large files). Most of these messages are meeting related. The second
    subset, 'Enron-Random', was formed by uniformly sampling a user name (out of 158 users) and then
    randomly sampling an email from that user.
  • As a second type of informal text, we also annotated a collection of newsgroups postings. The
    'Newsgroups' dataset was extracted from the 20Newsgroups corpus, by Vitor R. Carvalho.
  • These datasets are given here in a Minorthird format (plain text, with separate labels files), as well as
    in a 'general' format, where the personal labels are embedded in the text using XML tags.
  • The given zipped files construct a directory tree. The separation into train and test folders corresponds
    to the data splits described in the abovementioned paper. Further separation is for convenience purposes.


If this information is inaccurate or incomplete, please submit an update through this form.