Jump to navigation
Who We Are
Policies & Guidelines
Learn at LTI
Explore Our Work
Back to the catalogue
If you have a problem in filling in this form, contact lti.catalogue AT gmail.com.
Fields marked with (
) are required.
The email is only for internal purposes.
the submission will not be considered until you confirm that you own this email.
You will receive a confirmation email upon submitting. The confirmation email might get into your junk email, so check your SPAM folder. If you do not receive an email, please contact us.
A proof for not being spam
This information will appear in the catalogue.
Natural Language Processing/Computational Linguistics
Information Retrieval, Text Mining and Analytics
Spoken Interfaces and Dialogue Processing
Keywords (comma separated, internal and not shown to public)
Direct Download Link
(If you provide a direct download link, please also provide the IP agreement, and the Required Acknowledgement)
<p> Michael Denkowski and Alon Lavie</p>
<p>The Meteor automatic evaluation metric scores machine translation hypotheses by aligning them to one or more reference translations. Alignments are based on exact, stem, synonym, and paraphrase matches between words and phrases. Segment and system level metric scores are calculated based on the alignments between hypothesis-reference pairs. The metric includes several free parameters that are tuned to emulate various human judgment tasks including WMT ranking and NIST adequacy. The current version also includes a tuning configuration for use with MERT and MIRA. Meteor has extended support (paraphrase matching and tuned parameters) for the following languages: English, Czech, German, French, Spanish, and Arabic. Meteor is implemented in pure Java and requires no installation or dependencies to score MT output. On average, hypotheses are scored at a rate of 500 segments per second per CPU core. Meteor consistently demonstrates high correlation with human judgments in independent evaluations such as <a href="http://statmt.org/wmt11/papers.html">EMNLP WMT 2011</a> and <a href="http://www.itl.nist.gov/iad/mig/tests/metricsmatr/2010/results/">NIST Metrics MATR 2010</a>.</p>
Availability (e.g. source code, binary only, XML file, etc.)
<p><a href="https://www.cs.cmu.edu/~alavie/METEOR/">Download</a></p> <p><a href="https://github.com/mjdenkowski/meteor">Source code</a></p>
Support Status (e.g. as-is, maintained, etc.)
Prerequisites (e.g. Windows XP, Java 1.6, etc.)
Required Acknowledgement (e.g. paper to cite)
<p> Michael Denkowski and Alon Lavie, <strong>"Meteor Universal: Language Specific Translation Evaluation for Any Target Language"</strong>, <em>Proceedings of the EACL 2014 Workshop on Statistical Machine Translation, 2014</em> [<a href="https://www.cs.cmu.edu/%7Ealavie/METEOR/pdf/meteor-1.5.pdf">PDF</a>] [<a href="https://www.cs.cmu.edu/%7Ealavie/METEOR/pdf/meteor-1.5.bib">bib</a>]</p>
might be helpful)
<p> <a href="https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCAQFjAA&url=https%3A%2F%2Fwww.gnu.org%2Flicenses%2Flgpl.html&ei=mctbVPnRLe3dsATxwYG4AQ&usg=AFQjCNGH9ieWI16PE5PZdMfgQzZZkmOHLw&sig2=BZFlTNwLytKFXnfiijLk5Q&bvm=bv.78677474,d.cWc">GNU Lesser General Public License</a></p>
Contact (e.g. e-mail)
Additional Comments (internal and not shown to public)