Stemming Evaluated in 6 Languages by Hummingbird SearchServer™ at CLEF 2001

Stephen Tomlinson. In Carol Peters, Martin Braschler, Julio Gonzalo and Michael Kluck, editors, Evaluation of Cross-Language Information Retrieval Systems: Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany, September 3-4, 2001. Revised Papers. Springer LNCS 2406 (© Springer-Verlag).

Abstract

Hummingbird submitted ranked result sets for all 5 Monolingual Information Retrieval tasks (German, French, Italian, Spanish and Dutch) of the Cross-Language Evaluation Forum (CLEF) 2001. SearchServer's Intuitive Searching™ produced the highest average precision score in the German task of the 12 groups submitting automatic, Title+Description runs. Enabling stemming in SearchServer increased average precision by 43% in German, 30% in Dutch, 18% in French, 16% in Italian, 12% in Spanish and 12% in English. All points in the 95% confidence interval for the impact of stemming on average precision in German (based on the two-sided Wilcoxon signed rank test) were greater than all points in the corresponding intervals for French, English and Italian, evidence that stemming is more beneficial in German than in French, English or Italian.

Full Paper

• Final version (Nov 9, 2001): (online abstract) (password-protected full paper) (order here) The publisher requests that authors delay posting their papers on their personal web pages until 12 months after their printed publication (which was by August 2002).
• Draft version (Aug 4, 2001): Stephen Tomlinson. Hummingbird's Fulcrum SearchServer at CLEF 2001. In Carol Peters, editor, Results of the CLEF 2001 Cross-Language System Evaluation Campaign: Working Notes for the CLEF 2001 Workshop. 3 September, Darmstadt, Germany. DELOS Network of Excellence on Digital Libraries Workshop Series.

Related Information

• CLEF Home Page (alternate CLEF link) (another alternate CLEF link)
• List of Presentations made at CLEF 2001 Conference, including Slides
• Overview of CLEF 2001 Results (top-scoring groups in average precision for each task)
• Draft Papers (Working Notes) of CLEF 2001
• Table of Contents of Final CLEF 2001 Papers
• Order the CLEF 2001 Proceedings (68 euros)
• Previous year's CLEF Proceedings (CLEF 2000) Table of Contents
• Springer's Lecture Notes in Computer Science
• Hummingbird web site
• CLEF 2002 paper (following year)
• Other Papers by Stephen Tomlinson

Last Updated: 2003 Feb 25

Comments are welcome at comments@stephent.com.

Copyright © 2001-2003 Stephen Tomlinson http://www.stephent.com/ir/papers/clef01.html