Term-specific smoothing for the language modeling approach to information retrieval
- 11 August 2002
- proceedings article
- Published by Association for Computing Machinery (ACM)
Abstract
This paper follows a formal approach to information retrieval based on statistical language models. By introducing some simple reformulations of the basic language modeling approach we introduce the notion of importance of a query term. The importance of a query term is an unknown parameter that explicitly models which of the query terms are generated from the relevant documents (the important terms), and which are not (the unimportant terms). The new language modeling approach is shown to explain a number of practical facts of today's information retrieval systems that are not very well explained by the current state of information retrieval theory, including stop words, mandatory terms, coordination level ranking and retrieval using phrasesKeywords
This publication has 15 references indexed in Scilit:
- The Importance of Prior Probabilities for Entry Page SearchPublished by Association for Computing Machinery (ACM) ,2002
- Predicting the cost-quality trade-off for information retrieval queriesPublished by Association for Computing Machinery (ACM) ,2001
- A study of smoothing methods for language models applied to Ad Hoc information retrievalPublished by Association for Computing Machinery (ACM) ,2001
- Document language models, query models, and risk minimization for information retrievalPublished by Association for Computing Machinery (ACM) ,2001
- A probabilistic justification for using tf×idf term weighting in information retrievalInternational Journal on Digital Libraries, 2000
- A general language model for information retrievalPublished by Association for Computing Machinery (ACM) ,1999
- Term-weighting approaches in automatic text retrievalInformation Processing & Management, 1988
- Optimization of inverted vector searchesPublished by Association for Computing Machinery (ACM) ,1985
- Relevance weighting of search termsJournal of the American Society for Information Science, 1976
- A STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVALJournal of Documentation, 1972