Skip to main navigation Skip to search Skip to main content

The document as an ergodic Markov Chain

  • Eduard Hoenkamp*
  • , Dawei Song
  • *Corresponding author for this work
  • Radboud University Nijmegen
  • University of Queensland

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In recent years, statistical language models are being proposed as alternative to the vector space model. Viewing documents as language samples introduces the issue of defining a joint probability distribution over the terms. The present paper models a document as the result of a Markov process. It argues that this process is ergodic, which is theoretically plausible, and easy to verify in practice. The theoretical result is that the joint distribution can be easily obtained. This can also be applied for search resolutions other than the document level. We verified this in an experiment on query expansion demonstrating both the validity and the practicability of the method. This holds a promise for general language models.

Original languageEnglish
Title of host publicationProceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery (ACM)
Pages496-497
Number of pages2
ISBN (Print)1581138814, 9781581138818
DOIs
Publication statusPublished - 2004
Externally publishedYes
EventProceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - Sheffield, United Kingdom
Duration: 25 Jul 200429 Jul 2004

Publication series

NameProceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

ConferenceProceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
Country/TerritoryUnited Kingdom
CitySheffield
Period25/07/0429/07/04

Keywords

  • Ergodic process
  • Language models
  • Semantic space

Fingerprint

Dive into the research topics of 'The document as an ergodic Markov Chain'. Together they form a unique fingerprint.

Cite this