Micro-blog post topic drift detection based on LDA model

Quanchao Liu; Heyan Huang; Chong Feng

doi:10.1007/978-3-319-04048-6_10

Micro-blog post topic drift detection based on LDA model

Quanchao Liu, Heyan Huang, Chong Feng

School of Computer Science and Technology

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Citations (Scopus)

Abstract

Micro-blog posts imply a large number of topics, which contain a lot of useful information as well as a lot of junk information making the micro-blog post topic a characteristic of high drift. The changes of micro-blog post topic over time and noises introduced with the increase of the number of micro-blog posts are two main aspects of micro-blog post topic drift. We propose a method of topic drift detection based on LDA model, using Gibbs sampling algorithm to obtain the probability distribution of micro-blog post words based on words correlation, identifying the topic boundary in dynamic constant method, extracting topic words by computing lexical information entropy in the topic field, and detecting the topic drift by topic words sequence alignment based on discrete-time model. According to the experiment on topic drift detection based on LDA model, we find our method very effective in micro-blog post topic drift detection.

Original language	English
Title of host publication	Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers
Publisher	Springer Verlag
Pages	106-118
Number of pages	13
ISBN (Print)	9783319040479
DOIs	https://doi.org/10.1007/978-3-319-04048-6_10
Publication status	Published - 2013
Event	2013 International Workshop on Behavior and Social Informatics and Computing, BSIC 2013 - Beijing, China Duration: 3 Aug 2013 → 9 Aug 2013

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8178 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2013 International Workshop on Behavior and Social Informatics and Computing, BSIC 2013
Country/Territory	China
City	Beijing
Period	3/08/13 → 9/08/13

Keywords

Evolutionary analysis
LDA model
Micro-blog post
Topic drift
Topic drift detection

Access to Document

10.1007/978-3-319-04048-6_10

Cite this

Liu, Q., Huang, H., & Feng, C. (2013). Micro-blog post topic drift detection based on LDA model. In Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers (pp. 106-118). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8178 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-04048-6_10

Liu, Quanchao ; Huang, Heyan ; Feng, Chong. / Micro-blog post topic drift detection based on LDA model. Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers. Springer Verlag, 2013. pp. 106-118 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2f82fc2a064846acadf457567e28bd8f,

title = "Micro-blog post topic drift detection based on LDA model",

abstract = "Micro-blog posts imply a large number of topics, which contain a lot of useful information as well as a lot of junk information making the micro-blog post topic a characteristic of high drift. The changes of micro-blog post topic over time and noises introduced with the increase of the number of micro-blog posts are two main aspects of micro-blog post topic drift. We propose a method of topic drift detection based on LDA model, using Gibbs sampling algorithm to obtain the probability distribution of micro-blog post words based on words correlation, identifying the topic boundary in dynamic constant method, extracting topic words by computing lexical information entropy in the topic field, and detecting the topic drift by topic words sequence alignment based on discrete-time model. According to the experiment on topic drift detection based on LDA model, we find our method very effective in micro-blog post topic drift detection.",

keywords = "Evolutionary analysis, LDA model, Micro-blog post, Topic drift, Topic drift detection",

author = "Quanchao Liu and Heyan Huang and Chong Feng",

year = "2013",

doi = "10.1007/978-3-319-04048-6\_10",

language = "English",

isbn = "9783319040479",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "106--118",

booktitle = "Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers",

address = "Germany",

note = "2013 International Workshop on Behavior and Social Informatics and Computing, BSIC 2013 ; Conference date: 03-08-2013 Through 09-08-2013",

}

Liu, Q, Huang, H & Feng, C 2013, Micro-blog post topic drift detection based on LDA model. in Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8178 LNAI, Springer Verlag, pp. 106-118, 2013 International Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Beijing, China, 3/08/13. https://doi.org/10.1007/978-3-319-04048-6_10

Micro-blog post topic drift detection based on LDA model. / Liu, Quanchao; Huang, Heyan ; Feng, Chong.
Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers. Springer Verlag, 2013. p. 106-118 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8178 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Micro-blog post topic drift detection based on LDA model

AU - Liu, Quanchao

AU - Huang, Heyan

AU - Feng, Chong

PY - 2013

Y1 - 2013

N2 - Micro-blog posts imply a large number of topics, which contain a lot of useful information as well as a lot of junk information making the micro-blog post topic a characteristic of high drift. The changes of micro-blog post topic over time and noises introduced with the increase of the number of micro-blog posts are two main aspects of micro-blog post topic drift. We propose a method of topic drift detection based on LDA model, using Gibbs sampling algorithm to obtain the probability distribution of micro-blog post words based on words correlation, identifying the topic boundary in dynamic constant method, extracting topic words by computing lexical information entropy in the topic field, and detecting the topic drift by topic words sequence alignment based on discrete-time model. According to the experiment on topic drift detection based on LDA model, we find our method very effective in micro-blog post topic drift detection.

AB - Micro-blog posts imply a large number of topics, which contain a lot of useful information as well as a lot of junk information making the micro-blog post topic a characteristic of high drift. The changes of micro-blog post topic over time and noises introduced with the increase of the number of micro-blog posts are two main aspects of micro-blog post topic drift. We propose a method of topic drift detection based on LDA model, using Gibbs sampling algorithm to obtain the probability distribution of micro-blog post words based on words correlation, identifying the topic boundary in dynamic constant method, extracting topic words by computing lexical information entropy in the topic field, and detecting the topic drift by topic words sequence alignment based on discrete-time model. According to the experiment on topic drift detection based on LDA model, we find our method very effective in micro-blog post topic drift detection.

KW - Evolutionary analysis

KW - LDA model

KW - Micro-blog post

KW - Topic drift

KW - Topic drift detection

UR - http://www.scopus.com/inward/record.url?scp=84894120920&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-04048-6_10

DO - 10.1007/978-3-319-04048-6_10

M3 - Conference contribution

AN - SCOPUS:84894120920

SN - 9783319040479

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 106

EP - 118

BT - Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers

PB - Springer Verlag

T2 - 2013 International Workshop on Behavior and Social Informatics and Computing, BSIC 2013

Y2 - 3 August 2013 through 9 August 2013

ER -

Liu Q, Huang H , Feng C. Micro-blog post topic drift detection based on LDA model. In Behavior and Social Computing - Int. Workshop on Behavior and Social Informatics, BSI 2013 and Int. Workshop on Behavior and Social Informatics and Computing, BSIC 2013, Revised Selected Papers. Springer Verlag. 2013. p. 106-118. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-04048-6_10

Micro-blog post topic drift detection based on LDA model

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this