A hybrid word embedding model based on admixture of poisson-gamma latent dirichlet allocation model and distributed word-document- topic representation

Bala, Ibrahim Bakari and Saringat, Mohd Zainuri and Mustapha, Aida (2020) A hybrid word embedding model based on admixture of poisson-gamma latent dirichlet allocation model and distributed word-document- topic representation. Journal of Theoretical and Applied Information Technology, 98 (9). pp. 1446-1456. ISSN 1992-8645

Text AJ 2020 (203).pdf Restricted to Registered users only Download (410kB) | Request a copy

Abstract

This paper proposes a hybrid Poisson-Gamma Latent Dirichlet Allocation (PGLDA) model designed for modelling word dependencies to accommodate the semantic representation of words. The new model simultaneously overcomes the shortcomings of complexity by using LDA as the baseline model as well as adequately capturing the words contextual correlation. The Poisson document length distribution was replaced with the admixture of Poisson-Gamma for words correlation modelling when there is a hub word that connects words and topics. Furthermore, the distributed representation of documents (Doc2Vec) and topics (Topic2Vec) vectors are then averaged to form new vectors of words representation to be combined with topics with largest likelihood from PGLDA. Model estimation was achieved by combining the Laplacian approximation of log-likelihood for PGLDA and Feed-Forward Neural Network (FFN) approaches of Doc2Vec and Topic2Vec. The proposed hybrid method was evaluated for precision, recall, and F1 score based on 20 Newsgroups and AG’s News datasets. Comparative analysis of F1 score showed that the proposed hybrid model outperformed other methods.

Item Type:	Article
Uncontrolled Keywords:	Poisson-Gamma Distribution; Topic Model; LDA; Word2vec; Doc2vec; Topic2Vec
Subjects:	T Technology > TP Chemical technology
Divisions:	Faculty of Computer Science and Information Technology > Department of Software Engineering
Depositing User:	Miss Afiqah Faiqah Mohd Hafiz
Date Deposited:	26 Jan 2022 08:41
Last Modified:	26 Jan 2022 08:41
URI:	http://eprints.uthm.edu.my/id/eprint/6132

Actions (login required)

View Item