Embedding-based Topic Modeling
Uses transformer embeddings to capture semantic meaning, then applies UMAP dimensionality reduction and HDBSCAN clustering to discover topics. Powered by BERTopic, this approach creates semantically coherent topics based on deep contextual understanding.
Best For
- Semantic coherence: Capture meaning beyond word co-occurrence
- Short texts: Tweets, reviews, survey responses
- Multilingual: Handles multiple languages
Usage
Automatic topic discovery with density-based clustering. Requires Python with sentence-transformers, umap-learn, and hdbscan packages.
Key Components
- Sentence Transformers: Generate contextual embeddings. Selectable: MiniLM (fast), MPNet (higher quality), BGE small/base v1.5, E5 base v2, Nomic v2 (multilingual), GTE multilingual.
- UMAP: Dimensionality reduction preserving local structure
- HDBSCAN: Density-based clustering for topic discovery
- c-TF-IDF: Class-based TF-IDF for topic keyword extraction
Quality Metrics
- NPMI: Normalized Pointwise Mutual Information on top terms; coherence axis less sensitive to rare words.
- Topic Diversity: Proportion of unique words across topics' top-K terms (higher = less topic overlap).
- Silhouette Score: Embedding-space cluster separation (-1 to 1; higher is better).
References
- Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794. https://arxiv.org/abs/2203.05794
- Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (pp. 3982–3992). https://arxiv.org/abs/1908.10084
- McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426. https://arxiv.org/abs/1802.03426
- Campello, R. J. G. B., Moulavi, D., & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. In PAKDD 2013, Lecture Notes in Computer Science (Vol. 7819, pp. 160–172). Springer. https://doi.org/10.1007/978-3-642-37456-2_14
- Hoyle, A., Goel, P., Hian-Cheong, A., Peskov, D., Boyd-Graber, J., & Resnik, P. (2021). Is automated topic model evaluation broken? The incoherence of coherence. NeurIPS 2021. arXiv:2107.02173