As large language models (LLMs) have become all the rage recently, we can look to small scale modeling again as a useful tool to researchers in the field with strictly defined research questions that limit the use of language parsing and modeling to the bi term topic modeling procedure. In this blog post I discuss the procedure for bi-term topic modeling (BTM) in the R programming language.
One indication of when to use the procedure is when there is short text with a large "n" to be parsed. An example of this is using it on twitter applications, and related social media postings. To be sure, such applications of text are becoming harder to harvest from online, but secondary data sources can still yield insightful information, and there are other uses for the BTM outside of twitter that can bring insights into short text, such as from open ended questions in surveys.
In this case the tidyverse and readxl libraries are called. Then on the next lines the data is read into the R environment, with the specification that NA cells be dropped. This is because the BTM library will throw errors with empty cells later in step 3.
The final step calls for an integration of object "d" in step 2, and object "m" from the BTM model. The code produces the final output seen in the output figure below, which showcases topic 1, with topic relevance metric at lambda = 1.
References
Yan et al. (2013) have suggested that the procedure of BTM with its Gibbs sampling procedure handles short text better than latent Dirichlet analysis (LDA) models. This conclusion is reached by the determination that term co-occurances can be discovered and calculated to help topic learning, and the sparce word occurence problem is solved through the grouping of patterns in the whole corpus space. So long as these two points can be defended, the BTM can be seen as useful beyond the LDA model.
What follows next is a 4 step process for working through the BTM procedure in R. Note: the spacyr package is used in this workflow and should be installed ahead of time.
Step 1 involves ingesting the data.
Step 2 involves parsing the data with the spacyr library. Spacyr must be initialized, and if properly installed, will call a conda environment through the backend. Once initial commenting has indicated that the conda environment has been contacted, it is safe to proceed with parsing text. Note: when parsing is finished, spacyr must be closed, or finalized.
Step 2 indicates that once the initialization has been complete, the command spacy_parse() must be run with the first argument taking the text column in the data frame, with further arguments specified. In this case, entities was kept as false, but tag and lemma were kept as true. Then, dplyr arguments were used to filter parts of speech, keeping both nouns and adjectives only, with selection of both columns document id and lemmas. These were used to feed the object into the next line, which is the actual BTM modeling process. Kappa was subjectively kept at k = 25, with everything else in place as seen. "detailed = T" was cut off in this photograph of the code, but can be included now that it is known.
Step 3 is the display of terms and their probabilities. This is done with simple code.
The terms come in a list, and with simple breakdown, the useR can specify which topic to view. The list can be further broken down with the unlist() command.
Step 4 comes with the R CRAN instructions for the BTM package itself (Wijffels, 2023).
The final step calls for an integration of object "d" in step 2, and object "m" from the BTM model. The code produces the final output seen in the output figure below, which showcases topic 1, with topic relevance metric at lambda = 1.
Wijffels, J. (2023). BTM: Biterm topic models for short text. https://cran.r-project.org/web/packages/BTM/index.html
Yan, X., Guo, J., Lan, Y, and Cheng, X. (2013). A biterm topic model for short texts. Proceedings of the 22nd international conference on the world wide web. pp. 1445- 1456. https://doi.org/10.1145/2488388.2488514