Skip to main content

Bi-Term topic modeling in R

As large language models (LLMs) have become all the rage recently, we can look to small scale modeling again as a useful tool to researchers in the field with strictly defined research questions that limit the use of language parsing and modeling to the bi term topic modeling procedure. In this blog post I discuss the procedure for bi-term topic modeling (BTM) in the R programming language. One indication of when to use the procedure is when there is short text with a large "n" to be parsed. An example of this is using it on twitter applications, and related social media postings. To be sure, such applications of text are becoming harder to harvest from online, but secondary data sources can still yield insightful information, and there are other uses for the BTM outside of twitter that can bring insights into short text, such as from open ended questions in surveys. 

 Yan et al. (2013) have suggested that the procedure of BTM with its Gibbs sampling procedure handles short text better than latent Dirichlet analysis (LDA) models. This conclusion is reached by the determination that term co-occurances can be discovered and calculated to help topic learning, and the sparce word occurence problem is solved through the grouping of patterns in the whole corpus space. So long as these two points can be defended, the BTM can be seen as useful beyond the LDA model. What follows next is a 4 step process for working through the BTM procedure in R. Note: the spacyr package is used in this workflow and should be installed ahead of time. Step 1 involves ingesting the data.
In this case the tidyverse and readxl libraries are called. Then on the next lines the data is read into the R environment, with the specification that NA cells be dropped. This is because the BTM library will throw errors with empty cells later in step 3. 

 Step 2 involves parsing the data with the spacyr library. Spacyr must be initialized, and if properly installed, will call a conda environment through the backend. Once initial commenting has indicated that the conda environment has been contacted, it is safe to proceed with parsing text. Note: when parsing is finished, spacyr must be closed, or finalized.
Step 2 indicates that once the initialization has been complete, the command spacy_parse() must be run with the first argument taking the text column in the data frame, with further arguments specified. In this case, entities was kept as false, but tag and lemma were kept as true. Then, dplyr arguments were used to filter parts of speech, keeping both nouns and adjectives only, with selection of both columns document id and lemmas. These were used to feed the object into the next line, which is the actual BTM modeling process. Kappa was subjectively kept at k = 25, with everything else in place as seen. "detailed = T" was cut off in this photograph of the code, but can be included now that it is known. 

Step 3 is the display of terms and their probabilities. This is done with simple code. 


The terms come in a list, and with simple breakdown, the useR can specify which topic to view. The list can be further broken down with the unlist() command.

Step 4 comes with the R CRAN instructions for the BTM package itself (Wijffels, 2023). 


The final step calls for an integration of object "d" in step 2, and object "m" from the BTM model.  The code produces the final output seen in the output figure below, which showcases topic 1, with topic relevance metric at lambda = 1. 



References

Wijffels, J. (2023). BTM: Biterm topic models for short text. https://cran.r-project.org/web/packages/BTM/index.html

Yan, X., Guo, J., Lan, Y, and Cheng, X. (2013). A biterm topic model for short texts. Proceedings of the 22nd international conference on the world wide web. pp. 1445- 1456. https://doi.org/10.1145/2488388.2488514 

Labels:

Popular posts from this blog

Survey Validation based on Synthetic Data with the Help of Claude

survey creation and validation survey creation and validation Author dr. mario martinez Published August 18, 2025 Introduction This code was generated with the help of Claude Set up synthetic data # Load necessary libraries library (survey) library (lavaan) library (psych) library (dplyr) library (ggplot2) library (missMethods) # Set seed for reproducibility set.seed ( 123 ) # Create simulated survey data n <- 500 survey_data <- data.frame ( id = 1 : n, q1 = sample ( 1 : 5 , n, replace = TRUE ), q2 = sample ( 1 : 5 , n, replace = TRUE ), q3 = sample ( 1 : 5 , n, replace = TRUE ), q4 = sample ( 1 : 5 , n, replace = TRUE ), q5 = sample ( 1 : 5 , n, replace = TRUE ), q6 = sample ( 1 : 5 , n, replace = TRUE ) ) survey_data1 <- impute_median (survey_data) # Round to whole numbers an...
Read more

Striking a Balance: How Much Should I Cast My own Site Variables, Versus Referring to the Literature When Doing Predictive Machine Learning as a Data Scientist?

 Recently I came back from the TAIR 2025 conference and I was struck by the number of presenters that focused on using either auto machine learning or artificial intelligence in creating models for predictive analytics in higher education. One of the striking things about the works presented is that the independent variables were somewhat similar to each other but yet different from each other enough to raise the question. How much should there be consistency between predictive machine learning models? Or, how generalizable should any given model be? These two questions strike at the limits of what local work should aim towards. One way to look at the issue is the pressing need to look at all available variables locally and use them to forage a way forward at predictions about issues like retention, enrollment, and so forth at the university level. To a certain degree this is a moot point, as some would argue that data science is about creating actionable insights.  That is, u...
Read more

The "Cram" Method of Validation for Machine learning

Our department at the University of Houston has a paper that is accepted and will be presented at the 2026 conference of the American Educational Research Association (AERA) in Los Angeles, CA. Without getting too deep into the purpose of the paper, while simultaneously aiming for the purpose of this writing, which is showcasing our use of novel methodology in action, we used machine learning ML algorithms to make our case. As a result, we needed to validate the machine learning outcomes. Traditionally this is done with an 80/20 holdout reservation of part of the data. Indeed, one of our critics asked us to use a holdout method in a particular way. However, as a data scientist, I am always championing the newest methodologies out there available in the R coding sphere (read as a linguistic sphere altogether). As I was looking out during the break for newest releases, I came across the work of Jia, Imai, and Li (2024/2025). Their methods turn the traditional 80/20 holdout and cross va...
Read more