Skip to main content

Getting past the two column PDF to extract text into RQDA: Literature reviews await

One of the great promises of working with RQDA is conceiving of it as computer assisted literature review software. This requires balancing the right amount of coding with text that can be used as warrants and backing in arguments. In theory it is a great idea--using computer assisted qualitative data analysis software (CAQDAS) for literature reviews, but how do you get the article PDFs into R and RQDA in a human readable format? By this I mean that many empirical articles are written in two column formats, and text extraction with standard tools produces text on the diagonal. Extracting PDF texts under this circumstance can be daunting when using some R packages such as 'pdftools', either with or without the assistance of  the 'tesseract' package.

If you are working on a windows based computer, you can install three packages and Java to do the trick. First gather the literature articles that you would like to mark up in RQDA. Put them into a folder, and away you go.


Fig 1. Preparing files for reading into the R global environment.

Before looking at Figure 1, it is  necessary to install R packages through the 'install.packages()' command, and the 'library()' command. The required packages for this quick project are 'magrittr', 'rJava', and 'tabulizer'. All packages get separately installed by putting the respective names into the parentheses part of the install.packages and library commands described above (unless you want to make or borrow a function that will automate the process).

Once you have installed the packages, download Java SE development kit (I downloaded version 8, which comes with new terms and conditions. Be sure to read the terms and conditions carefully). This version will run with 'rJava' but you should remember where you install Java on your computer because you will need that path later. Once you have saved your path to Java and you have installed the requisite packages, it is time to run code in Figure 1.

Figure 1, Line 1 comments that the next two lines will automate importing pdfs. Line 2 sets the working directory to the path of the folder containing all of the pdfs. Line 5 tells us that we must point the package 'rJava' to the location of Java so the two can work in tandem with each other. Line 6 points 'rJava' to the place where Java lives on the computer. Line 8 explicitly calls the 'magrittr' package (a piping package; it helps to make the code clear to see, and it returns the processes into a single object, 'b'). Then, lines 10-13 are run together, producing the list of PDF contents that are read into R. Line 15 takes the names listed from line 3, pastes them together with  a comma separator, and line 16 associates the file 'b' with the list of names from line 15.

When 'b' is further processed as part of the RQDA function write.FileList(), the user can then import the object b into RQDA as a list of files with associated names. With this simple code the problem of how to use RQDA as a computer assisted literature review software is solved with three R packages: 'rJava', 'tabulizer', and 'magrittr'. These packages help Java to get the job complete, as they take two column PDF contents, and create output with one column. The default single column PDF contents are preserved as one column on the output as well.


References

  Thomas J. Leeper (2018). tabulizer: Bindings for Tabula PDF Table Extractor Library. R package version 0.2.2.

  Simon Urbanek (2020). rJava: Low-Level R to Java Interface. R package version 0.9-12. https://CRAN.R-project.org/package=rJava

Stefan Milton Bache and Hadley Wickham (2014). magrittr: A Forward-Pipe Operator for R. R package version 1.5. https://CRAN.R-project.org/package=magrittr

Labels:

Popular posts from this blog

Survey Validation based on Synthetic Data with the Help of Claude

survey creation and validation survey creation and validation Author dr. mario martinez Published August 18, 2025 Introduction This code was generated with the help of Claude Set up synthetic data # Load necessary libraries library (survey) library (lavaan) library (psych) library (dplyr) library (ggplot2) library (missMethods) # Set seed for reproducibility set.seed ( 123 ) # Create simulated survey data n <- 500 survey_data <- data.frame ( id = 1 : n, q1 = sample ( 1 : 5 , n, replace = TRUE ), q2 = sample ( 1 : 5 , n, replace = TRUE ), q3 = sample ( 1 : 5 , n, replace = TRUE ), q4 = sample ( 1 : 5 , n, replace = TRUE ), q5 = sample ( 1 : 5 , n, replace = TRUE ), q6 = sample ( 1 : 5 , n, replace = TRUE ) ) survey_data1 <- impute_median (survey_data) # Round to whole numbers an...
Read more

Striking a Balance: How Much Should I Cast My own Site Variables, Versus Referring to the Literature When Doing Predictive Machine Learning as a Data Scientist?

 Recently I came back from the TAIR 2025 conference and I was struck by the number of presenters that focused on using either auto machine learning or artificial intelligence in creating models for predictive analytics in higher education. One of the striking things about the works presented is that the independent variables were somewhat similar to each other but yet different from each other enough to raise the question. How much should there be consistency between predictive machine learning models? Or, how generalizable should any given model be? These two questions strike at the limits of what local work should aim towards. One way to look at the issue is the pressing need to look at all available variables locally and use them to forage a way forward at predictions about issues like retention, enrollment, and so forth at the university level. To a certain degree this is a moot point, as some would argue that data science is about creating actionable insights.  That is, u...
Read more

The "Cram" Method of Validation for Machine learning

Our department at the University of Houston has a paper that is accepted and will be presented at the 2026 conference of the American Educational Research Association (AERA) in Los Angeles, CA. Without getting too deep into the purpose of the paper, while simultaneously aiming for the purpose of this writing, which is showcasing our use of novel methodology in action, we used machine learning ML algorithms to make our case. As a result, we needed to validate the machine learning outcomes. Traditionally this is done with an 80/20 holdout reservation of part of the data. Indeed, one of our critics asked us to use a holdout method in a particular way. However, as a data scientist, I am always championing the newest methodologies out there available in the R coding sphere (read as a linguistic sphere altogether). As I was looking out during the break for newest releases, I came across the work of Jia, Imai, and Li (2024/2025). Their methods turn the traditional 80/20 holdout and cross va...
Read more