Skip to main content

Getting past the two column PDF to extract text into RQDA: Literature reviews await

One of the great promises of working with RQDA is conceiving of it as computer assisted literature review software. This requires balancing the right amount of coding with text that can be used as warrants and backing in arguments. In theory it is a great idea--using computer assisted qualitative data analysis software (CAQDAS) for literature reviews, but how do you get the article PDFs into R and RQDA in a human readable format? By this I mean that many empirical articles are written in two column formats, and text extraction with standard tools produces text on the diagonal. Extracting PDF texts under this circumstance can be daunting when using some R packages such as 'pdftools', either with or without the assistance of  the 'tesseract' package.

If you are working on a windows based computer, you can install three packages and Java to do the trick. First gather the literature articles that you would like to mark up in RQDA. Put them into a folder, and away you go.


Fig 1. Preparing files for reading into the R global environment.

Before looking at Figure 1, it is  necessary to install R packages through the 'install.packages()' command, and the 'library()' command. The required packages for this quick project are 'magrittr', 'rJava', and 'tabulizer'. All packages get separately installed by putting the respective names into the parentheses part of the install.packages and library commands described above (unless you want to make or borrow a function that will automate the process).

Once you have installed the packages, download Java SE development kit (I downloaded version 8, which comes with new terms and conditions. Be sure to read the terms and conditions carefully). This version will run with 'rJava' but you should remember where you install Java on your computer because you will need that path later. Once you have saved your path to Java and you have installed the requisite packages, it is time to run code in Figure 1.

Figure 1, Line 1 comments that the next two lines will automate importing pdfs. Line 2 sets the working directory to the path of the folder containing all of the pdfs. Line 5 tells us that we must point the package 'rJava' to the location of Java so the two can work in tandem with each other. Line 6 points 'rJava' to the place where Java lives on the computer. Line 8 explicitly calls the 'magrittr' package (a piping package; it helps to make the code clear to see, and it returns the processes into a single object, 'b'). Then, lines 10-13 are run together, producing the list of PDF contents that are read into R. Line 15 takes the names listed from line 3, pastes them together with  a comma separator, and line 16 associates the file 'b' with the list of names from line 15.

When 'b' is further processed as part of the RQDA function write.FileList(), the user can then import the object b into RQDA as a list of files with associated names. With this simple code the problem of how to use RQDA as a computer assisted literature review software is solved with three R packages: 'rJava', 'tabulizer', and 'magrittr'. These packages help Java to get the job complete, as they take two column PDF contents, and create output with one column. The default single column PDF contents are preserved as one column on the output as well.


References

  Thomas J. Leeper (2018). tabulizer: Bindings for Tabula PDF Table Extractor Library. R package version 0.2.2.

  Simon Urbanek (2020). rJava: Low-Level R to Java Interface. R package version 0.9-12. https://CRAN.R-project.org/package=rJava

Stefan Milton Bache and Hadley Wickham (2014). magrittr: A Forward-Pipe Operator for R. R package version 1.5. https://CRAN.R-project.org/package=magrittr

Labels:

Popular posts from this blog

Persisting through reading technical CRAN documentation

 In my pursuit of self learning the R programming language, I have mostly mastered the art of reading through CRAN documentation of R libraries as they are published. I have gone through everything from mediocre to very well documented sheets and anything in between. I am sharing one example of a very good function that was well documented in the 'survey' library by Dr. Thomas Lumley that for some reason I could not process and make work with my data initially. No finger pointing or anything like that here. It was merely my brain not readily able to wrap around the idea that the function passed another function in its arguments.  fig1: the  svyby function in the 'survey' library by Thomas Lumley filled in with variables for my study Readers familiar with base R will be reminded of another function that works similarly called the aggregate  function, which is mirrored by the work of the svyby function, in that both call on data and both call on a function toward...
Post a Comment
Read more

Digital Humanities Methods in Educational Research

Digital Humanities based education Research This is a backpost from 2017. During that year, I presented my latest work at the 2017  SERA conference in Division II (Instruction, Cognition, and Learning). The title of my paper was "A Return to the Pahl (1978) School Leavers Study: A Distanced Reading Analysis." There are several motivations behind this study, including Cheon et al. (2013) from my alma mater .   This paper accomplished two objectives. First, I engaged previous claims made about the United States' equivalent of high school graduates on the Isle of Sheppey, UK, in the late 1970s. Second, I used emerging digital methods to arrive at conclusions about relationships between unemployment, participants' feelings about their  (then) current selves, their possible selves, and their  educational accomplishm ents. I n the image to the left I show a Ward Hierarchical Cluster reflecting the stylometrics of 153...
Read more

Bi-Term topic modeling in R

As large language models (LLMs) have become all the rage recently, we can look to small scale modeling again as a useful tool to researchers in the field with strictly defined research questions that limit the use of language parsing and modeling to the bi term topic modeling procedure. In this blog post I discuss the procedure for bi-term topic modeling (BTM) in the R programming language. One indication of when to use the procedure is when there is short text with a large "n" to be parsed. An example of this is using it on twitter applications, and related social media postings. To be sure, such applications of text are becoming harder to harvest from online, but secondary data sources can still yield insightful information, and there are other uses for the BTM outside of twitter that can bring insights into short text, such as from open ended questions in surveys.   Yan et al. (2013) have suggested that the procedure of BTM with its Gibbs sampling procedure handles sho...
Read more