Embedding R/exams Exercises as Forms in R/Markdown or Quarto Documents

Overview

The package exams2forms provides several building blocks for embedding exercises written with the R package exams (also known as R/exams) in interactive documents or quizzes written with rmarkdown or quarto.

The idea is that the dynamic exercises in R/exams' Rmd (R/Markdown) or Rnw (R/LaTeX) format can also be reused in HTML documents, web pages, or online books. This facilitates their use for self-paced learning and self-assessment without the need for a learning management system etc. (By default the correct answers are obfuscated in the documents so that they are not obvious when inspecting the HTML source code.)

For (summative) assessments the same dynamic exercises could then be exported to different learning management systems or employed in written exams.

All R/exams exercise types are supported:

Single-choice (schoice).
Multiple-choice (mchoice).
Numeric (num).
Text (string).
Cloze combining all of the previous elements (cloze).

Many of the ideas as well as the code in the package have been adapted from the webexercises package, authored by Dale Barr and Lisa DeBruine.

First examples

As a quick demonstration for R/exams exercises embedded into an HTML document, the two examples from the First Steps tutorial are included below: The single-choice exercise swisscapital and the numeric exercise deriv, both in three random variations.

What is the derivative of \(f(x) = x^{2} e^{3.2 x}\), evaluated at \(x = 0.59\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{2}\) and \(h(x) := e^{3.2 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 2 x^{2 - 1} \cdot e^{3.2 x} + x^{2} \cdot e^{3.2 x} \cdot 3.2 \\ &= e^{3.2 x} \cdot(2 x^1 + 3.2 x^{2}) \\ &= e^{3.2 x} \cdot x^1 \cdot (2 + 3.2 x). \end{aligned} \] Evaluated at \(x = 0.59\), the answer is \[ e^{3.2 \cdot 0.59} \cdot 0.59^1 \cdot (2 + 3.2 \cdot 0.59) = 15.153964. \] Thus, rounded to two digits we have \(f'(0.59) = 15.15\).

What is the derivative of \(f(x) = x^{4} e^{3.5 x}\), evaluated at \(x = 0.65\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{4}\) and \(h(x) := e^{3.5 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 4 x^{4 - 1} \cdot e^{3.5 x} + x^{4} \cdot e^{3.5 x} \cdot 3.5 \\ &= e^{3.5 x} \cdot(4 x^3 + 3.5 x^{4}) \\ &= e^{3.5 x} \cdot x^3 \cdot (4 + 3.5 x). \end{aligned} \] Evaluated at \(x = 0.65\), the answer is \[ e^{3.5 \cdot 0.65} \cdot 0.65^3 \cdot (4 + 3.5 \cdot 0.65) = 16.763849. \] Thus, rounded to two digits we have \(f'(0.65) = 16.76\).

What is the derivative of \(f(x) = x^{7} e^{2.8 x}\), evaluated at \(x = 0.72\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{7}\) and \(h(x) := e^{2.8 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 7 x^{7 - 1} \cdot e^{2.8 x} + x^{7} \cdot e^{2.8 x} \cdot 2.8 \\ &= e^{2.8 x} \cdot(7 x^6 + 2.8 x^{7}) \\ &= e^{2.8 x} \cdot x^6 \cdot (7 + 2.8 x). \end{aligned} \] Evaluated at \(x = 0.72\), the answer is \[ e^{2.8 \cdot 0.72} \cdot 0.72^6 \cdot (7 + 2.8 \cdot 0.72) = 9.430757. \] Thus, rounded to two digits we have \(f'(0.72) = 9.43\).

In addition to the question and the interaction element, there are three buttons providing the following functionality.

Button	Function
✓	Check the answer and display whether it is correct or not. When clicked, the symbol is toggled and ↰ is displayed, which can be clicked to hide the solution again.
?	Display the full correct solution explanation.
↺	Switch to the next question.

Inclusion of the solution explanation is optional and the next question button is only displayed if there is more than one random variation of a question. The icons and mouseover text can also be adapted (see below).

To set up a similar standalone file with these two exercises, the exams2webquiz() interface from the exams2forms package can be used:

library("exams2forms")
exams2webquiz(c("swisscapital.Rmd", "deriv.Rmd"), n = 3)

More elaborate examples

To showcase some more exercise types, the following examples from the R/exams package are used: capitals (multiple-choice), function (string/text), fruit (numeric with table and images), lm2 (cloze containing string, multiple-choice, numeric, and single-choice elements as well as an embedded data file).

Which of the following cities are the capital of the corresponding country?

True. Tokyo is the capital of Japan.
True. Warsaw is the capital of Poland.
False. The capital of New Zealand is Wellington.
True. Astana is the capital of Kazakhstan.
False. The de facto capital of Switzerland is Bern.

What is the name of the R function for negative binomial regression?

glm.nb is the R function for negative binomial regression. See ?glm.nb for the corresponding manual page.

Given the following information:

\(+\)	\(+\)	=	\(143\)
\(+\)	\(+\)	=	\(437\)
\(+\)	\(+\)	=	\(433\)

Compute:

\(+\)

\(+\)

=

\(\text{?}\)

The information provided can be interpreted as the price for three fruit baskets with different combinations of the three fruits. This corresponds to a system of linear equations where the price of the three fruits is the vector of unknowns \(x\):

\(x_1 =\)

\(x_2 =\)

\(x_3 =\)

The system of linear equations is then: \[ \begin{aligned} \left( \begin{array}{rrr} 1 & 2 & 0 \\ 2 & 0 & 1 \\ 0 & 2 & 1 \end{array} \right) \cdot \left( \begin{array}{r} x_1 \\ x_2 \\ x_3 \end{array} \right) & = & \left( \begin{array}{r} 143 \\ 437 \\ 433 \end{array} \right) \end{aligned} \] This can be solved using any solution algorithm, e.g., elimination: \[ x_1 = 49, \, x_2 = 47, \, x_3 = 339. \] Based on the three prices for the different fruits it is straightforward to compute the total price of the fourth fruit basket via:

	\(+\)		\(+\)		=
\(x_1\)	\(+\)	\(x_2\)	\(+\)	\(x_3\)	=
\(49\)	\(+\)	\(47\)	\(+\)	\(339\)	=	\(435\)

Theory: Consider a linear regression of y on x. It is usually estimated with which estimation technique (three-letter abbreviation)?

This estimator yields the best linear unbiased estimator (BLUE) under the assumptions of the Gauss-Markov theorem. Which of the following properties are required for the errors of the linear regression model under these assumptions?

Application: Using the data provided in linreg.csv estimate a linear regression of y on x. What are the estimated parameters?

Intercept:

Slope:

In terms of significance at 5% level:

Theory: Linear regression models are typically estimated by ordinary least squares (OLS). The Gauss-Markov theorem establishes certain optimality properties: Namely, if the errors have expectation zero, constant variance (homoscedastic), no autocorrelation and the regressors are exogenous and not linearly dependent, the OLS estimator is the best linear unbiased estimator (BLUE).

Application: The estimated coefficients along with their significances are reported in the summary of the fitted regression model, showing that x and y are not significantly correlated (at 5% level).


Call:
lm(formula = y ~ x, data = d)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.59846 -0.17142 -0.01458  0.13740  0.68566 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept)  0.03055    0.02507   1.219    0.226
x           -0.03080    0.04702  -0.655    0.514

Residual standard error: 0.2504 on 98 degrees of freedom
Multiple R-squared:  0.00436,   Adjusted R-squared:  -0.005799 
F-statistic: 0.4292 on 1 and 98 DF,  p-value: 0.5139

Code: The analysis can be replicated in R using the following code.

## data
d <- read.csv("linreg.csv")
## regression
m <- lm(y ~ x, data = d)
summary(m)
## visualization
plot(y ~ x, data = d)
abline(m)

Again, the exams2webquiz() function can be used to set up a standalone file based on the same exercises:

exams2webquiz(c("capitals.Rmd", "function.Rmd", "fruit.Rmd", "lm2.Rmd"))

Building blocks

To accomplish the functionality demonstrated above, the package provides the following contents:

exams2forms(): Main workhorse function from the package. Like other exams2xyz() interfaces this takes a vector or list of exercise files and returns Markdown text, including HTML snippets, than can be included in rmarkdown or quarto documents. This includes questions, suitable interactions for the different types of answers (with correct solutions also suitably encoded in the HTML), and optionally full solution explanations.
webex.css, webex.js: CSS (Cascading Style Sheets) and Javascript files shipped within the package and providing the code necessary for the exercise and quiz display and processing the different types of interactions.
webquiz(): Small wrapper function that creates a knitr::html_document() but includes the CSS and Javascript files above.
exams2webquiz(): Convenience interface that combines all of the above elements. It sets up a webquiz() document in which exams2forms() is used to embed the specified exercises, calls rmarkdown::render() to process it, and by default displays it in the browser. This is most useful for quickly trying out how R/exams exercises can work in HTML documents.
forms_num(), forms_string(), forms_schoice(), forms_mchoice(): Helper functions for just embedding the user interactions for the different types of exercises. Typically not called directly by the user.

Demo files

While exams2webquiz() is convenient for quickly setting up an HTML document containing certain exercises, further customizations are typicallly needed for more elaborate documents. To demonstrate how this works the exams2forms package provides two demo rmarkdown files which can also be downloaded here:

The first file quiz.Rmd renders a number of different exercises into a quiz using a single exams2forms() call, indicating that the results should be included "asis":

---
title: "R/exams quiz"
output: exams2forms::webquiz
---

```{r setup, include = FALSE}
## package and list of various exercises
library("exams2forms")
exm <- list(
  c("swisscapital.Rmd", "capitals.Rmd"),
  "deriv.Rnw",
  "deriv2.Rnw",
  "fruit.Rmd",
  "boxplots.Rmd",
  "ttest.Rmd",
  "function.Rmd",
  "lm2.Rnw",
  "fourfold2.Rmd"
)
```

```{r quiz, echo = FALSE, message = FALSE, results = "asis"}
exams2forms(exm, n = 3)
```

The questions.Rmd file looks similar but contains different sections, each with a single question set up via exams2forms().

Both files can be rendered to HTML via rmarkdown::render() or by clicking the knit button after opening the files in RStudio etc.

Setup and customization

When setting up a more elaborate document or even a full webpage or online book with rmarkdown or quarto, then the simple webquiz() HTML document provided by exams2forms is probably not sufficient. In this case, it is best to take the CSS and Javascript files from the package or download them here:

The files can then be placed in the same folder as the rmarkdown or quarto project. They can also be adapted relatively easily by changing the definitions of colors, icons, text, etc. in the first few lines of each file.

To include the CSS and Javascript in an rmarkown project, the YAML header should include:

output:
  html_document:
    css: webex.css
    includes:
      after_body: webex.js

Similarly, in a quarto project the YAML header or the _quarto.yml file should include:

format:
  html:
    css: webex.css
    include-after-body: webex.js

Standalone interaction forms

While it is not the primary focus of the exams2forms package, it is also possible to directly include interaction forms in documents like in the webexercises package, i.e., without setting up full R/exams exercises. A few simple examples are inlcuded below for numeric, text, single-choice, and multiple-choice interactions (both using drop-down interactions here), respectively.

What is the answer to the ultimate question of life, the universe, and everything?

Which superhero is the secret identity of Bruce Wayne?

Which of the following villains is not an adversary of Batman?

Which of the following characters are romantic interests of Spider-Man?

Mary Jane Watson
Pepper Potts
Selina Kyle
Gwen Stacy

The corresponding code snippets included in the inline code chunks are:

forms_num(42, width = 10)
forms_string("Batman", width = 20, usecase = FALSE)
forms_schoice(c("Bane", "Riddler", "Thanos", "Poison Ivy"), c(FALSE, FALSE, TRUE, FALSE), display = "dropdown")
forms_mchoice(c("Mary Jane Watson", "Pepper Potts", "Selina Kyle", "Gwen Stacy"), c(TRUE, FALSE, FALSE, TRUE), display = "dropdown")