Using my own R functions in webR in an Express JS API, and thoughts on building web apps with Node & webR

Find all my posts about webR here.

Note: the first post of this series explaining roughly what webR is, I won’t introduce it again here.

Note 2: webR and the tools I’ve been writing are moving fast and if you’re reading this from the future, some of the things here might be obsolete.

The problem

Ok, so now that we have our webR / NodeJS machinery up and running, let’s try something more interesting: use our own R functions inside webR.

How can I do that?

There are at least three ways I could think of:

1️⃣ Writing a function inside the JS code to define a function, something like :

await globalThis.webR.evalR("my_fun <- function(x){...}");

But that doesn’t check what I would expect from something I’ll use in prod and I’m pretty sure you don’t need me to detail why 😅

• ❌ Well organized
• ❌ Documented
• ❌ Tested
• ❌ Safely installable

2️⃣ Simply create an R script and source it. Something like:

const fs = require('fs');
const path = require('path');
const script = path.join(__dirname, 'script.R')
const data = fs.readFileSync(script);
await globalThis.webR.FS.writeFile(
  "/home/web_user/script.R",
  data
);
await globalThis.webR.evalR("source('/home/web_user/script.R')");

That’s a bit better, we can at least organize our code in a script and it will be:

• ✅ Well organized (more or less)
• ✅ Documented (more or less)
• ❌ Tested
• ❌ Safely installable

3️⃣ I bet you saw me coming, the best way let’s put stuff into an R package, so that we can check all the boxes.

• ✅ Well organized
• ✅ Documented
• ✅ Tested
• ✅ Safely installable

Jeroen has written a Docker image to compile an R package to WASM, but I was looking for something that wouldn’t involve compiling via a docker container every time I make a change on my R package (even if that does sound appealing, I’m pretty sure this wouldn’t make for a seamless workflow).

So here is what I’m thinking should be a well structured NodeJS / WebR app:

• Putting all the web stuff inside the NodeJS app, because, well, NodeJS is really good at doing that.
• Putting all the “business logic”, data-crunching, modeling stuff (and everything R is really good at) into an R package.
• load webR, write my R package to the webR file system, and pkgload::load_all() it into webR.

That way, I can enjoy the best of both worlds:

• NodeJS is really good at doing web related things, and there are plenty of ways to test and deploy the code.
• And same goes for the R package: if you’re reading my blog I’m pretty sure I don’t need to convince you of why packages are the perfect tool for sharing production code.

The how

Let’s start by creating our project:

mkdir webr-preload-funs
cd webr-preload-funs
npm init -y
touch index.js
npm install express webr
R -e "usethis::create_package('rfuns', rstudio = FALSE)"

Let’s now create a simple function :

> usethis::use_r("sw")

#' @title Star Wars by Species
#' @description Return a tibble of Star Wars characters by species
#' @import dplyr
#' @export
#' @param species character
#' @return tibble
#' @examples
#' star_wars_by_species("Human")
#' star_wars_by_species("Droid")
#' star_wars_by_species("Wookiee")
#' star_wars_by_species("Rodian")
star_wars_by_species <- function(species){
  dplyr::starwars |>
    filter(species == )
}

We can now add {dplyr} and {pkgload} to our DESCRIPTION (we’ll need {pkgload} to load_all() the package).

usethis::use_package("dplyr")
usethis::use_package("pkgload")
devtools::document()

Now that we have a package skeleton, we’ll have to upload it to webR.

As described in the previous post, I’ve started a webrtools NodeJS module, which will contains function to play with webR. Before this post, it had one function, loadPackages, that was used to build a webR dependency library (see Preloading your R packages in webR in an Express JS API for more info).

We’ll need to add two features :

• Install deps from DESCRIPTION (not just a package name), so a wrapper around the Rscript ./node_modules/webrtools/r/install.R dplyr from before
• Copy the package folder in NodeJS, so a more generic version of loadPackages, that can load any folder to the webR filesystem.

First, in R, we’ll need to read the DESCRIPTION and build the lib:

download_packs_and_deps_from_desc <- function (
  description,
  path_to_installation = "./webr_packages"
)
{
    if (!file.exists(description)) {
        stop("DESCRIPTION file not found")
    }
    deps <- desc::desc_get_deps(description)
    for (pak in deps$package) {
        webrtools::download_packs_and_deps(pak, path_to_installation = path_to_installation)
    }
}

Note: the code of webrtools::download_packs_and_deps() is a wrapper around the R code described in Preloading your R packages in webR in an Express JS API

And in Node, we’ll rework our loadPackages and split it into two functions — one to load into any folder, and one to load into the package library.

Note that this function is bundled in the webrtools npm module.

async function loadFolder(webR, dirPath, outputdir = "/usr/lib/R/library") {
  // function from webrtools
  const files = getDirectoryTree(
    dirPath
  )
  for await (const file of files) {
    if (file.type === 'directory') {
      await globalThis.webR.FS.mkdir(
        `${outputdir}/${file.path}`,
      );
    } else {
      const data = fs.readFileSync(`${dirPath}/${file.path}`);
      await globalThis.webR.FS.writeFile(
        `${outputdir}/${file.path}`,
        data
      );
    }
  }
}

async function loadPackages(webR, dirPath) {
  await loadFolder(webR, dirPath, outputdir = "/usr/lib/R/library");
}

The end app

We now have everything we need!

npm i webrtools@0.0.2
Rscript ./node_modules/webrtools/r/install_from_desc.R $(pwd)/rfuns/DESCRIPTION

And now, to our index.js

const app = require('express')()
const path = require('path');
const { loadPackages, loadFolder } = require('webrtools');
const { WebR } = require('webr');

(async () => {
  globalThis.webR = new WebR();
  await globalThis.webR.init();

  console.log("🚀 webR is ready 🚀");

  await loadPackages(
    globalThis.webR,
    path.join(__dirname, 'webr_packages')
  )

  await loadFolder(
    globalThis.webR,
    path.join(__dirname, 'rfuns'),
    "/home/web_user"
  )

  console.log("📦 Packages written to webR 📦");

  // see https://github.com/r-wasm/webr/issues/292
  await globalThis.webR.evalR("options(expressions=1000)")
  await globalThis.webR.evalR("pkgload::load_all('/home/web_user')");

  app.listen(3000, '0.0.0.0', () => {
    console.log('http://localhost:3000')
  })

})();

app.get('/', async (req, res) => {
  let result = await globalThis.webR.evalR(
    'unique(dplyr::starwars$species)'
  );
  let js_res = await result.toJs()
  res.send(js_res.values)
})


app.get('/:n', async (req, res) => {
  let result = await globalThis.webR.evalR(
    'star_wars_by_species(n)',
    { env: { n: req.params.n } }
    );
  try {
    const result_js = await result.toJs();
    res.send(result_js)
  } finally {
    webR.destroy(result);
  }
});

Let’s now try from another terminal:

curl http://localhost:3000

["Human","Droid","Wookiee","Rodian","Hutt","Yoda's species","Trandoshan","Mon Calamari","Ewok","Sullustan","Neimodian","Gungan",null,"Toydarian","Dug","Zabrak","Twi'lek","Vulptereen","Xexto","Toong","Cerean","Nautolan","Tholothian","Iktotchi","Quermian","Kel Dor","Chagrian","Geonosian","Mirialan","Clawdite","Besalisk","Kaminoan","Aleena","Skakoan","Muun","Togruta","Kaleesh","Pau'an"]

curl http://localhost:3000/Rodian

{"type":"list","names":["name","height","mass","hair_color","skin_color","eye_color","birth_year","sex","gender","homeworld","species","films","vehicles","starships"],"values":[{"type":"character","names":null,"values":["Greedo"]},{"type":"integer","names":null,"values":[173]},{"type":"double","names":null,"values":[74]},{"type":"character","names":null,"values":[null]},{"type":"character","names":null,"values":["green"]},{"type":"character","names":null,"values":["black"]},{"type":"double","names":null,"values":[44]},{"type":"character","names":null,"values":["male"]},{"type":"character","names":null,"values":["masculine"]},{"type":"character","names":null,"values":["Rodia"]},{"type":"character","names":null,"values":["Rodian"]},{"type":"list","names":null,"values":[{"type":"character","names":null,"values":["A New Hope"]}]},{"type":"list","names":null,"values":[{"type":"character","names":null,"values":[]}]},{"type":"list","names":null,"values":[{"type":"character","names":null,"values":[]}]}]}

Yeay 🎉 .

You can find the code here, and see it live at srv.colinfay.me/webr-preload-funs/.

You can also try it with

docker run -it -p 3000:3000 colinfay/webr-preload-funs