Never Block, Never Copy

mirai + mori from Shiny to the Cluster

Charlie Gao

Open Source, Posit PBC

Performance

Photo by Lucas Cogrossi on Unsplash

Scale further. Scale safely.

Photo by Margarita B on Unsplash

A Shiny server is an event loop

Async is never waiting for a result

blocking waiting… Shiny host 10 s task event loop frozen async Shiny host daemon loop keeps turning The task runs off the loop — the result lands when it's ready ExtendedTask

Scaling safely

Many users at once — the queue fills

users many, at once Shiny host dispatcher memory daemon 1 daemon 2 daemon 3 OOM launcher Every queued task holds its data at the dispatcher daemons(3)

The dispatcher runs out of memory

users many, at once Shiny host dispatcher memory daemon 1 daemon 2 daemon 3 OOM launcher One process dies — and task flow stops for every user daemons(3)

Switch the dispatcher off — the queue spreads

users many, at once Shiny host dispatcher memory daemon 1 daemon 2 daemon 3 OOM launcher Sub-optimal: a slow task strands everything queued behind it daemons(3, dispatcher = FALSE)

A daemon dies — and only its queue goes with it

users many, at once Shiny host dispatcher memory daemon 1 daemon 2 daemon 3 OOM launcher Only that daemon's queue is lost — everyone else keeps going daemons(3, dispatcher = FALSE)

Relaunch on demand
 

users many, at once Shiny host dispatcher memory daemon 1 daemon 2 daemon 3 OOM launcher A replacement is launched on demand — work continues launch_remote(1)

Recovery — not prevention

Bounded queue + try_mirai()

The bound

users Shiny host dispatcher bound memory daemon 1 daemon 2 daemon 3 event loop frozen NULL The queue fills to the bound — and no further daemons(memory = …)

At the line, mirai() blocks

users Shiny host dispatcher bound memory daemon 1 daemon 2 daemon 3 event loop frozen NULL Every user's session greys out — unacceptable for Shiny mirai() blocks the event loop

try_mirai() → NULL

users Shiny host dispatcher bound memory daemon 1 daemon 2 daemon 3 event loop frozen NULL Control returns instantly — the app chooses what to do try_mirai() returns NULL

user warning · automatic retry · fallback

library(mori)

Photo by Zoshua Colah on Unsplash

One copy. Many readers.

What is actually in that queue

a private copy per daemon one shared region same physical pages, per machine users Shiny host dispatcher bound 200 MB each memory daemon 1 daemon 2 daemon 3 Every queued task carries its own 200 MB copy daemons(memory = …)

share() once — 200 MB becomes 200 B

a private copy per daemon one shared region same physical pages, per machine users Shiny host dispatcher bound 200 B each memory daemon 1 daemon 2 daemon 3 200 bytes per task — the same four tasks share(x)

You just stop hitting the limit

a private copy per daemon one shared region same physical pages, per machine users Shiny host dispatcher bound 200 B each memory daemon 1 daemon 2 daemon 3 100× the tasks — the same bound, headroom to spare share(x)

Same headroom supports many more users

Deploy everywhere

Photo by Matt Artz on Unsplash

Scaling production workloads — where we left you in 2025

Your desktop: app and daemons together

users your desktop daemons(3) Shiny host daemon daemon daemon your servers ssh_config() scheduler HPC cluster cluster_config() REST API Kubernetes · cloud http_config() …including Posit Workbench, auto-configured One machine — your app and its daemons ssh_config() · cluster_config() · http_config()

Your own servers, over SSH

users your desktop daemons(3) Shiny host daemon daemon daemon your servers ssh_config() scheduler HPC cluster cluster_config() REST API Kubernetes · cloud http_config() …including Posit Workbench, auto-configured Your own servers, over SSH ssh_config() · cluster_config() · http_config()

HPC clusters, through their scheduler

users your desktop daemons(3) Shiny host daemon daemon daemon your servers ssh_config() scheduler HPC cluster cluster_config() REST API Kubernetes · cloud http_config() …including Posit Workbench, auto-configured HPC clusters, through their scheduler ssh_config() · cluster_config() · http_config()

New: cloud, Kubernetes, Workbench

users your desktop daemons(3) Shiny host daemon daemon daemon your servers ssh_config() scheduler HPC cluster cluster_config() REST API Kubernetes · cloud http_config() …including Posit Workbench, auto-configured If it has a REST API, it's a launcher ssh_config() · cluster_config() · http_config()

Deploy everywhere — now with further reach

The whole toolbox

reach — where the compute can live memory model users Shiny host dispatcher bound memory daemons(memory = …) try_mirai() your desktop daemons(n) your servers ssh_config() HPC cluster cluster_config() cloud · Kubernetes · Workbench http_config() one shared region per machine share() One deployment. Compute anywhere. Memory under control

Photo by Todd Quackenbush on Unsplash

Never block. Never copy.
install.packages(c("mirai", "mori"))

Photo by Todd Quackenbush on Unsplash