Otel Demonstration

Preparation - for Pydantic Logfire

Go to https://pydantic.dev/logfire, click Get Started. Choose data region, authorize with GitHub account.
Create a new project. In settings click generate write token.
Add these lines to .Renviron:

OTEL_TRACES_EXPORTER=http
OTEL_EXPORTER_OTLP_ENDPOINT="https://logfire-us.pydantic.dev"
OTEL_EXPORTER_OTLP_HEADERS="Authorization=<your-write-token>"

URL should use “eu” instead of “us” for the region if applicable

In R, install dependencies:

pak::pak(
  c(
    "otel",
    "otelsdk",
    "rstudio/shiny#4269", # "rstudio/shiny@c3f414b"
    "rstudio/promises", # "rstudio/promises@870ea76"
    "mirai",
    "bslib"
  ),
  upgrade = TRUE,
  ask = FALSE
)

Demonstration - using Pydantic Logfire

Go to: https://logfire-us.pydantic.dev (or https://logfire-eu.pydantic.dev)
Select the project that was created and the Live view.
Shiny / mirai integrated example

Run any of the vignette demos at https://mirai.r-lib.org/articles/v02-promises.html, e.g.:

library(shiny)
library(bslib)
library(mirai)

ui <- page_fluid(
  p("The time is ", textOutput("current_time", inline = TRUE)),
  hr(),
  numericInput("n", "Sample size (n)", 1000),
  numericInput("delay", "Seconds to take for plot", 1),
  input_task_button("btn", "Plot uniform distribution"),
  plotOutput("plot")
)

server <- function(input, output, session) {
  output$current_time <- renderText({
    invalidateLater(1000)
    format(Sys.time(), "%H:%M:%S %p")
  })

  task <- ExtendedTask$new(
    function(...) mirai({Sys.sleep(y); runif(x)}, ...)
  ) |> bind_task_button("btn")

  observeEvent(input$btn, task$invoke(x = input$n, y = input$delay))

  output$plot <- renderPlot(hist(task$result()))

}

# run app using 1 local daemon
daemons(1)

# automatically shutdown daemons when app exits
onStop(function() daemons(0))

shinyApp(ui = ui, server = server)

Close app.

See the spans appear on the live view.

Click the pause button to pause the scrolling.

Sample output

Talking points

Otel concepts

Traces - trace some kind of request through an entire application - which could consist of several R packages, components in several languages, database queries etc.
Spans are small units of work within a trace.
- Span ID - each span has a parent span (unless at the top level)
- Links - to other spans (can show other relationships as there can only be one parent)
- Attributes - to record useful metadata
- Events - record specific things that happened within the span
- Status - error, ok or unset
- Kind - ‘internal’ by default, can be ‘client’ or ‘server’ if useful to designate as such

Applied to Shiny

Shiny reactive updates are traced
Every time a slider is moved, button is pressed, an output updates etc. In this example:
- Every second the clock updates
- When the button is clicked - this starts a mirai-backed ExtendedTask
- When the ExtendedTask finishes, this triggers another update to render the plot
Can monitor behaviour of entire system when deployed i.e. under realistic load rather than just when profiling using a tool like ReactLog
Identify bottlenecks -> opportunity to use async
Pinpoint and get notified of errors in real time if some part starts failing

Applied to mirai

Can see when a mirai request is made, and as a child span, when it is actually evaluated
This relationship is easy to see, even though the evaluation could be on a totally different machine - e.g. cloud instance
A large gap between them could mean that the request is being queued before evaluation -> need to add more daemons to scale up capacity

UIs (like Pydantic Logfire)

Designed to easily let you handle and query large numbers of small spans
Advanced capabilities for searching and filtering, e.g. by
- Service: R
- Scope: shiny, mirai
- Anything else!

Combine with OpenTelemetry logs and metrics

Logs - using the otel package - advantage is that you now have everything in one place
Metrics - through 3rd party tools - take measurements for a system e.g. CPU, memory, disk usage

Preparation - for Jaeger (local collection)

Download and install Docker Desktop from: https://www.docker.com/products/docker-desktop/.
In a terminal, pull and start a Jaeger container.

docker run --rm --name jaeger \
 -p 16686:16686 \
 -p 4317:4317 \
 -p 4318:4318 \
 -p 5778:5778 \
 -p 9411:9411 \
 jaegertracing/jaeger:2.9.0

Docker Desktop will now show the running container with link to web UI at http://localhost:16686.
Add following line to .Renviron (may need sudo to modify). This can’t just be set within an R session as needs to apply to all new processes.

OTEL_TRACES_EXPORTER=http

In R, install dependencies:

pak::pak(
  c(
    "otel",
    "otelsdk",
    "rstudio/shiny#4269", # "rstudio/shiny@c3f414b"
    "rstudio/promises", # "rstudio/promises@870ea76"
    "mirai",
    "bslib"
  ),
  upgrade = TRUE,
  ask = FALSE
)

Demonstration - using Jaeger

Open up Docker Desktop.
Start up Jaeger in a terminal:

docker run --rm --name jaeger \
 -p 16686:16686 \
 -p 4317:4317 \
 -p 4318:4318 \
 -p 5778:5778 \
 -p 9411:9411 \
 jaegertracing/jaeger:2.9.0

Open web UI: http://localhost:16686
Shiny / mirai integrated example

Run any of the vignette demos at https://mirai.r-lib.org/articles/v02-promises.html, e.g.:

library(shiny)
library(bslib)
library(mirai)

ui <- page_fluid(
  p("The time is ", textOutput("current_time", inline = TRUE)),
  hr(),
  numericInput("n", "Sample size (n)", 1000),
  numericInput("delay", "Seconds to take for plot", 1),
  input_task_button("btn", "Plot uniform distribution"),
  plotOutput("plot")
)

server <- function(input, output, session) {
  output$current_time <- renderText({
    invalidateLater(1000)
    format(Sys.time(), "%H:%M:%S %p")
  })

  task <- ExtendedTask$new(
    function(...) mirai({Sys.sleep(y); runif(x)}, ...)
  ) |> bind_task_button("btn")

  observeEvent(input$btn, task$invoke(x = input$n, y = input$delay))

  output$plot <- renderPlot(hist(task$result()))

}

# run app using 1 local daemon
daemons(1)

# automatically shutdown daemons when app exits
onStop(function() daemons(0))

shinyApp(ui = ui, server = server)

Close app.

In the Jaeger web UI http://localhost:16686:

Service select R
Operation select mirai::mirai
click Find Traces

Result

A reactive_update with 11 child spans.

Reactive update creates a mirai ExtendedTask, and initially returns
Evaluation of the mirai on daemon takes 1s
Triggers another reactive_update which updates the plot