Parallel Integration

mirai provides an alternative communications backend for R. This functionality was developed to fulfil a request by R Core at R Project Sprint 2023.

make_cluster() creates a cluster object of class ‘miraiCluster’, which is fully-compatible with parallel cluster types.

  • Specify ‘n’ to launch nodes on the local machine.
  • Specify ‘url’ for receiving connections from remote nodes.
  • Optionally, specify ‘remote’ to launch remote daemons using a remote configuration generated by remote_config() or ssh_config().

Created clusters may be used for any function in the parallel base package such as parallel::clusterApply() or parallel::parLapply(), or the load-balanced versions such as parallel::parLapplyLB().

library(mirai)

cl <- make_cluster(4)
cl
#> < miraiCluster | ID: `0` nodes: 4 active: TRUE >

parallel::parLapply(cl, iris, mean)
#> $Sepal.Length
#> [1] 5.843333
#> 
#> $Sepal.Width
#> [1] 3.057333
#> 
#> $Petal.Length
#> [1] 3.758
#> 
#> $Petal.Width
#> [1] 1.199333
#> 
#> $Species
#> [1] NA

status() may be called on a ’miraiCluster` to query the number of connected nodes at any time.

status(cl)
#> $connections
#> [1] 4
#> 
#> $daemons
#> [1] "abstract://dbfed51655e80d0b27a24cc6"

stop_cluster(cl)

Making a cluster specifying ‘url’ without ‘remote’ causes the shell commands for manual deployment of nodes to be printed to the console.

cl <- make_cluster(n = 2, url = host_url())
#> Shell commands for deployment on nodes:
#> 
#> [1]
#> Rscript -e "mirai::daemon('tcp://hostname:42471',rs=c(10407,1544508607,-732721164,-718427547,-1315073054,-812390917,-1575491328))"
#> 
#> [2]
#> Rscript -e "mirai::daemon('tcp://hostname:42471',rs=c(10407,1287587055,-2075729006,752482380,-1547133862,-1770470198,1209753632))"

stop_cluster(cl)