Moju-kapu（モジュカプ）Modular Encapsulation

A New Software Design Paradigm

Introduction

Moju-kapu （モジュカプ） is shorthand for modular encapsulation （モジュラーカプセル化）.

It is a software design paradigm which emphasises the inherent balance in building effective stand-alone tools that together form entire integrated systems. It requires modularity such that each piece of software is self-contained, contributing unique functionality in its own right, but at the same time extensible and readily encapsulated by other systems.

Moju-kapu is about building out core functionality, and recognising the natural limits of a piece of software such that it does not become monolithic, but harks to the Unix philosophy of doing one thing and doing it well. However it extends this idea with providing interfaces for developers as well as end-users, to enable the software to be easily extended in foreseeable ways.

A layered engineering approach is advocated, where functionality can be filled at any layer, with interfaces to solutions, existing or future, at others. It does not attempt to simplistically define software as modular tools, nor as empty encompassing frameworks.

Example

{mirai} ¹ is a package implementing asynchronous evaluation for R ² – fast parallel code execution and distributed computing. It follows the ‘moju-kapu’ paradigm by creating a tight core proposition, with extensions enabled by a complement of external interfaces.

Modular

The package has an inherently modular internal design. The adoption of ‘compute profiles’, allows each to keep its own internal state. In the terminology of the package, it allows different ‘daemons’ settings concurrently, where ‘daemons’ are background processes (local or remote) that accept computations.

This internal modularity allows it to scale massively, and fits workflows where certain computations need to be sent to specific workers with special resources such as GPUs or accelerators. It also allows segregating different types of usage such that the user interface may function independently of those created by other packages using {mirai} as a backend (see below).

This is functionality that is essential to {mirai} and implemented at its core.

Encapsulation

The package has the following explicit external interfaces:

• user interface - minimalist consisting mainly of two functions - mirai() and daemons().
• developer interface - functions that provide an interface specfically for extension packages.
• ‘parallel’ ³ interface - creates ‘cluster’ objects that provide a backend for the ‘parallel’ base R package.
• ‘promises’ ⁴ interface - provides a method that enables ‘mirai’ to be used as ‘promises’.

The last two interfaces are not inherent to the functionality of the package itself, hence would not exist if modularity were the sole design goal. However, they provide the necessary interfaces for mirai to be encapsulated by packages that already provide fundamental building blocks in the R ecosystem. Putting in these interfaces enhances these existing packages by making them more performant, or extending their functionality to distributed computing etc.

It allows, for example, {shiny} ⁵ and {plumber} ⁶, both promises-powered packages, to easily scale and distribute long-running tasks to servers over the network.

The developer interface provides safe and easy (read-only) access to mirai internals (the modular compute profiles) for extension packages that provide alternative launchers of ‘daemon’ processes. This has been designed for use by extension packages, and has notably been used by {crew} ⁷, the new default for high-performance computing in {targets} ⁸.

{crew} extends {mirai} to different computing environments such as traditional clusters or the cloud. It also has functionality for auto-scaling daemons according to demand, which is important due to the potential cost of resources in these high-powered environments. This is a key example of functionality being filled at the most appropriate layer - in this case {crew} where it is most applicable, rather than at {mirai} where it would be an under-utilised feature in the majority of contexts.

Conclusion

By adopting ‘moju-kapu’ as its overall design ethos, {mirai} serves a much wider section of the R ecosystem, and is inherently more impactful than it would be solely as a modular ‘tool’.

---

1. Gao (2024), mirai: Minimalist Async Evaluation Framework for R, https://doi.org/10.5281/zenodo.7912722, https://github.com/shikokuchuo/mirai↩︎

2. R Core Team (2023), R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/↩︎

3. https://shikokuchuo.net/mirai/articles/parallel.html↩︎

4. https://shikokuchuo.net/mirai/articles/promises.html↩︎

5. https://shikokuchuo.net/mirai/articles/shiny.html↩︎

6. https://shikokuchuo.net/mirai/articles/plumber.html↩︎

7. Landau WM (2023), crew: A Distributed Worker Launcher Framework, https://wlandau.github.io/crew/, https://github.com/wlandau/crew↩︎

8. Landau, W. M., (2021), The targets R package: a dynamic Make-like function-oriented pipeline toolkit for reproducibility and high-performance computing. Journal of Open Source Software, 6(57), 2959, https://doi.org/10.21105/joss.02959↩︎