shikokuchuo{net}: nanonext - a web toolkit

shikokuchuo

The previous two articles have centered on the main uses which led to the creation of Nanonext - the desire to bridge code written in different languages, as well as the ability to perform actions concurrently.

This article aims to highlight the additional features that have been built around the core capabilities in the NNG library that actually make it a very good tool for interacting with the web.

This is especially relevant as version 0.5.5 just released to CRAN integrates the ‘mbedtls’ library providing TLS support for secure websites and websocket connections across all platforms.

The package has also made it into the ‘Web Technologies’ CRAN Task View under ‘Core Tools For HTTP Requests’: https://cran.r-project.org/view=WebTechnologies

library(nanonext)

ncurl - a minimalist (async) http(s) client

For normal use, it takes just the URL.

It can follow redirects.

ncurl("https://httpbin.org/headers")

$status
[1] 200

$headers
NULL

$raw
  [1] 7b 0a 20 20 22 68 65 61 64 65 72 73 22 3a 20 7b 0a 20 20 20 20
 [22] 22 48 6f 73 74 22 3a 20 22 68 74 74 70 62 69 6e 2e 6f 72 67 22
 [43] 2c 20 0a 20 20 20 20 22 58 2d 41 6d 7a 6e 2d 54 72 61 63 65 2d
 [64] 49 64 22 3a 20 22 52 6f 6f 74 3d 31 2d 36 33 64 32 64 65 61 31
 [85] 2d 32 64 66 34 30 61 36 34 30 35 33 33 64 33 34 65 30 34 39 64
[106] 37 61 30 63 22 0a 20 20 7d 0a 7d 0a

$data
[1] "{\n  \"headers\": {\n    \"Host\": \"httpbin.org\", \n    \"X-Amzn-Trace-Id\": \"Root=1-63d2dea1-2df40a640533d34e049d7a0c\"\n  }\n}\n"

Whilst it is designed to be minimalist and easy to use, the real power however lies in its ability to use other methods such as POST or PUT, and the ability of the arguments ‘headers’ and ‘data’ to take arbitrary values that are sent in the HTTP request.

This makes it perfect as a client for making REST API calls, and is indeed a rather performant solution.

res <- ncurl("http://httpbin.org/post",
             async = TRUE,
             convert = FALSE,
             method = "POST",
             headers = c(`Content-Type` = "application/json", Authorization = "Bearer APIKEY"),
             data = '{"key": "value"}',
             response = c("Date", "Server"))

Above:

‘async’ is set to TRUE to return an ‘ncurlAio’ object immediately, with the request happening asynchronously. The data will be available once resolved, or if called explicitly (which will wait).
‘convert’ is set to FALSE so time is not wasted converting the raw data to characters, which is useful when, for example, a JSON parser can directly parse the raw bytes.
‘response’ is specified to return the requested response headers.

res

< ncurlAio >
 - $status for response status code
 - $headers for response headers
 - $raw for raw message
 - $data for message data

call_aio(res)$status

[1] 200

res$headers

$Date
[1] "Thu, 26 Jan 2023 20:12:17 GMT"

$Server
[1] "gunicorn/19.9.0"

res$raw

  [1] 7b 0a 20 20 22 61 72 67 73 22 3a 20 7b 7d 2c 20 0a 20 20 22 64
 [22] 61 74 61 22 3a 20 22 7b 5c 22 6b 65 79 5c 22 3a 20 5c 22 76 61
 [43] 6c 75 65 5c 22 7d 22 2c 20 0a 20 20 22 66 69 6c 65 73 22 3a 20
 [64] 7b 7d 2c 20 0a 20 20 22 66 6f 72 6d 22 3a 20 7b 7d 2c 20 0a 20
 [85] 20 22 68 65 61 64 65 72 73 22 3a 20 7b 0a 20 20 20 20 22 41 75
[106] 74 68 6f 72 69 7a 61 74 69 6f 6e 22 3a 20 22 42 65 61 72 65 72
[127] 20 41 50 49 4b 45 59 22 2c 20 0a 20 20 20 20 22 43 6f 6e 74 65
[148] 6e 74 2d 4c 65 6e 67 74 68 22 3a 20 22 31 36 22 2c 20 0a 20 20
[169] 20 20 22 43 6f 6e 74 65 6e 74 2d 54 79 70 65 22 3a 20 22 61 70
[190] 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 22 2c 20 0a 20 20 20
[211] 20 22 48 6f 73 74 22 3a 20 22 68 74 74 70 62 69 6e 2e 6f 72 67
[232] 22 2c 20 0a 20 20 20 20 22 58 2d 41 6d 7a 6e 2d 54 72 61 63 65
[253] 2d 49 64 22 3a 20 22 52 6f 6f 74 3d 31 2d 36 33 64 32 64 65 61
[274] 31 2d 35 66 62 35 36 64 66 38 31 61 31 65 30 33 64 32 31 32 61
[295] 32 35 33 34 64 22 0a 20 20 7d 2c 20 0a 20 20 22 6a 73 6f 6e 22
[316] 3a 20 7b 0a 20 20 20 20 22 6b 65 79 22 3a 20 22 76 61 6c 75 65
[337] 22 0a 20 20 7d 2c 20 0a 20 20 22 6f 72 69 67 69 6e 22 3a 20 22
[358] 31 38 35 2e 32 32 35 2e 34 35 2e 34 39 22 2c 20 0a 20 20 22 75
[379] 72 6c 22 3a 20 22 68 74 74 70 3a 2f 2f 68 74 74 70 62 69 6e 2e
[400] 6f 72 67 2f 70 6f 73 74 22 0a 7d 0a

The function is named ‘ncurl’ after the ubiquitous ‘curl’, but it uses a completely different technology stack, leveraging the ‘NNG’ and ‘MbedTLS’ libraries instead.

stream - websocket client

stream() exposes NNG’s low-level byte stream interface for communicating with raw sockets. This may be used for connecting to arbitrary non-NNG endpoints.

Perhaps its most important use (in connection with the web at least), is for communicating with (secure) websocket servers. The argument textframes = TRUE can be specified where the websocket server uses text rather than binary frames, which is often the case.

# official demo API key used below
s <- stream(dial = "wss://ws.eodhistoricaldata.com/ws/forex?api_token=OeAFFmMliFG5orCUuwAKQ8l4WWFQ67YX",
            textframes = TRUE)
s

< nanoStream >
 - type: dialer
 - url: wss://ws.eodhistoricaldata.com/ws/forex?api_token=OeAFFmMliFG5orCUuwAKQ8l4WWFQ67YX
 - textframes: TRUE

send() and recv(), as well as their asynchronous counterparts send_aio() and recv_aio() can be used on Streams in the same way as Sockets.

This affords a great deal of flexibility in ingesting, manipulating and processing streaming data.

s |> recv(keep.raw = FALSE)

[1] "{\"status_code\":200,\"message\":\"Authorized\"}"

s |> send('{"action": "subscribe", "symbols": "EURUSD"}')

[1] 0

s |> recv(keep.raw = FALSE)

[1] "{\"s\":\"EURUSD\",\"a\":1.08901,\"b\":1.08894,\"dc\":\"-0.2792\",\"dd\":\"-0.0030\",\"ppms\":false,\"t\":1674763938000}"

s |> recv(keep.raw = FALSE)

[1] "{\"s\":\"EURUSD\",\"a\":1.08901,\"b\":1.08899,\"dc\":\"-0.2792\",\"dd\":\"-0.0030\",\"ppms\":false,\"t\":1674763938000}"

close(s)

sha[224|256|384|512] - cryptographic hash and HMAC algorithms

As ‘nanonext’ now links to the ‘mbedtls’ library as well as ‘NNG’, the series of SHA-2 crypographic hash functions have been added to the package: sha224(), sha256(), sha384() and sha512().

These call the secure, optimized implementations from the ‘MbedTLS’ library and return a hash as a raw vector. These can be compared directly for authentication. Alternatively, as.character() may be used to return a character string of the hash value.

To generate an HMAC (hash-based message authentication code), simply supply the value ‘key’ to use as the secret key. Many REST APIs require the request strings to be signed, and now the ‘nanonext’ package provides a fast and reliable method of generating a SHA-256 HMAC for this purpose.

sha256("hello world!")

[1] "7509e5bda0c762d2bac7f90d758b5b2263fa01ccbc542ab5e3df163be08e6ca9"

as.character(sha256("hello world!"))

[1] "7509e5bda0c762d2bac7f90d758b5b2263fa01ccbc542ab5e3df163be08e6ca9"

sha256("hello world!", key = "MY_SECRET")

[1] "d8f0e2d368ff632682d55e2c1ccd49c15f8a6a3862d8eb68f1906b6ee658890a"

messenger - console-based instant messaging

There is also messenger() which is not so easy to demonstrate here as it is by nature interactive, but it is in effect a 2-way walkie talkie which can be connected to a TCP/IP or other socket address. This is a rather fun demonstration of how a multi-threaded application can be built using the NNG framework.

Whilst this function has been around for quite a few versions of ‘nanonext’, the recent addition of authentication based on a pre-shared key makes it a somewhat viable solution rather than just something for fun. We encourage you to give it a try and play around with it.

?messenger

Package website: https://shikokuchuo.net/nanonext/ On CRAN: https://cran.r-project.org/package=nanonext

nanonext - a web toolkit

ncurl - a minimalist (async) http(s) client

stream - websocket client

sha[224|256|384|512] - cryptographic hash and HMAC algorithms

messenger - console-based instant messaging

Citation