Datamonger for R · r · 0.0.0.9000

Retrieve and verify data offline

Johan Larsson

Datamonger can retrieve and decode data without contacting a registry or an upstream provider. This example uses the five-row mixed CSV from the shared conformance corpus, which R, Python, and Julia use to check the same logical values and digest.

The package includes the corpus and its test registry. The setup copies their known bytes into a fresh temporary cache. Every retrieval below uses offline = TRUE, so a missing cache entry causes an error. In a research workflow, populate your cache with the selected registry and artifacts before disconnecting from the network.

Retrieve the verified artifact

fetch_artifact returns the cached CSV after checking its size and SHA-256 digest against the immutable registry record. The fully qualified reference fixes the dataset source, name, and version.

reference <- "conformance:mixed_csv@1"
artifact <- fetch_artifact(reference, registry = registry,
                           cache_dir = cache_dir, offline = TRUE)
artifact_sha256 <- digest::digest(file = artifact, algo = "sha256")
stopifnot(identical(artifact_sha256, payload_sha256))
artifact_sha256
#> [1] "8a46d390c070778b3617d11ba9a0ea0cb3516a2c54aeafdc53c9196367208c1e"

Decode the logical values

fetch_data also checks the registry's expected shape and canonical logical digest. The decoder follows the declared column types, quoting, missing values, and source row order rather than guessing them from the CSV.

result <- fetch_data(reference, registry = registry, cache_dir = cache_dir,
                     offline = TRUE, return_info = TRUE)
data <- result$data
stopifnot(
  identical(dim(data), c(5L, 4L)),
  identical(data$measurement, c(1.5, -0.0, NA_real_, 20, 0)),
  identical(as.character(data$count), c("1", NA, "3", "4", "5")),
  identical(data$label, c("alpha", "quoted, value", 'says "hello"', NA, "omega")),
  identical(data$enabled, c(TRUE, FALSE, NA, TRUE, FALSE)),
  identical(result$info$verification, "decoded"),
  identical(result$info$canonical_form, cases$canonical_form),
  identical(result$info$canonical_digest, case$expected_sha256)
)

The decoded result has 5 rows and 4 columns. Missing values remain missing, and the quoted comma and double quotes remain part of their labels. The count column uses datamonger_int64, which stores exact decimal strings so that large integers do not lose precision.

display <- data
display$count <- as.character(data$count)
knitr::kable(display, row.names = FALSE)

Retain verification metadata

The artifact digest identifies the original bytes. The canonical digest identifies their decoded logical values, including column types and missing values. Retain both with the registry selector when recording an analysis.

verification <- data.frame(
  Field = c("Dataset", "Registry release", "Verification", "Canonical form",
            "Registry SHA-256", "Artifact SHA-256", "Canonical SHA-256"),
  Value = c(result$info$dataset_id, result$info$registry_release,
            result$info$verification, result$info$canonical_form,
            result$info$registry_index_sha256, artifact_sha256,
            result$info$canonical_digest)
)
knitr::kable(verification, row.names = FALSE)

Inspect the measurements

The third row has no measurement. Plotting the observed rows preserves their source positions and makes the larger value in the fourth row visible.

This small example verifies the complete path from cached artifact bytes to decoded values, reproducibility metadata, and a rendered figure. It uses a private temporary cache and removes that cache after execution.