How can I simplify reading JSON files?

[mikhail-putilov]

Hi!

I've been trying to externalize commonly repeated code into a utils file:

./utils.pkl:

import "pkl:json" as json

function require(path) = new json.Parser {}.parse(read(path)) as Dynamic

./main.pkl:

import "utils.pkl"

local val = utils.require("sample.json")

// intentionally simplified main.pkl

output {
  value = val
  renderer = new YamlRenderer{}
}

./sample.json:

{
  "text": "Hello, world!"
}

It works as long as all files are in the same directory. When I move main.pkl and sample.json into a subfolder dedicated to an environment ./prod/, then it breaks. I suspect this is because of lazy evaluation of read.

---

As a solution, I found that I can pass Resource into require like this:
./utils.pkl:

import "pkl:json" as json

// takes Resource instead of plain path
function require(res) = new json.Parser {}.parse(res) as Dynamic

./prod/main.pkl:

import "../utils.pkl"

// Must wrap into `read` before calling `require`
local val = utils.require(read("sample.json"))

// intentionally simplified main.pkl

output {
  value = val
  renderer = new YamlRenderer{}
}

./prod/sample.json:

{
  "text": "Hello, world!"
}

Is there anything I am missing? I wanted to have utility functions to simplify reading external sources, but I always need to remember that I must call read before :-)

[StefMa]

It seems that read() loads from the current module path. The module of it is until. Since until.pkl is one level up, you need to define the path as ./prod/sample.json in your ./prod/main.pkl.

Might look a bit strange, but since you also import utils from one level up, maybe it's okay... 🤔 🤷‍♂️

[mikhail-putilov]

Ah, that makes sense now. I think I missed that part in the documentation

Thank you!

[HT154]

Yup, to be specific: a read (or an import) with a relative path will read a file relative to where the read occurs. So here's what's happening:

1. pkl eval prod/main.pkl evaluates your main module which has URI file://<base path>/prod/main.pkl
2. prod/main.pkl imports ../utils.pkl, which resolves to URI file://<base path>/utils.pkl
3. utils.pkl reads sample.json, which it will resolve to URI file://<base path>/sample.json (which in your case doesn't exist)

This is why moving the read into prod/main.pkl or reading prod/sample.json from utils.pkl works: you end up with the correct read URI of file://<base path>/prod/sample.json.

Keep in mind that this relative resolution happens regardless of URI scheme, such as when when you use read inside of a module that's part of a package (with scheme package://). If, for example, you import utils.pkl from a package (where it has URI package://<package base>#/utils.pkl) and this reads sample.json, this file would be expected to be in the package with URI package://<package base>#/sample.json.