partial.lenses: how to use asynchronous operations in optics with L.collect?

Question

I understand that partial.lenses isn't necessarily the best solution to the following problem and that's besides the point in this question. I was trying to list files from a directory using L.collect from the partial.lenses library. Goal is just to get a flattened array of file names.

Problem: instead of using fs.readdirSync I would like to make use of asynchronous, Promise returning versions of the Node's fs API, in my optics.

The following would be the promisified version of the readdir:

const util = require('util')
const fs = require('fs')
const readdirAsync = util.promisify(fs.readdir)

Below is the actual implementation. I would like to know how I can replace the synchronous fs.readdirSync in the readdir function with an asynchronous version.

const L = require("partial.lenses")
const fs = require("fs")
const path = require("path")
const _ = require("lodash")

const basePath = path.basename(`${__dirname}/..`)

const isDirectory = dirent => {
  return dirent instanceof fs.Dirent ? dirent.isDirectory() : false
}

const readDir = path => () => {
  return fs.readdirSync(path, { withFileTypes: true })
}

const leafs = nodePath => {
  return L.cond(
    [
      _.isArray,
      L.lazy(() => [
        L.elems,
        L.choose(({ name }) => leafs(path.join(nodePath, name)))
      ])
    ],
    [
      isDirectory,
      L.lazy(() => [
        readDir(nodePath),
        L.elems,
        L.choose(({ name }) => leafs(path.join(nodePath, name)))
      ])
    ],
    [L.identity]
  )
}

const listFiles = async () =>
  L.collect(
    leafs(basePath),
    fs.readdirSync(basePath, { withFileTypes: true })
  )

There is no need to use Lodash here. You can replace that dependency by switching `_.isArray` to `Array.isArray`. — toastal, Nov 18 '18 at 16:20

score 4 · Accepted Answer · answered Nov 16 '18 at 07:58

This is an interesting question in the sense that Partial Lenses can work on asynchronous problems like this, but the library currently only provides a little bit of direct support for asynchronous operations out-of-the-box. In the future the library could certainly be extended to provide more support for performing asynchronous operations.

Traversals in Partial Lenses build applicative operations. By using different applicatives different kinds of operations, like collecting elements, computing a minimum of elements, or computing a new version of the data structure, can be performed.

Many operations, like collecting elements or computing minimums, have an underlying monoid. Any monoid, like array concatenation

const ConcatArray = {empty: () => [], concat: (l, r) => l.concat(r)}

can be converted to an applicative. In Partial Lenses the L.concat and L.concatAs operations do that internally.

So, for collecting elements asynchronously, we could use an asynchronous version of a concatenating monoid. We can get there by creating a function that converts any monoid to an asynchronous monoid:

const asyncMonoid = m => ({
  empty: async () => m.empty(),
  concat: async (l, r) => m.concat(await l, await r)
})

We can now define an asynchronous version of L.collect as follows:

const collectAsync = L.concatAs(
  async x => [x],
  asyncMonoid(ConcatArray)
)

One more thing we need for solving this problem is a way to get the result of an asynchronous operation so that we can zoom into it using optics. For that we could define a new primitive optic function that awaits for the focus before passing it forward in the optic composition:

const awaitIt = async (x, i, F, xi2yF) => xi2yF(await x, i)

Using the above awaitIt optic, we can now define asynchronous traversals over the files in a file system given a suitable asynchronous readDirectory function:

const filesUnderEntries = L.lazy(() => [
  awaitIt,
  L.elems,
  L.ifElse(L.get('isDir'), ['path', filesUnderDirectory], 'path')
])

const filesUnderDirectory = [L.reread(readDirectory), filesUnderEntries]

We can use the above traversals to examine a directory structure and select files from under it. We can also further compose operations to read those files and examine data from those files. For example

collectAsync(
  [
    filesUnderDirectory,
    L.when(R.test(/\.my$/)),
    L.reread(readFile),
    awaitIt
  ]
)

defines an asynchronous operation that traverses a directory tree and produces the contents of files with the .my extension under the directory tree.

Here is a playground with full code and examples using faked file system operations.

partial.lenses: how to use asynchronous operations in optics with L.collect?

1 Answers1