TypeScript: async generator

Question

I'd like to have a function like this:

export async function* iterateDir(dir: string) {
    let list = await fs.readdir(dir); // fs-promise implementation of readdir
    for (let file of list) {
        yield file;
    }
}

Which I would use like:

for (let file in iterateDir(dir)) {
    processFile(file);
}

This doesn't work because a function cannot be both async and a generator.

How would I structure the code to achieve the same?

1. If I change the await fs.readdir to callbacks, I assume the outer for..of loop would not wait.
2. If I get rid of the generator and the directory is huge, iterateDir() will be slow.

For reference: async generator function proposal

Answer 1

This is supported in TypeScript 2.3 - tracked issue

It introduces a few new types, notably:

interface AsyncIterable<T> {
    [Symbol.asyncIterator](): AsyncIterator<T>;
}

but most importantly it also introduces for await ... of

for await (const line of readLines(filePath)) {
  console.log(line);
}

where

async function* readLines(path) {
   //await and yield ...
}

Be aware that if you want to try this you will need to configure typescript to let it know you have run-time support (add "esnext.asynciterable" to lib list) you will probably need to polyfill Symbol.asyncIterator. see TS2318: Cannot find global type 'AsyncIterableIterator' - async generator

Answer 2

Stream of future values is called Observable. So the most natural thing is to use a library like RxJS or most.js. If you don't want to introduce a new complex library only to use it once, use the good old approach with callbacks:

const path = require("path");
const fs = require("mz/fs");

async function listDirectory(dirName, cb, level = 0) {
  for (let fileName of await fs.readdir(dirName)) {
    let absName = path.resolve(dirName, fileName);
    let isDirectory = (await fs.stat(absName)).isDirectory();
    cb({ level, fileName });
    if (isDirectory) {
      await listDirectory(absName, cb, level + 1);
    }
  }
}

listDirectory(".", ({ level, fileName }) => {
  console.log(" ".repeat(level) + fileName);
});

If you try to convert the callback to some better abstraction, you will rediscover RxJS sooner or later. It's similar to converting one-shot callbacks to promises.