1

Say I have a "type" like this:

{
  a: {
    b: {
      c: {
        d: string
        e: boolean
      }
    },
    x: string
    y: number
    z: string
  }
}

At each object node, I want to get notified if all of the children are "resolved" to a value. So for example:

const a = new TreeObject()
a.watch((a) => console.log('a resolved', a))

const b = a.createObject('b')
b.watch((b) => console.log('b resolved', b))

const c = b.createObject('c')
c.watch((c) => console.log('c resolved', c))

const d = c.createLiteral('d')
d.watch((d) => console.log('d resolved', d))

const e = c.createLiteral('e')
e.watch((e) => console.log('e resolved', e))

const x = a.createLiteral('x')
x.watch((x) => console.log('x resolved', x))

const y = a.createLiteral('y')
y.watch((y) => console.log('y resolved', y))

d.set('foo')
// logs:
// d resolved

e.set('bar')
// logs:
// e resolved
// c resolved
// b resolved

y.set('hello')
// logs:
// y resolved

x.set('world')
// logs:
// x resolved
// a resolved

That is the base case. The more complex case, which is what I've been trying to solve for, is matching against a subset of properties, like this:

// receive 'b' only if b.c.d is resolved.
// '3' for 3 args
a.watch3('b', {
  c: {
    d: true
  }
}, () => {
  console.log('b with b.c.d resolved')
})

You can have multiple "watchers" per property node, like this:

a.watch3('b', { c: { d: true } }, () => {
  console.log('get b with b.c.d resolved')
})

a.watch3('b', { c: { e: true } }, () => {
  console.log('get b with b.c.e resolved')
})

a.watch2('x', () => {
  console.log('get x when resolved')
})

// now if were were to start from scratch setting properties fresh:
x.set('foo')
// logs:
// get x when resolved

e.set('bar')
// logs:
// get b with b.c.e resolved

How can you neatly set this up? I have been trying for a long time to wrap my head around it but not getting far (as seen in this TS playground.

type Matcher = {
  [key: string]: true | Matcher
}

type Callback = () => void

class TreeObject {
  properties: Record<string, unknown>

  callbacks: Record<string, Array<{ matcher?: Matcher, callback: Callback }>>

  parent?: TreeObject

  resolved: Array<Callback>

  constructor(parent?: TreeObject) {
    this.properties = {}
    this.callbacks = {}
    this.parent = parent
    this.resolved = []
  }

  createObject(name: string) {
    const tree = new TreeObject(this)
    this.properties[name] = tree
    return tree
  }
  
  createLiteral(name: string) {
    const tree = new TreeLiteral(this, () => {
      // somehow start keeping track of decrementing what we have matched so far
      // and when it is fully decremented, trigger the callback up the chain.
    })
    this.properties[name] = tree
    return tree
  }

  watch3(name: string, matcher: Matcher, callback: Callback) {
    const list = this.callbacks[name] ??= []
    list.push({ matcher, callback })
  }

  watch2(name: string, callback: Callback) {
    const list = this.callbacks[name] ??= []
    list.push({ callback })
  }

  watch(callback: Callback) {
    this.resolved.push(callback)
  }
}

class TreeLiteral {
  value: any

  parent: TreeObject

  callback: () => void

  resolved: Array<Callback>

  constructor(parent: TreeObject, callback: () => void) {
    this.value = undefined
    this.parent = parent
    this.callback = callback
    this.resolved = []
  }

  set(value: any) {
    this.value = value
    this.resolved.forEach(resolve => resolve())
    this.callback()
  }

  watch(callback: Callback) {
    this.resolved.push(callback)
  }
}

const a = new TreeObject()
a.watch(() => console.log('a resolved'))

const b = a.createObject('b')
b.watch(() => console.log('b resolved'))

const c = b.createObject('c')
c.watch(() => console.log('c resolved'))

const d = c.createLiteral('d')
d.watch(() => console.log('d resolved'))

const e = c.createLiteral('e')
e.watch(() => console.log('e resolved'))

const x = a.createLiteral('x')
x.watch(() => console.log('x resolved'))

const y = a.createLiteral('y')
y.watch(() => console.log('y resolved'))

d.set('foo')
// logs:
// d resolved

e.set('bar')
// logs:
// e resolved
// c resolved
// b resolved

y.set('hello')
// logs:
// y resolved

x.set('world')
// logs:
// x resolved
// a resolved

How can you define the watch3 and related methods to accept their "matchers" and callback, and properly call the callback when the matchers' properties are all fulfilled?

It gets tricky because you can work in two directions:

  1. The value could have already been resolved in the past, before you added your watchers/listeners. It should still be notified right away in that case.
  2. The value can be resolved in the future, after you added your watchers. It should be notified only once fulfilled.

Note, the "matcher" syntax is sort of like a GraphQL query, where you simply build an object tree with the leaves set to true on what you want.

Lance
  • 75,200
  • 93
  • 289
  • 503
  • ① Looks like you're just trying to re-implement promises from scratch for some reason. Have you tried using promises? ② How many times can you `set()` a single node? ③ Right now this question is almost certainly too broad for SO; you've got a bunch of pieces of things and your example code doesn't seem to work for the "base case" you're not directly asking about. Is it possible to break this down so that you're clearly asking a single well-defined question? Either get the basic stuff working yourself and then ask about subsets, or forget about the subsets and ask about the basic stuff? – jcalz Jun 16 '23 at 13:23
  • 1
    (more): "At each object node, I want to get notified if any of the children are "resolved" to a value." That *any* should probably be changed to *all*, right? Otherwise you'd get notified at `a` whenever anything under it was `set`. Anyway I'll probably disengage now, good luck! – jcalz Jun 16 '23 at 13:59
  • What if an object is resolved, but then *after* that has happened, you still call `createLiteral` on that object or one of its descendants? Should the object turn back to an unresolved state? Should any callback notifications be called a second time when also *that* literal is resolved? – trincot Jun 16 '23 at 14:46
  • For `watch` you have defined that the callback gets an argument, but looking at the desired logs, it seems that value is supposed to print as an empty string. Please clarify what that argument is supposed to be. For `watch2` and `watch3` there is no such argument in your examples. Is this difference intended? – trincot Jun 16 '23 at 14:58
  • Why in `a.watch3('b', {c: {d: true}}, cb)` is `b` given as separate argument? Why not integrate that info in the matcher, like `a.watch3({b: {c: {d: true}}}, cb)`? – trincot Jun 16 '23 at 15:25
  • What if the matcher points to a structure that has not yet been defined, but gets defined later? – trincot Jun 16 '23 at 15:27
  • @trincot the `name` attribute like `b` is what is returned, but the matcher is what is matched against is how I think of it. For the first question, can you paint an example of when that would occur? Sorry, thanks for the clarifying questions. For `watch`, let's just say the callback takes no arguments. – Lance Jun 16 '23 at 18:21
  • @trincot for the last question, I'm not sure what you mean, you mean the matcher is like a thunk? Or are you saying if the matcher matches a structure which hasn't been created (i.e. the tree object hasn't been created), what happens? In that case, as the structure is created it would count down the properties until it is resolved, like setting the property any other way I think. – Lance Jun 16 '23 at 18:33
  • @jcalz yes any => all, my bad. – Lance Jun 16 '23 at 18:34
  • For the first question: we could say that a callback function is called at most once, and then never again, even if the structure (downstream) is extended with more literals and they get values assigned to them. Would that be as desired? For the last question: OK, I think I got my answer. – trincot Jun 16 '23 at 18:34
  • @jcalz the hard part I'm struggling with is just a small portion of the overall code, which I simplified a lot from my project. The problem I'm facing is just how to write even a rough algorithm to call the callback when the matcher is matched. You have to copy the matcher partially down as you create subnodes, and it gets really convoluted in my attempts. – Lance Jun 16 '23 at 18:36
  • @trincot yes ideally the callbacks are called only once, but I haven't yet imagined cases where it might be necessary to call it more than once. So for now let's just say they are only called once, and like you are saying. Thank you. – Lance Jun 16 '23 at 18:37
  • (don't mind the naming but) [here](https://gist.github.com/lancejpollard/44b707b59e12ed373cfed5f2d191d896) is my latest brainstorm attempt at solving the problem, copied straight from the project. Extra convoluted. – Lance Jun 16 '23 at 18:42

1 Answers1

0

Some preliminary thoughts:

  • From my understanding the first argument of a.watch3('b', {c: {d: true}}, cb) is a name that must match one of the properties, and the Matcher object should "map" to that value of that property. But then I would suggest to map the Matcher object with the current object (this) and put b inside that Matcher, so that you can omit the first argument:

    a.watch3({b: {c: {d: true}}}, cb);
    
  • I would use one watch method for all signatures, where the callback is always the first argument, and the Matcher object is the optional second argument. The name argument is in my opinion not necessary (previous point).

  • I will assume that a callback can only be called once. This is an assumption that becomes important in the following scenario:

    const a = new TreeObject();
    const b = a.createObject('b');
    const c = b.createObject('c');
    const d = c.createLiteral('d');
    
    a.watch(() => console.log("a resolved"));
    
    d.set('foo'); // This triggers "a resolved"
    
    // This makes a unresolved again
    const e = c.createLiteral('e');
    // but resolving a again will not trigger the same callback
    e.set('bar'); // This does not trigger "a resolved" anymore
    // Let's do that again: unresolve a...
    const f = c.createLiteral('f');
    // But now we add a new callback before the resolution occurs:
    a.watch(() => console.log("a resolved AGAIN"));
    f.set('baz'); // This triggers "a resolved AGAIN" (only)
    

    This assumption means that a callback can/must be unregistered once it gets called.

  • If a callback is registered when there are no literals yet, the object will be considered as not yet resolved -- to become resolved, there must be at least one literal in the (downstream) object structure, and all downstream literals must have received a value (or a subset, in case a Matcher object is provided)

  • If a Matcher object is provided that references a structure that is not (completely) present, the registered callback will not be called until that structure has been completely built, and the corresponding literals have received values. So we would need a kind of "pending matchers" property that needs to be checked whenever a missing property gets created that would enable one or more matchers to apply to that new property.

  • If the Matcher object has a true value where the actual object structure has a deeper nested object structure instead of a literal, that true will be interpreted as "all below this point" must have received values.

  • If the Matcher object has an object where the actual object has a literal, that matcher will never get resolved.

  • I updated this answer so that matchers are turned into standard watchers on the endpoint nodes (without matcher) whenever that becomes possible (as the corresponding structure is completed), so that all can be managed with counters that are updated upstream from the literal up to the root. When a counter becomes zero, it means all necessary items are resolved. One important detail here, is that a matcher object will have its own callbacks created for each of its endpoints, and when those are called it will keep track of a separate counter. When that one becomes zero, the original callback is called.

Here is how that could be coded:

type Matcher = true | {
    [key: string]: Matcher
};

type Callback = () => void;

type Listener = { callback: Callback, matcher: Matcher };

type TreeNode = TreeObject | TreeLiteral;

abstract class TreeElement  {
    #parent?: TreeObject;
    #unresolvedCount = 0;
    #hasLiterals = false;
    #callbacks: Array<Callback> = [];
    
    constructor(parent?: TreeObject) {
        this.#parent = parent;
    }

    notify(isResolved: boolean) { // bubbles up from a TreeLiteral, when created and when resolved
        if (isResolved) {
            this.#unresolvedCount--;
            if (this.#unresolvedCount == 0) {
                for (const cb of this.#callbacks.splice(0)) {
                    cb();
                }
            }
        } else {
            this.#unresolvedCount++;
            this.#hasLiterals = true;
        }
        this.#parent?.notify(isResolved); // bubble up
    }
    
    watch(callback: Callback) {
        if (this.#hasLiterals && this.#unresolvedCount == 0) {
            callback();
        } else {
            this.#callbacks.push(callback);
        }
    }

}

class TreeObject extends TreeElement {
    #properties: Record<string, TreeNode> = {};
    #pendingMatchers: Record<string, Array<Listener>> = {};

    #attach(name: string, child: TreeNode) {
        this.#properties[name] = child;
        // If this name is used by one or more pending matchers, remove them as pending,
        //   and watch the nested matcher(s) on the newly created child.
        if (this.#pendingMatchers[name]) {
            for (const {callback, matcher} of this.#pendingMatchers[name].splice(0)) {
                child.watch(callback, matcher);
            }
        }
    }

    createObject(name: string) {
        if (this.#properties[name]) throw new Error(`Cannot create ${name}: it is already used`);
        const obj = new TreeObject(this);
        this.#attach(name, obj);
        return obj;
    }

    createLiteral(name: string) {
        if (this.#properties[name]) throw new Error(`Cannot create ${name}: it is already used`);
        const obj = new TreeLiteral(this);
        this.#attach(name, obj);
        return obj;
    }

    watch(callback: Callback, matcher: Matcher=true) {
        if (matcher === true) {
            super.watch(callback);
        } else {
            let counter = Object.keys(matcher).length;
            // Create a new callback that will call the original callback when all toplevel
            //   entries specified by the matcher have been resolved.
            const newCallback = () => {
                counter--;
                if (counter == 0) {
                    callback();
                }
            };
            for (const key of Object.keys(matcher)) {
                if (this.#properties[key]) {
                    this.#properties[key].watch(newCallback, matcher[key]);
                } else { // suspend the watch until the structure is there
                    (this.#pendingMatchers[key] ??= []).push({
                        callback: newCallback,
                        // Copy the matcher so the caller cannot mutate our matcher
                        matcher: JSON.parse(JSON.stringify(matcher[key]))
                    });
                }
            }
        }

    }
}

class TreeLiteral extends TreeElement {
    #literalValue: any;

    constructor(parent?: TreeObject) {
        super(parent);
        this.notify(false); // Notifiy to the ancestors that there is a new literal
    }

    set(value: any) {
        this.#literalValue = value;
        this.notify(true); // Notifiy to the ancestors that this literal resolved
    }

    valueOf() {
        return this.#literalValue;
    }

    watch(callback: Callback, matcher: Matcher=true) {
        if (matcher === true) {
            super.watch(callback);
        } // else, the matcher references an endpoint that will never be created
    }
}

See it with some test functions on TS Playground

trincot
  • 317,000
  • 35
  • 244
  • 286
  • Interesting and very elegant solution. My only concern with calling `notify()` on every node and iterating through every watcher is in the program I am working on it is for a cyclic dependency resolving compiler, and there could be 2,000 modules each with at least 10 "needs" for calling `el.watch`, with ballpark 200 objects per module, all synchronous. So calling set 400,000 times, or iterating the watchers 4M times, will that be a performance problem? – Lance Jun 16 '23 at 21:16
  • I think this is partly why I couldn't find such a streamlined solution as this, because I was trying to only trigger the minimum stuff on each `literal.set`, somehow... I couldn't see it though, so maybe this is a better approach since it is easier to reason about. – Lance Jun 16 '23 at 21:18
  • Ah also, as it bubbles up, it checks the matcher down all the way, but the property might have already been matched from a child, so is there a way to prevent duplicate checking? If I can somehow formulate a clearer question maybe I can ask in another SO question, not sure. – Lance Jun 16 '23 at 21:20
  • 1
    Let me think about that. I'll come back to you, but need a night's sleep first ;-) – trincot Jun 16 '23 at 21:21
  • 1
    I have updated my answer. I hope this takes into consideration what you mentioned here. Now the `notify` call will only involve an upward walk through the tree to the root, updating counters and potentially calling callbacks, but there is no longer a downstream traversal that is initiated from that. – trincot Jun 17 '23 at 12:46