1

1 The Problematic JavaScript Function

I am dealing with a very problematic function in JS that invokes a callback function an iterated number of times. In particular, the function takes a yaml string and runs a callback function for each number of yaml documents found within the string:

var yaml = require('js-yaml');

yaml.safeLoadAll(data, function (doc) {
  console.log(doc);
  });

So here, if data contains 2 yaml documents, then we will see 2 logs in our console.

2 Dealing with it in ClojureScript

Suppose that string has an unknown number of yaml documents. I'd like to place each of these documents into a javascript array using a core.async channel.

First, I make a function which jams each yaml document into a channel: 

(defn yaml-string->yaml-chan [string]
  (let [c (chan)]
    (go 
      (.safeLoadAll 
       yaml
       string 
      (fn [current-yaml-object] 
        (go 
          (>! c current-yaml-object)
          ;(close! c) ; cant close here or we only get one doc!
        )
      ))  
    ) c ; here we return the channel
  )
)

Then I make a function which sucks up each yaml document from the channel and sticks them into a javascript array (encapsulated in another channel).

(defn yaml-chan->array-chan [c]
  (let [arr (js/Array.) arr-chan (chan) a (atom true)]
    (go
      (reset! a (<! c))
      (while (not-nil? @a)
    (.push arr @a)
    (reset! a (<! c))
      )
      (>! arr-chan arr)
    ) arr-chan
  )
)

Then I attempt to execute the result:

(go (println <! (yaml-chan->yaml-array-chan (yaml-string->yaml-chan string)))

And all I get is #object[cljs.core.async.impl.channels.ManyToManyChannel] :( I think it's because I never closed the original channel for the yaml objects. But how do I do this with that iterated callback function? Where and how do I close that channel?

George
  • 6,927
  • 4
  • 34
  • 67

2 Answers2

1

I don't think that core.async is going to help a whole lot in this situation. This would be difficult in plain javascript as well. safeLoadAll just fires at a callback as long as there are more documents to load. Other than using some sort of wonky timeout check, there is no way to know if it is finished or not (which would not guarentee that everything is loaded, just that no activity has happened within a time threshold).

If you did not need to collect all documents into an array, then you could just process each result using the core.async channel processing. If you truly do need an array, then you should find another method (loop over all documents and load them individually) so you can determine when all file loading is complete.

rabidpraxis
  • 556
  • 3
  • 10
  • So basically, I need to build a custom function that determinues how many documents are in a string (call that number `k`), and then use that knowledge to know when to stop sucking objects from a channel into an array of size `k`. Does that sound about right? – George Jan 03 '16 at 12:14
  • I think that would get you to your desired result, yes. Just out of curiosity, in what circumstance would you have an unknown number of yaml documents within a string? – rabidpraxis Jan 03 '16 at 16:04
0

I would not have your callback generate a channel for each time it is called. Instead, I would create a function which closes over a channel which you pass in as the callback. All the documents will then be pushed onto this channel, which you can then read to get the documents when your ready.

Tim X
  • 4,158
  • 1
  • 20
  • 26
  • AFAIK, that's the approach taken in my example. The problem is that I don't know how to close that channel. – George Jan 03 '16 at 04:19
  • To clarify: the problem is that I don't know how to close that channel: either I close it after the first callback is called (in which case I miss the other callbacks), or I never close it and the channel never terminates! – George Jan 03 '16 at 04:32