2

I am processing data with a variable number of functions, depending on parameters. Each of the processing functions would receive data from its predecessor, process it and pass it on to the next function.

(defn example [data]
  (do-things-to data))

My application flow would be

  1. check parameters and store needed functions in a vector
  2. create a function containing all neccessary steps
  3. call a wrapper functions which does file management and finally applies the function

Mockup:

(let [my-big-fun (reduce comp (filter identity) vector-of-functions)]
  (wrapper lots-a-arguments big-fun)

Now I figured out I need not only pass the data to the functions, but another dataset, too.

(defn new-fun-example [root data]
  (do-things-to-both root data))

Is there a way to do something similar to the reduction I did to the arity-1 function? A simple juxt won't do, as each function changes the data the next one needs. Returning a seq of '(root data) or similars would require a great deal of rewriting in many functions.

Any ideas? I guess the answer is "macro", but I never fiddled with these...

Edit1:

The second argument is a reference to a growing graph data structure, so it doesn't need to be processed by the functions, just passed on somehow. But the functions might originate from different namespaces, so I can't simply put the root in a higher scope to access it. A global def would be possible, but extremely ugly...

While writing this I just thought I might map the functions to partial somehow, before comping them.

Edit2:

The filter identity caused much confusion, it's not part of my question. I should not have included it in my sample at first place. I solved the task as suggested by the quick brown fox and apologise for being obscure at times. Minimal solution-like example:

(defn example [root data]
  (swap! root + data))

(defn fn-chainer [vector-of-functions]
  (let [the-root (atom 0)
    ; this filter step is just required to remove erroneously apperaring nils 
    ; from the vector of functions - not part of the question
    vector-of-functions (filter identity vector-of-functions)
    ; bake the atom to the functions
    vector-of-functions (mapv #(partial % the-root) vector-of-functions)
    ; now chain each funcion's result as argument to the next one
    my-big-fun (reduce comp vector-of-functions)]
    ; let the function chain process some dataset
    (my-big-fun 5))

; test some function vectors
(fn-chainer [example])
=> 5    ; = 0 + 5
(fn-chainer [example example])
=> 10   ; = 0 + 5 +5
(fn-chainer [example nil example example nil nil])10
=> 20   ; = 0 + 5 + 5 + 5 + 5, nils removed
waechtertroll
  • 607
  • 3
  • 17
  • I'm not sure how you'd compose two functions with arity 2 as each function can only produce one output. Is the output of each function a sequence of two items? – TheQuickBrownFox Sep 18 '15 at 11:28
  • No, it's not. On the other hand, the second input (`root` in the examples) is constant, it's a ref to a data structure root. – waechtertroll Sep 18 '15 at 12:00

2 Answers2

1

As you mention in your edit, you can indeed map your functions into new functions which have root baked in:

(mapv #(partial % root) vector-of-functions)
TheQuickBrownFox
  • 10,544
  • 1
  • 22
  • 35
0

First of all, I feel like there's a mix of many things happening here:

  1. (filter identity) is a transducer, but nothing says the other fns return transducers or if wrapper expects a transducer, and given some of them are going to receive two arguments we can safely say they are not transducers. You probably wanted (partial filter identity) or #(filter identity %).

  2. Why are you using (reduce comp (filter identity) vector-of-functions) instead of (apply comp (cons (partial filter identity) vector-of-functions)?

Focusing on how to compose the functions given that some of them receive more arguments for which you already have the value, you can use partial:

(let [root [1 2 3]
      other-root [4 5 6]
      vector-of-functions [(partial filter identity) example (partial my-fun-example root) (partial other-fun-example root other-root)]
      my-big-fun (apply comp vector-of-functions)]
  (wrapper lots-a-arguments big-fun))

EDIT: I was wrong about using reverse for the apply comp above, (reduce comp [fn1 fn2 fn3]) will return the same result as (apply comp [fn1 fn2 fn3]) when applied

nberger
  • 3,659
  • 17
  • 19
  • Good questions. 1. I just filter the vector of functions so I won't accidently comp `nil`s in the devel-phase 2. because I'm new to clojure and never quite grasped the concept of `apply` :-) Would it be suited better, and if yes - why? I just edited thoughts about using `partial` while you were typing your answer, so I'm delightet you suggest that too. The only "but" in your solution is: the vector is populated programmatically, so I'd need some call to `map`? to add the `partial root` part I'm not sure about. – waechtertroll Sep 18 '15 at 12:18
  • Re "just filter the vector of functions": So an element in the vector of functions might be nil, or an element in the input (or output, after the comp) sequence of data? If it's the former, I'd do the filter before you get to this point. A precondition of function would be that `vector-of-functions` doesn't have any nils. – nberger Sep 18 '15 at 12:49
  • Re apply: See http://stackoverflow.com/questions/3153396/clojure-reduce-vs-apply. You can use `(reduce comp ...)` but it's less idiomatic IMO. But you'll see some say the opposite. YMMV. Anyways, I think it's important to be more clear about the `(filter identity)` first. Better so if we can remove it from the question :) – nberger Sep 18 '15 at 13:11
  • Re "the vector is populated programmatically": It doesn't matter if it's populated "programatically" (by conjoining functions one by one, by concat smaller vectors of functions, etc) or "by hand" (with a vector literal), in both cases you'll have a reference to the vector in `vector-of-fuctions`, so it doesn't seem you don't need a map or anything else apart from `(apply comp vector-of-functions)` (or `reduce`) – nberger Sep 18 '15 at 13:14
  • *filtering* I wanted to remove unexpected nils from the function vector, not the data. It is just a quick hack for the devel version and can be ignored. *apply* Thanks for the link. I understand, but my mind bends around reducers more easily. I'll try in the near future, I promise ;-) *vector population* My anwser was quite chaotic. I just meant that I still had to map the "partial..." statement programmatically, which you simply did by hand. Maybe it's clearer with my new example now? – waechtertroll Sep 18 '15 at 14:25
  • Instead of `(apply f (cons x xs))`, you can just write `(apply f x xs)`. – amalloy Sep 18 '15 at 19:06
  • Re "I just meant that I still had to map the "partial...": I thought that some of the functions would be of 2-arity, and some 1-arity. That's why I left `example` without the partial application of `root` and `my-fun-example` with `(partial root ...)`. If all of them have the same first argument, sure, it's better to map all of them. – nberger Sep 18 '15 at 22:03
  • 1
    Ah, now I understand your concerns. Thanks for the explanations to apply, anyhow. – waechtertroll Sep 19 '15 at 13:32