Clojure rearrangement and mapping of vector of vectors

Question

Here is the situation: I have a vector of vectors ("data"), a set of headers, a subset of headers ("primary headers"), a constant ("C"), an element-wise function ("f"), and the remaining headers ("secondary headers"). My goal is to take the "data" and produce a new vector of vectors.

Example data:

[[1.0 "A" 2.0]
[1.0 "B" 4.0]]

Example headers:

["o1" "i1" "i2"]

Example primary headers:

 ["i1" "i2"]

Example secondary headers:

 ["o1"]

Example new vector of vectors:

[[(f "A") (f 2.0) C (f 1.0)]
[(f "B") (f 4.0) C (f 1.0)]]

My current attempt is to mapv each row, then map-indexed each element with an if to check for primary membership, then the constant, then map-indexed each element with an if to check for secondary membership, finally conj on the results. But I am not getting it to work right.

Example code:

(mapv (fn [row] (conj (vec (flatten (map-indexed
                                    (fn [idx item] (let [header-name (nth headers idx)] 
                                                        (if (= (some #{header-name} primary-headers) headers-name) (f item))))
                                    row)))

                  C
                  (vec (flatten (map-indexed
                                    (fn [idx item] (let [header-name (nth headers idx)] 
                                                        (if (= (some #{header-name} secondary-headers) headers-name) (f item))))
                                    row)))))
 data)

Answer 1

You should consider using core.matrix for stuff like this. It is a very flexible tool for multi-dimensional array programming in Clojure.

Most array-manipulation operations are likely to be 1-2 liners.....

(def DATA [[1.0 "A" 2.0]
           [1.0 "B" 4.0]])

(emap (partial str "f:") (transpose (mapv #(get-column DATA %) [1 0 2])))
=> [["f:A" "f:1.0" "f:2.0"] 
    ["f:B" "f:1.0" "f:4.0"]]

You might need to look up the column names to calculate the [1 0 2] vector but hopefully this gives you a good idea how to do this....

Answer 2

Not sure if I got your problem right, but looks like you want something like this:

(defn magic [data h p s f]
  (let [idx (map (into {} (map-indexed #(vector %2 %1) h))
                 (concat p s))]
    (mapv #(mapv (comp f (partial get %))
                 idx)
          data)))

Here is an example of my magic function:

(magic [[1.0 "A" 2.0]
        [1.0 "B" 4.0]]
       ["o1" "i1" "i2"]
       ["i1" "i2"]
       ["o1"]
       #(str "<" % ">"))

[["<A>" "<2.0>" "<1.0>"]
 ["<B>" "<4.0>" "<1.0>"]]

Let's get a closer look at it.

First of all, I'm calculating permutation index idx. In your case it's (1 2 0). In order to calculate it I'm turning ["o1" "i1" "i2"] into a hash map {"o1" 0, "i1" 1, "i2" 2} and then using it on ("i1" "i2" "o1") sequence of primary and secondary headers.

Then I'm using idx to rearrange data matrix. On this step I'm also applying f function to each element of new rearranged matrix.

Update

I thought that it'll be best to split my complicated magic function into three simpler ones:

(defn getPermutation [h1 h2]
  (map (into {} (map-indexed #(vector %2 %1) h1))
       h2))

(defn permutate [idx data]
  (mapv #(mapv (partial get %) idx)
        data)))

(defn mmap [f data]
  (mapv (partial mapv f)
        data))

Each function here is atomic (i.e. performing a single task), and they all could be easily combined to do exactly what magic function do:

(defn magic [data h p s f]
  (let [idx (getPermutation h (concat p s))]
    (->> data
         (permutate idx)
         (mmap f))))

getPermutation function here calculates idx permutation index vector.

permutate rearranges columns of a matrix data according to given idx vector.

mmap applies function f to each element of a matrix data.

Update 2

Last time I missed the part about adding a constant. So, in order to do so we'll need to change some of the code. Let's change permutate function allowing it to insert new values to the matrix.

(defn permutate [idx data & [default-val]]
  (mapv #(mapv (partial get %) idx (repeat default-val))
        data)))

Now, it'll use default-val if it won't be able to get the element with the specified index idx.

We'll also need a new magic function:

(defn magic2 [data h p s f c]
  (let [idx (getPermutation h (concat p [nil] s))]
    (permutate idx (mmap f data) c)))

I changed the order of applying mmap and permutate functions because it seems that you don't want to apply f to your constant.

And it works:

(magic2 [[1.0 "A" 2.0]
         [1.0 "B" 4.0]]
        ["o1" "i1" "i2"]
        ["i1" "i2"]
        ["o1"]
        #(str "<" % ">")
        "-->")

[["<A>" "<2.0>" "-->" "<1.0>"]
 ["<B>" "<4.0>" "-->" "<1.0>"]]

Answer 3

Given

(def data [[1.0 "A" 2.0] [1.0 "B" 4.0]])
(def headers ["o1" "i1" "i2"])
(def primaries ["i1" "i2"])
(def secondaries ["o1"])

(defn invert-sequence [s] (into {} (map-indexed (fn [i x] [x i]) s)))

... this does the job:

(defn produce [hs ps ss f data const]
  (let [perms (map #(mapv (invert-sequence hs) %) [ps ss])]
    (mapv (fn [v] (->> perms
                       (map #(map (comp f v) %))
                       (interpose [const])
                       (apply concat)
                       vec))
          data)))

Using the example in the question:

(produce headers primaries secondaries #(list 'f %) data 'C)
; [[(f "A") (f 2.0) C (f 1.0)] [(f "B") (f 4.0) C (f 1.0)]]

Using Leonid Beschastny's example:

(produce headers primaries secondaries #(str "<" % ">") data 'C)
; [["<A>" "<2.0>" C "<1.0>"] ["<B>" "<4.0>" C "<1.0>"]]

Using str:

(produce headers primaries secondaries str data 'C)
; [["A" "2.0" C "1.0"] ["B" "4.0" C "1.0"]]

Using identity:

(produce headers primaries secondaries identity data 'C)
; [["A" 2.0 C 1.0] ["B" 4.0 C 1.0]]