4

For a few days I was wondering on how to efficiently implement weighted majority voting of m experts in matlab. Here is an example of what I want. Suppose we have 3 experts with weights vector

w=[7 2 6]

Suppose they are voting n times on the options A/B/C/D, so we for example get the following n x m voting matrix, where the columns are votes of each expert.

A B B
C A A
D B A
A A C

Now I'd like to find weighted majority vote for each row. We calculate it by adding the weights of experts which voted for each option, and selecting the maximal weight. For example, in the first row, the option A has cumulative weight of 7 (vote of expert 1) and B has cumulative weight of 8 (votes of expert 2 and 3), and hence the final vote is B. So we get the following cumulative weights matrix and final votes:

A B C D
- - - -
7 8 0 0 -> B
8 0 7 0 -> A
6 2 0 7 -> D
9 0 6 0 -> A

Now, the implementation of this using for loops over number of rows n is more or less straightforward. I am now looking for solution, which doesn't require this potentially lengthy loop and instead uses vector arithmetic. I have had a few ideas, but ran into some problems with each of them, so will not mention them now. If anyone have had similar situation before, please share your solutions.

Thanks.

shiftyscales
  • 441
  • 4
  • 20
  • Can you please share how exactly you find the weights here? What method/algorithm you are following for this? – MaxSteel May 01 '13 at 10:25
  • 1
    Well this really depends on your area. I was implementing DWM as described in Kolter, J. Z., & Maloof, M. A. (2007). Dynamic weighted majority: An ensemble method for drifting concepts. The Journal of Machine Learning Research. This is the method for classification of online data in non-stationary environment. You might be also interested in the following papers: Shapley, L., & Grofman, B. (1984). Optimizing group judgmental accuracy in the presence of interdependencies. Littlestone, N., & Warmuth, M. K. (1994). The Weighted Majority Algorithm. Information and Computation – shiftyscales May 01 '13 at 15:00
  • Thanks. I would read the papers you have suggested. – MaxSteel May 06 '13 at 16:43

1 Answers1

4
w=[7 2 6];

votes = ['A' 'B' 'B'
         'C' 'A' 'A'
         'D' 'B' 'A'
         'A' 'A' 'C'];

options = ['A', 'B', 'C', 'D']';
%'//Make a cube of the options that is number of options by m by n
OPTIONS = repmat(options, [1, size(w, 2), size(votes, 1)]);

%//Compare the votes (streched to make surface) against a uniforma surface of each option
B = bsxfun(@eq, permute(votes, [3 2 1]) ,OPTIONS);

%//Find a weighted sum
W = squeeze(sum(bsxfun(@times, repmat(w, size(options, 1), 1), B), 2))'

%'//Find the options with the highest weighted sum
[xx, i] = max(W, [], 2);
options(i)

result:

B
A
D
A
Dan
  • 45,079
  • 17
  • 88
  • 157
  • Thanks, this works well! Funny, I have actually thought of this solution and implemented it, albeit far less cleanly as you did. But was stuck at the last step, taking maximums row-wise :) – shiftyscales Mar 19 '13 at 13:49
  • haha, I very almost didn't bother with that last step! Glad it worked. Does it speed up the process? Next time post your code if you get so close to the final solution – Dan Mar 19 '13 at 14:27
  • 1
    Did it speed up the process you ask? Well to perform 10k votes with 4 options and variable number of voters (among other stuff, which unfortunately cannot be done without loops) my old code required 300 seconds. With your version, it is 13 seconds! So I am absolutely happy to spend 2 days on this. BTW here is a minor generalization of the code for anyone following this - replacing 4 in the line W = squeeze(sum(bsxfun(@times, repmat(w, 4, 1), B), 2))' with size(options,1). Perhaps you can edit your code to include this, so I don't add another answer. – shiftyscales Mar 19 '13 at 20:21