Algo: find max Xor in array for various interval limis, given N inputs, and p,q where 0<=p<=i<=q<=N

Question

the problem statement is the following:

Xorq has invented an encryption algorithm which uses bitwise XOR operations extensively. This encryption algorithm uses a sequence of non-negative integers x1, x2, … xn as key. To implement this algorithm efficiently, Xorq needs to find maximum value for (a xor xj) for given integers a,p and q such that p<=j<=q. Help Xorq to implement this function.

Input

First line of input contains a single integer T (1<=T<=6). T test cases follow.

First line of each test case contains two integers N and Q separated by a single space (1<= N<=100,000; 1<=Q<= 50,000). Next line contains N integers x1, x2, … xn separated by a single space (0<=xi< 2^15). Each of next Q lines describe a query which consists of three integers ai,pi and qi (0<=ai< 2^15, 1<=pi<=qi<= N).

Output

For each query, print the maximum value for (ai xor xj) such that pi<=j<=qi in a single line.

int xArray[100000];
cin >>t; 
for(int j =0;j<t;j++)   
{        
    cin>> n >>q;
    
    //int* xArray = (int*)malloc(n*sizeof(int));
    int i,a,pi,qi;        
    
    for(i=0;i<n;i++)
    {
        cin>>xArray[i];                         
    }
    
    for(i=0;i<q;i++)
    {
      cin>>a>>pi>>qi;
        int max =0;
      for(int it=pi-1;it<qi;it++)
      {
          int t =  xArray[it] ^ a;
          if(t>max)               
              max =t;
          
      }   
      cout<<max<<"\n" ;
    } 

No other assumptions may be made except for those stated in the text of the problem (numbers are not sorted). The code is functional but not fast enough; is reading from stdin really that slow or is there anything else I'm missing?

Answer 1

XOR flips bits. The max result of XOR is 0b11111111.

To get the best result

• if 'a' on ith place has 1 then you have to XOR it with key that has ith bit = 0
• if 'a' on ith place has 0 then you have to XOR it with key that has ith bit = 1

saying simply, for bit B you need !B

Another obvious thing is that higher order bits are more important than lower order bits.

That is:

• if 'a' on highest place has B and you have found a key with highest bit = !B
• then ALL keys that have highest bit = !B are worse that this one

This cuts your amount of numbers by half "in average".

How about building a huge binary tree from all the keys and ordering them in the tree by their bits, from MSB to LSB. Then, cutting the A bit-by-bit from MSB to LSB would tell you which left-right branch to take next to get the best result. Of course, that ignores PI/QI limits, but surely would give you the best result since you always pick the best available bit on i-th level.

Now if you annotate the tree nodes with low/high index ranges of its subelements (performed only done once when building the tree), then later when querying against a case A-PI-QI you could use that to filter-out branches that does not fall in the index range.

The point is that if you order the tree levels like the MSB->LSB bit order, then the decision performed at the "upper nodes" could guarantee you that currently you are in the best possible branch, and it would hold even if all the subbranches were the worst:

Being at level 3, the result of

0b111?????

can be then expanded into

0b11100000
0b11100001
0b11100010

and so on, but even if the ????? are expanded poorly, the overall result is still greater than

0b11011111

which would be the best possible result if you even picked the other branch at level 3rd.

I habe absolutely no idea how long would preparing the tree cost, but querying it for an A-PI-QI that have 32 bits seems to be something like 32 times N-comparisons and jumps, certainly faster than iterating randomly 0-100000 times and xor/maxing. And since you have up to 50000 queries, then building such tree can actually be a good investment, since such tree would be build once per keyset.

Now, the best part is that you actually dont need the whole tree. You may build such from i.e. first two or four or eight bits only, and use the index ranges from the nodes to limit your xor-max loop to a smaller part. At worst, you'd end up with the same range as PiQi. At best, it'd be down to one element.

But, looking at the max N keys, I think the whole tree might actually fit in the memory pool and you may get away without any xor-maxing loop.

Answer 2

I've spent some time google-ing this problem and it seams that you can find it in the context of various programming competitions. While the brute force approach is intuitive it does not really solve the challenge as it is too slow. There are a few contraints in the problem which you need to speculate in order to write a faster algorithm:

• the input consists of max 100k numbers, but there are only 32768 (2^15) possible numbers
  • for each input array there are Q, max 50k, test cases; each test case consists of 3 values, a,pi,and qi. Since 0<=a<2^15 and there are 50k cases, there is a chance the same value will come up again.

I've found 2 ideas for solving the problem: splitting the input in sqrt(N) intervals and building a segment tree ( a nice explanation for these approaches can be found here )

The biggest problem is the fact that for each test case you can have different values for a, and that would make previous results useless, since you need to compute max(a^x[i]), for a small number of test cases. However when Q is large enough and the value a repeats, using previous results can be possible.

I will come back with the actual results once I finish implementing both methods