coll: reduce_scatter_block: add butterfly algorithm

[mkurnosov]

Implements butterfly algorithm for MPI_Reduce_scatter_block.
The algorithm can be used both by commutative and non-commutative operations, for power-of-two and non-power-of-two number of processes.

Signed-off-by: Mikhail Kurnosov [email protected]

[ompiteam-bot]

Can one of the admins verify this patch?

[bosilca]

OK to test.

[bosilca]

typo.

[bosilca]

Not that I really think that it is performance critical, but you can implement this much simpler by extracting the last bit from value and pushing it left on perm.

for( int i = 0; i < nbits; i++ ) {
    perm |= (value & 0x1);
    value >>= 1;
    perm <<= 1;
 }

[jsquyres]

bot:ompi:retest

[mkurnosov]

for( int i = 0; i < nbits; i++ ) {
    perm |= (value & 0x1);
    value >>= 1;
    perm <<= 1;
 }

@bosilca This algorithm forms incorrect result. For example: ompi_mirror_permutation(1,2)=4, but correct result is 2.

I changed the algorithm and corrected a few typos. The new version of ompi_mirror_permutation:

int perm = 0;
for (int i = nbits - 1; i >= 0; i--) {
    perm |= (value & 0x1) << i;
    value >>= 1;
}

[bosilca]

@mkurnosov this is the bit reversal problem, and is heavily use in signal processing. There are many fast solutions, some of them described here.

I really like the one in constant operations:

// swap odd and even bits
value = ((value >> 1) & 0x55555555) | ((value & 0x55555555) << 1);
// swap consecutive pairs
value = ((value >> 2) & 0x33333333) | ((value & 0x33333333) << 2);
// swap nibbles ... 
value = ((value >> 4) & 0x0F0F0F0F) | ((value & 0x0F0F0F0F) << 4);
// swap bytes
value = ((value >> 8) & 0x00FF00FF) | ((value & 0x00FF00FF) << 8);
// swap 2-byte long pairs
value = ( value >> 16                                ) | ( value               << 16);

[mkurnosov]

@bosilca Thanks for your comment. I am familiar with this problem. New version is based on log(n) algorithm (from "Bit Twiddling Hacks" and "Hacker's Delight").