# How much Algebra does C2 Know? Part 2: Distributivity

In part one of this series of posts, I looked at how important associativity and independence are for fast loops. C2 seems to utilise these properties to generate unrolled and pipelined machine code for loops, achieving higher throughput even in cases where the kernel of the loop is 3x slower according to vendor advertised instruction throughputs. C2 has a weird and wonderful relationship with distributivity, and hints from the programmer can both and help hinder the generation of good quality machine code.

### Viability and Correctness

Distributivity is the simple notion of factoring out brackets. Is this, in general, a viable loop rewrite strategy? This can be utilised to transform the method Scale into FactoredScale, both of which perform floating point arithmetic:


@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public double Scale(DoubleData state) {
double value = 0D;
double[] data = state.data1;
for (int i = 0; i < data.length; ++i) {
value += 3.14159 * data[i];
}
return value;
}

@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public double FactoredScale(DoubleData state) {
double value = 0D;
double[] data = state.data1;
for (int i = 0; i < data.length; ++i) {
value += data[i];
}
return 3.14159 * value;
}


Running the project at github with the argument --include .*scale.*, there may be a performance gain to be had from this rewrite, but it isn’t clear cut:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale thrpt 1 10 7.011606 0.274742 ops/ms 100000
FactoredScale thrpt 1 10 0.621515 0.026853 ops/ms 1000000
Scale thrpt 1 10 6.962434 0.240180 ops/ms 100000
Scale thrpt 1 10 0.671042 0.011686 ops/ms 1000000

With the real numbers it would be completely valid, but floating point arithmetic is not associative. Joseph Darcy explains why in this deep dive on floating point semantics. Broken associativity of addition entails broken distributivity of any operation over it, so the two loops are not equivalent, and they give different outputs (e.g. 15662.513298516365 vs 15662.51329851632 for one sample input). The rewrite isn’t correct even for floating point data, so it isn’t an optimisation that could be applied in good faith, except in a very small number of cases. You have to rewrite the loop yourself and figure out if the small but inevitable differences are acceptable.

### Counterintuitive Performance

Integer multiplication is distributive over addition, and we can check if C2 does this rewrite by running the same code with 32 bit integer values, for now fixing a scale factor of 10 (which seems like an innocuous value, no?)


@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public int Scale_Int(IntData state) {
int value = 0;
int[] data = state.data1;
for (int i = 0; i < data.length; ++i) {
value += 10 * data[i];
}
return value;
}

@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public int FactoredScale_Int(IntData state) {
int value = 0;
int[] data = state.data1;
for (int i = 0; i < data.length; ++i) {
value += data[i];
}
return 10 * value;
}


The results are fascinating:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale_Int thrpt 1 10 28.339699 0.608075 ops/ms 100000
FactoredScale_Int thrpt 1 10 2.392579 0.506413 ops/ms 1000000
Scale_Int thrpt 1 10 33.335721 0.295334 ops/ms 100000
Scale_Int thrpt 1 10 2.838242 0.448213 ops/ms 1000000

The code is doing thousands more multiplications in less time when the multiplication is not factored out of the loop. So what the devil is going on? Inspecting the assembly for the faster loop is revealing

  0x000001c89e499400: vmovdqu ymm8,ymmword ptr [rbp+r13*4+10h]
0x000001c89e499407: movsxd  r10,r13d
0x000001c89e49940a: vmovdqu ymm9,ymmword ptr [rbp+r10*4+30h]
0x000001c89e499411: vmovdqu ymm13,ymmword ptr [rbp+r10*4+0f0h]
0x000001c89e49941b: vmovdqu ymm12,ymmword ptr [rbp+r10*4+50h]
0x000001c89e499422: vmovdqu ymm4,ymmword ptr [rbp+r10*4+70h]
0x000001c89e499429: vmovdqu ymm3,ymmword ptr [rbp+r10*4+90h]
0x000001c89e499433: vmovdqu ymm2,ymmword ptr [rbp+r10*4+0b0h]
0x000001c89e49943d: vmovdqu ymm0,ymmword ptr [rbp+r10*4+0d0h]
0x000001c89e499447: vpslld  ymm11,ymm8,1h
0x000001c89e49944d: vpslld  ymm1,ymm0,1h
0x000001c89e499452: vpslld  ymm0,ymm0,3h
0x000001c89e49945b: vpslld  ymm0,ymm2,3h
0x000001c89e499460: vpslld  ymm7,ymm3,3h
0x000001c89e499465: vpslld  ymm10,ymm4,3h
0x000001c89e49946a: vpslld  ymm15,ymm12,3h
0x000001c89e499470: vpslld  ymm14,ymm13,3h
0x000001c89e499476: vpslld  ymm1,ymm9,3h
0x000001c89e49947c: vpslld  ymm2,ymm2,1h
0x000001c89e499485: vpslld  ymm0,ymm3,1h
0x000001c89e49948e: vpslld  ymm0,ymm4,1h
0x000001c89e499497: vpslld  ymm0,ymm12,1h
0x000001c89e4994a1: vpslld  ymm0,ymm13,1h
0x000001c89e4994ab: vpslld  ymm0,ymm9,1h
0x000001c89e4994b5: vpslld  ymm0,ymm8,3h
0x000001c89e4994ca: vextracti128 xmm3,ymm0,1h
0x000001c89e4994d4: vmovd   xmm3,ebx
0x000001c89e4994dc: vmovd   r10d,xmm3
0x000001c89e4994eb: vextracti128 xmm3,ymm0,1h
0x000001c89e4994f5: vmovd   xmm3,r10d
0x000001c89e4994fe: vmovd   r11d,xmm3
0x000001c89e49950d: vextracti128 xmm0,ymm2,1h
0x000001c89e499517: vmovd   xmm0,r11d
0x000001c89e499520: vmovd   r10d,xmm0
0x000001c89e49952f: vextracti128 xmm3,ymm0,1h
0x000001c89e499539: vmovd   xmm3,r10d
0x000001c89e499542: vmovd   r11d,xmm3
0x000001c89e499551: vextracti128 xmm0,ymm2,1h
0x000001c89e49955b: vmovd   xmm0,r11d
0x000001c89e499564: vmovd   r10d,xmm0
0x000001c89e499573: vextracti128 xmm3,ymm0,1h
0x000001c89e49957d: vmovd   xmm3,r10d
0x000001c89e499586: vmovd   r11d,xmm3
0x000001c89e499595: vextracti128 xmm0,ymm2,1h
0x000001c89e49959f: vmovd   xmm0,r11d
0x000001c89e4995a8: vmovd   r10d,xmm0
0x000001c89e4995b7: vextracti128 xmm1,ymm2,1h
0x000001c89e4995c1: vmovd   xmm1,r10d
0x000001c89e4995ca: vmovd   ebx,xmm1


The loop is aggressively unrolled, pipelined, and vectorised. Moreover, the multiplication by ten results not in a multiplication but two left shifts (see VPSLLD) and an addition. Note that x << 1 + x << 3 = x * 10 and C2 seems to know it; this rewrite can be applied because it can be proven statically that the factor is always 10. The “optimised” loop doesn’t vectorise at all (and I have no idea why not – isn’t this a bug? Yes it is.)

  0x000002bbebeda3c8: add     ebx,dword ptr [rbp+r8*4+14h]
0x000002bbebeda3ef: cmp     r13d,r11d
0x000002bbebeda3f2: jl      2bbebeda3c0h


This is a special case: data is usually dynamic and variable, so the loop cannot always be proven to be equivalent to a linear combination of bit shifts. The routine is compiled for all possible parameters, not just statically contrived cases like the one above, so you may never see this assembly in the wild. However, even with random factors, the slow looking loop is aggressively optimised in a way the hand “optimised” code is not:


@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public int Scale_Int_Dynamic(ScaleState state) {
int value = 0;
int[] data = state.data;
int factor = state.randomFactor();
for (int i = 0; i < data.length; ++i) {
value += factor * data[i];
}
return value;
}

@CompilerControl(CompilerControl.Mode.DONT_INLINE)
@Benchmark
public int FactoredScale_Int_Dynamic(ScaleState state) {
int value = 0;
int[] data = state.data;
int factor = state.randomFactor();
for (int i = 0; i < data.length; ++i) {
value += data[i];
}
return factor * value;
}


Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale_Int_Dynamic thrpt 1 10 26.100439 0.340069 ops/ms 100000
FactoredScale_Int_Dynamic thrpt 1 10 1.918011 0.297925 ops/ms 1000000
Scale_Int_Dynamic thrpt 1 10 30.219809 2.977389 ops/ms 100000
Scale_Int_Dynamic thrpt 1 10 2.314159 0.378442 ops/ms 1000000

Far from seeking to exploit distributivity to reduce the number of multiplication instructions, it seems to almost embrace the extraneous operations as metadata to drive optimisations. The assembly for Scale_Int_Dynamic confirms this (it shows vectorised multiplication, not shifts, within the loop):


0x000001f5ca2fa200: vmovdqu ymm0,ymmword ptr [r13+r14*4+10h]
0x000001f5ca2fa207: vpmulld ymm11,ymm0,ymm2
0x000001f5ca2fa20c: movsxd  r10,r14d
0x000001f5ca2fa20f: vmovdqu ymm0,ymmword ptr [r13+r10*4+30h]
0x000001f5ca2fa216: vmovdqu ymm1,ymmword ptr [r13+r10*4+0f0h]
0x000001f5ca2fa220: vmovdqu ymm3,ymmword ptr [r13+r10*4+50h]
0x000001f5ca2fa227: vmovdqu ymm7,ymmword ptr [r13+r10*4+70h]
0x000001f5ca2fa22e: vmovdqu ymm6,ymmword ptr [r13+r10*4+90h]
0x000001f5ca2fa238: vmovdqu ymm5,ymmword ptr [r13+r10*4+0b0h]
0x000001f5ca2fa242: vmovdqu ymm4,ymmword ptr [r13+r10*4+0d0h]
0x000001f5ca2fa24c: vpmulld ymm9,ymm0,ymm2
0x000001f5ca2fa251: vpmulld ymm4,ymm4,ymm2
0x000001f5ca2fa256: vpmulld ymm5,ymm5,ymm2
0x000001f5ca2fa25b: vpmulld ymm6,ymm6,ymm2
0x000001f5ca2fa260: vpmulld ymm8,ymm7,ymm2
0x000001f5ca2fa265: vpmulld ymm10,ymm3,ymm2
0x000001f5ca2fa26a: vpmulld ymm3,ymm1,ymm2
0x000001f5ca2fa279: vextracti128 xmm0,ymm1,1h
0x000001f5ca2fa283: vmovd   xmm0,ebx
0x000001f5ca2fa28b: vmovd   r10d,xmm0
0x000001f5ca2fa29a: vextracti128 xmm0,ymm1,1h
0x000001f5ca2fa2a4: vmovd   xmm0,r10d
0x000001f5ca2fa2bc: vextracti128 xmm1,ymm0,1h
0x000001f5ca2fa2c6: vmovd   xmm1,r11d
0x000001f5ca2fa2cf: vmovd   r10d,xmm1
0x000001f5ca2fa2de: vextracti128 xmm0,ymm1,1h
0x000001f5ca2fa2e8: vmovd   xmm0,r10d
0x000001f5ca2fa2f1: vmovd   r11d,xmm0
0x000001f5ca2fa300: vextracti128 xmm1,ymm0,1h
0x000001f5ca2fa30a: vmovd   xmm1,r11d
0x000001f5ca2fa313: vmovd   r10d,xmm1
0x000001f5ca2fa322: vextracti128 xmm0,ymm1,1h
0x000001f5ca2fa32c: vmovd   xmm0,r10d
0x000001f5ca2fa335: vmovd   r11d,xmm0
0x000001f5ca2fa344: vextracti128 xmm1,ymm0,1h
0x000001f5ca2fa34e: vmovd   xmm1,r11d
0x000001f5ca2fa357: vmovd   r10d,xmm1
0x000001f5ca2fa366: vextracti128 xmm7,ymm1,1h
0x000001f5ca2fa370: vmovd   xmm7,r10d
0x000001f5ca2fa379: vmovd   ebx,xmm7


There are two lessons to be learnt here. The first is that what you see is not what you get. The second is about the correctness of asymptotic analysis. If hierarchical cache renders asymptotic analysis bullshit (linear time but cache friendly algorithms can, and do, outperform logarithmic algorithms with cache misses), optimising compilers render the field practically irrelevant.

# Zeroing Negative Values in Arrays Efficiently

Replacing negatives with zeroes in large arrays of values is a primitive function of several complex financial risk measures, including potential future exposure (PFE) and the liquidity coverage ratio (LCR). While this is not an interesting operation by any stretch of the imagination, it is useful and there is significant benefit in its performance. This is an operation that can be computed very efficiently using the instruction VMAXPD. For Intel Xeon processors, this instruction requires half a cycle to calculate and has a latency (how long before another instruction can use its result) of four cycles. There is currently no way to trick Java into using this instruction for this simple operation, though there is a placeholder implementation on the current DoubleVector prototype in Project Panama which may do so.

### C++ Intel Intrinsics

It’s possible to target instructions from different processor vendors, in my case Intel, by using intrinsic functions which expose instructions as high level functions. The code looks incredibly ugly but it works. Here is a C++ function for 256 bit ymm registers:


void zero_negatives(const double* source, double* target, const size_t length) {
for (size_t i = 0; i + 3 < length; i += 4) {
__m256d vector = _mm256_load_pd(source + i);
__m256d zeroed = _mm256_max_pd(vector, _mm256_setzero_pd());
_mm256_storeu_pd(target + i, zeroed);
}
}


The function loads doubles into 256 bit vectors, within each vector replaces the negative values with zero, and writes them back into an array. It generates the following assembly code (which, incidentally, is less of a shit show to access than in Java):


void zero_negatives(const double* source, double* target, const size_t length) {
00007FF746EE5110  mov         qword ptr [rsp+18h],r8
00007FF746EE5115  mov         qword ptr [rsp+10h],rdx
00007FF746EE511A  mov         qword ptr [rsp+8],rcx
00007FF746EE511F  push        r13
00007FF746EE5121  push        rbp
00007FF746EE5122  push        rdi
00007FF746EE5123  sub         rsp,250h
00007FF746EE512A  mov         r13,rsp
00007FF746EE512D  lea         rbp,[rsp+20h]
00007FF746EE5132  and         rbp,0FFFFFFFFFFFFFFE0h
00007FF746EE5136  mov         rdi,rsp
00007FF746EE5139  mov         ecx,94h
00007FF746EE513E  mov         eax,0CCCCCCCCh
00007FF746EE5143  rep stos    dword ptr [rdi]
00007FF746EE5145  mov         rcx,qword ptr [rsp+278h]
for (size_t i = 0; i + 3 < length; i += 4) {
00007FF746EE514D  mov         qword ptr [rbp+8],0
00007FF746EE5155  jmp         zero_negatives+53h (07FF746EE5163h)
00007FF746EE5157  mov         rax,qword ptr [rbp+8]
00007FF746EE515F  mov         qword ptr [rbp+8],rax
00007FF746EE5163  mov         rax,qword ptr [rbp+8]
00007FF746EE516B  cmp         rax,qword ptr [length]
00007FF746EE5172  jae         zero_negatives+0DDh (07FF746EE51EDh)
__m256d vector = _mm256_load_pd(source + i);
00007FF746EE5174  mov         rax,qword ptr [source]
00007FF746EE517B  mov         rcx,qword ptr [rbp+8]
00007FF746EE517F  lea         rax,[rax+rcx*8]
00007FF746EE5183  vmovupd     ymm0,ymmword ptr [rax]
00007FF746EE5187  vmovupd     ymmword ptr [rbp+180h],ymm0
00007FF746EE518F  vmovupd     ymm0,ymmword ptr [rbp+180h]
00007FF746EE5197  vmovupd     ymmword ptr [rbp+40h],ymm0
__m256d zeroed = _mm256_max_pd(vector, _mm256_setzero_pd());
00007FF746EE519C  vxorpd      xmm0,xmm0,xmm0
00007FF746EE51A0  vmovupd     ymmword ptr [rbp+200h],ymm0
00007FF746EE51A8  vmovupd     ymm0,ymmword ptr [rbp+40h]
00007FF746EE51B5  vmovupd     ymmword ptr [rbp+1C0h],ymm0
00007FF746EE51BD  vmovupd     ymm0,ymmword ptr [rbp+1C0h]
00007FF746EE51C5  vmovupd     ymmword ptr [rbp+80h],ymm0
_mm256_storeu_pd(target + i, zeroed);
00007FF746EE51CD  mov         rax,qword ptr [target]
00007FF746EE51D4  mov         rcx,qword ptr [rbp+8]
00007FF746EE51D8  lea         rax,[rax+rcx*8]
00007FF746EE51DC  vmovupd     ymm0,ymmword ptr [rbp+80h]
00007FF746EE51E4  vmovupd     ymmword ptr [rax],ymm0
}
00007FF746EE51E8  jmp         zero_negatives+47h (07FF746EE5157h)
}
00007FF746EE51ED  lea         rsp,[r13+250h]
00007FF746EE51F4  pop         rdi
00007FF746EE51F5  pop         rbp
00007FF746EE51F6  pop         r13
00007FF746EE51F8  ret


This code is noticeably fast. I measured the throughput averaged over 1000 iterations, with an array of 10 million doubles (800MB) uniformly distributed between +/- 1E7, to quantify the throughput in GB/s and iterations/s. This code does between 4.5 and 5 iterations per second, which translates to processing approximately 4GB/s. This seems high, and since I am unaware of best practices in C++, if the measurement is flawed, I would gratefully be educated in the comments.


void benchmark() {
const size_t length = 1E8;
double* values = new double[length];
fill_array(values, length);
double* zeroed = new double[length];
auto start = std::chrono::high_resolution_clock::now();
int iterations = 1000;
for (int i = 0; i < iterations; ++i) {
zero_negatives(values, zeroed, length);
}
auto end = std::chrono::high_resolution_clock::now();
auto nanos = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();
double thrpt_s = (iterations * 1E9) / nanos;
double thrpt_gbps = (thrpt_s * sizeof(double) * length) / 1E9;
std::cout << thrpt_s << "/s" << std::endl;
std::cout << thrpt_gbps << "GB/s" << std::endl;
delete[] values;
delete[] zeroed;
}


While I am sure there are various ways an expert could tweak this for performance, this code can’t get much faster unless there are 512 bit zmm registers available, in which case it would be wasteful. While the code looks virtually the same for AVX512 (just replace “256” with “512”), portability and efficiency are at odds. Handling the mess of detecting the best instruction set for the deployed architecture is the main reason for using Java in performance sensitive (but not critical) applications. But this is not the code the JVM generates.

### Java Auto-Vectorisation (Play Your Cards Right)

There is currently no abstraction modelling vectorisation in Java. The only access available is if the compiler engineers implement an intrinsic, or auto-vectorisation, which will try, and sometimes succeed admirably, to translate your code to a good vector implementation. There is currently a prototype project for explicit vectorisation in Project Panama. There are a few ways to skin this cat, and it’s worth looking at the code they generate and the throughput available from each approach.

There is a choice between copying the array and zeroing out the negatives, and allocating a new array and only writing the non-negative values. There is another choice between an if statement and branchless code using Math.max. This results in the following four implementations which I measure on comparable data to the C++ benchmark (10 million doubles, normally distributed with mean zero). To be fair to the Java code, as in the C++ benchmarks, the cost of allocation is isolated by writing into an array pre-allocated once per benchmark. This penalises the approaches where the array is copied first and then zeroed wherever the value is negative. The code is online at github.


@Benchmark
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
double[] data = state.data;
double[] result = state.target;
System.arraycopy(data, 0, result, 0, data.length);
for (int i = 0; i < result.length; ++i) {
if (result[i] < 0D) {
result[i] = 0D;
}
}
return result;
}

@Benchmark
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
public double[] BranchyNewArray(ArrayWithNegatives state) {
double[] data = state.data;
double[] result = state.target;
for (int i = 0; i < result.length; ++i) {
result[i] = data[i] < 0D ? 0D : data[i];
}
return result;
}

@Benchmark
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
public double[] NewArray(ArrayWithNegatives state) {
double[] data = state.data;
double[] result = state.target;
for (int i = 0; i < result.length; ++i) {
result[i] = Math.max(data[i], 0D);
}
return result;
}

@Benchmark
@CompilerControl(CompilerControl.Mode.DONT_INLINE)
double[] data = state.data;
double[] result = state.target;
System.arraycopy(data, 0, result, 0, data.length);
for (int i = 0; i < result.length; ++i) {
result[i] = Math.max(result[i], 0D);
}
return result;
}


None of these implementations comes close to the native code above. The best implementation performs 1.8 iterations per second which equates to processing approximately 1.4GB/s, vastly inferior to the 4GB/s achieved with Intel intrinsics. The results are below:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit
BranchyCopyAndMask thrpt 1 10 1.314845 0.061662 ops/s
BranchyNewArray thrpt 1 10 1.802673 0.061835 ops/s
CopyAndMask thrpt 1 10 1.146630 0.018903 ops/s
NewArray thrpt 1 10 1.357020 0.116481 ops/s

As an aside, there is a very interesting observation to make, worthy of its own post: if the array consists only of positive values, the “branchy” implementations run very well, at speeds comparable to the zero_negatives (when it ran with 50% negatives). The ratio of branch hits to misses is an orthogonal explanatory variable, and the input data, while I often don’t think about it enough, is very important.

I only looked at the assembly emitted for the fastest version (BranchyNewArray) and it doesn’t look anything like zero_negatives, though it does use some vectorisation – as pointed out by Daniel Lemire in the comments, this code has probably not been vectorised and is probably using SSE2 (indeed only quad words are loaded into 128 bit registers):

  0x000002ae309c3d5c: vmovsd  xmm0,qword ptr [rdx+rax*8+10h]
0x000002ae309c3d62: vxorpd  xmm1,xmm1,xmm1
0x000002ae309c3d66: vucomisd xmm0,xmm1


I don’t really understand, and haven’t thought about, the intent of the emitted code, but it makes extensive use of the instruction VUCOMISD for comparisons with zero, which has a lower latency but lower throughput than VMAXPD. It would certainly be interesting to see how Project Panama does this. Perhaps this should just be made available as a fail-safe intrinsic like Arrays.mismatch?

# Project Panama and Population Count

Project Panama introduces a new interface Vector, where the specialisation for long looks like a promising substrate for an explicitly vectorised bit set. Bit sets are useful for representing composable predicates over data sets. One obvious omission on this interface, required for an adequate implementation of a bit set, is a bit count, otherwise known as population count. Perhaps this is because the vector API aims to generalise across primitive types, whereas population count is only meaningful for integral types. Even so, if Vector can be interpreted as a wider integer, then it would be consistent to add this to the interface. If the method existed, what possible implementation could it have?

In x86, the population count of a 64 bit register is computed by the POPCNT instruction, which is exposed in Java as an intrinsic in Long.bitCount. There is no SIMD equivalent in any extension set until VPOPCNTD/VPOPCNTQ in AVX-512. Very few processors (at the time of writing) support AVX-512, and only the Knights Mill processor supports this extension; there are not even Intel intrinsics exposing these instructions yet.

The algorithm for vectorised population count adopted by the clang compiler is outlined in this paper, which develops on an algorithm designed for 128 bit registers and SSE instructions, presented by Wojciech Muła on his blog in 2008. This approach is shown in the paper to outperform scalar code using POPCNT and 64 bit registers, almost doubling throughput when 256 bit ymm registers are available. The core algorithm (taken from figure 10 in the paper) returns a vector of four 64 bit counts, which can then be added together in a variety of ways to form a population count, proceeds as follows:


// The Muła Function
__m256i count(__m256i v) {
__m256i lookup = _mm256_setr_epi8(
0, 1, 1, 2, 1, 2, 2, 3,
1, 2, 2, 3, 2, 3, 3, 4,
0, 1, 1, 2, 1, 2, 2, 3,
1, 2, 2, 3, 2, 3, 3, 4);
__m256i hi = _mm256_and_si256(_mm256_srli_epi32(v, 4), low_mask);
__m256i popcnt1 = _mm256_shuffle_epi8(lookup, lo);
__m256i popcnt2 = _mm256_shuffle_epi8(lookup, hi);
}


If you are struggling to read the code above, you are not alone. I haven’t programmed in C++ for several years – it’s amazing how nice the names in civilised languages like Java and python (and even bash) are compared to the black magic above. There is some logic to the naming though: read page 5 of the manual. You can also read an accessible description of some of the functions used in this blog post.

The basic idea starts from storing the population counts for each possible byte value in a lookup table, which can be looked up using bit level parallelism and ultimately added up. For efficiency’s sake, instead of bytes, 4 bit nibbles are used, which is why you only see numbers 0-4 in the lookup table. Various, occasionally obscure, optimisations are applied resulting in the magic numbers at the the top of the function. A large chunk of the paper is devoted to their derivation: if you are interested, go and read the paper – I could not understand the intent of the code at all until reading the paper twice, especially section 2.

The points I find interesting are:

• This algorithm exists
• It uses instructions all modern commodity processors have
• It is fast
• It is in use

Could this be implemented in the JVM as an intrinsic and exposed on Vector?

# Explicit Intent and Even Faster Hash Codes

I wrote a post recently about how disappointed I was that the optimiser couldn’t outsmart some clever Java code for computing hash codes. Well, here’s a faster hash code along the same lines.

The hash code implemented in Arrays.hashCode is a polynomial hash, it applies to any data type with a positional interpretation. It takes the general form $\sum_{i=0}^{n}x_{i}31^{n - i}$ where $x_0 = 1$. In other words, it’s a dot product of the elements of the array and some powers of 31. Daniel Lemire’s implementation makes it explicit to the optimiser, in a way it won’t otherwise infer, that this operation is data parallel. If it’s really just a dot product it can be made even more obvious at the cost of a loss of flexibility.

Imagine you are processing fixed or limited length strings (VARCHAR(255) or an URL) or coordinates of a space of fixed dimension. Then you could pre-compute the coefficients in an array and write the hash code explicitly as a dot product. Java 9 uses AVX instructions for dot products, so it should be very fast.


public class FixedLengthHashCode {

private final int[] coefficients;

public FixedLengthHashCode(int maxLength) {
this.coefficients = new int[maxLength + 1];
coefficients[maxLength] = 1;
for (int i = maxLength - 1; i >= 0; --i) {
coefficients[i] = 31 * coefficients[i + 1];
}
}

public int hashCode(int[] value) {
int result = coefficients[0];
for (int i = 0; i < value.length && i < coefficients.length - 1; ++i) {
result += coefficients[i + 1] * value[i];
}
return result;
}
}


This is really explicit, unambiguously parallelisable, and the results are remarkable.

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
HashCode.BuiltIn thrpt 1 10 10.323026 0.223614 ops/us 100
HashCode.BuiltIn thrpt 1 10 0.959246 0.038900 ops/us 1000
HashCode.BuiltIn thrpt 1 10 0.096005 0.001836 ops/us 10000
HashCode.FixedLength thrpt 1 10 20.186800 0.297590 ops/us 100
HashCode.FixedLength thrpt 1 10 2.314187 0.082867 ops/us 1000
HashCode.FixedLength thrpt 1 10 0.227090 0.005377 ops/us 10000
HashCode.Unrolled thrpt 1 10 13.250821 0.752609 ops/us 100
HashCode.Unrolled thrpt 1 10 1.503368 0.058200 ops/us 1000
HashCode.Unrolled thrpt 1 10 0.152179 0.003541 ops/us 10000

Modifying the algorithm slightly to support limited variable length arrays degrades performance slightly, but there are seemingly equivalent implementations which do much worse.


public class FixedLengthHashCode {

private final int[] coefficients;

public FixedLengthHashCode(int maxLength) {
this.coefficients = new int[maxLength + 1];
coefficients[0] = 1;
for (int i = 1; i >= maxLength; ++i) {
coefficients[i] = 31 * coefficients[i - 1];
}
}

public int hashCode(int[] value) {
final int max = value.length;
int result = coefficients[max];
for (int i = 0; i < value.length && i < coefficients.length - 1; ++i) {
result += coefficients[max - i - 1] * value[i];
}
return result;
}
}


Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FixedLength thrpt 1 10 19.172574 0.742637 ops/us 100
FixedLength thrpt 1 10 2.233006 0.115285 ops/us 1000
FixedLength thrpt 1 10 0.227451 0.012231 ops/us 10000

The benchmark code is at github.