Matrix Multiplication Revisited

In a recent post, I took a look at matrix multiplication in pure Java, to see if it can go faster than reported in SIMD Intrinsics on Managed Language Runtimes. I found faster implementations than the paper’s benchmarks implied was possible. Nevertheless, I found that there were some limitations in Hotspot’s autovectoriser that I didn’t expect to see, even in JDK10. Were these limitations somehow fundamental, or can other compilers do better with essentially the same input?

I took a look at the code generated by GCC’s autovectoriser to see what’s possible in C/C++ without resorting to complicated intrinsics. For a bit of fun, I went over to the dark side to squeeze out some a lot of extra performance, which gave inspiration to a simple vectorised Java implementation which can maintain intensity as matrix size increases.

Background

The paper reports a 5x improvement in matrix multiplication throughput as a result of using LMS generated intrinsics. Using GCC as LMS’s backend, I easily reproduced very good throughput, but I found two Java implementations better than the paper’s baseline. The best performing Java implementation proposed in the paper was blocked. This post is not about the LMS benchmarks, but this code is this post’s inspiration.


public void blocked(float[] a, float[] b, float[] c, int n) {
  int BLOCK_SIZE = 8; 
  // GOOD: attempts to do as much work in submatrices
  // GOOD: tries to avoid bringing data through cache multiple times
  for (int kk = 0; kk < n; kk += BLOCK_SIZE) {
    for (int jj = 0; jj < n; jj += BLOCK_SIZE) {
      for (int i = 0; i < n; i++) {
        for (int j = jj; j < jj + BLOCK_SIZE; ++j) {
          // BAD: manual unrolling, bypasses optimisations
          float sum = c[i * n + j]; 
          for (int k = kk; k < kk + BLOCK_SIZE; ++k) {
            // BAD: second read (k * n) requires a gather - bad for cache, bad for dTLB
            // BAD: horizontal sums are inefficient
            sum += a[i * n + k] * b[k * n + j]; 
          }
          c[i * n + j] = sum;
         }
       }
    }
  }
}

I proposed the following implementation for improved cache efficiency and expected it to vectorise automatically.

public void fast(float[] a, float[] b, float[] c, int n) {
   // GOOD: 2x faster than "blocked" - why?
   int in = 0;
   for (int i = 0; i < n; ++i) {
       int kn = 0;
       for (int k = 0; k < n; ++k) {
           float aik = a[in + k];
           // MIXED: passes over c[in:in+n] multiple times per k-value, "free" if n is small
           // MIXED: reloads b[kn:kn+n] repeatedly for each i, bad if n is large, "free" if n is small
           // BAD: doesn't vectorise but should
           for (int j = 0; j < n; ++j) {
               c[in + j] += aik * b[kn + j]; // sequential writes and reads, cache and vectoriser friendly
           }
           kn += n;
       }
       in += n;
    }
}

My code actually doesn’t vectorise, even in JDK10, which really surprised me because the inner loop vectorises if the offsets are always zero. In any case, there is a simple hack involving the use of buffers, which unfortunately thrashes the cache, but narrows the field significantly.

  
  public void fastBuffered(float[] a, float[] b, float[] c, int n) {
    float[] bBuffer = new float[n];
    float[] cBuffer = new float[n];
    int in = 0;
    for (int i = 0; i < n; ++i) {
      int kn = 0;
      for (int k = 0; k < n; ++k) {
        float aik = a[in + k];
        System.arraycopy(b, kn, bBuffer, 0, n);
        saxpy(n, aik, bBuffer, cBuffer);
        kn += n;
      }
      System.arraycopy(cBuffer, 0, c, in, n);
      Arrays.fill(cBuffer, 0f);
      in += n;
    }
  }

I left the problem looking like this, with the “JDKX vectorised” lines using the algorithm above with a buffer hack:

GCC Autovectorisation

The Java code is very easy to translate into C/C++. Before looking at performance I want to get an idea of what GCC’s autovectoriser does. I want to see the code generated at GCC optimisation level 3, with unrolled loops, FMA, and AVX2, which can be seen as follows:

g++ -mavx2 -mfma -march=native -funroll-loops -O3 -S mmul.cpp

The generated assembly code can be seen in full context here. Let’s look at the mmul_saxpy routine first:

static void mmul_saxpy(const int n, const float* left, const float* right, float* result) {
    int in = 0;
    for (int i = 0; i < n; ++i) {
        int kn = 0;
        for (int k = 0; k < n; ++k) {
            float aik = left[in + k];
            for (int j = 0; j < n; ++j) {
                result[in + j] += aik * right[kn + j];
            }
            kn += n;
        }
        in += n;
    }
}

This routine uses SIMD instructions, which means in principle any other compiler could do this too. The inner loop has been unrolled, but this is only by virtue of the -funroll-loops flag. C2 does this sort of thing as standard, but only for hot loops. In general you might not want to unroll loops because of the impact on code size, and it’s great that a JIT compiler can decide only to do this when it’s profitable.

.L9:
  vmovups  (%rdx,%rax), %ymm4
  vfmadd213ps  (%rbx,%rax), %ymm3, %ymm4
  addl  $8, %r10d
  vmovaps  %ymm4, (%r11,%rax)
  vmovups  32(%rdx,%rax), %ymm5
  vfmadd213ps  32(%rbx,%rax), %ymm3, %ymm5
  vmovaps  %ymm5, 32(%r11,%rax)
  vmovups  64(%rdx,%rax), %ymm1
  vfmadd213ps  64(%rbx,%rax), %ymm3, %ymm1
  vmovaps  %ymm1, 64(%r11,%rax)
  vmovups  96(%rdx,%rax), %ymm2
  vfmadd213ps  96(%rbx,%rax), %ymm3, %ymm2
  vmovaps  %ymm2, 96(%r11,%rax)
  vmovups  128(%rdx,%rax), %ymm4
  vfmadd213ps  128(%rbx,%rax), %ymm3, %ymm4
  vmovaps  %ymm4, 128(%r11,%rax)
  vmovups  160(%rdx,%rax), %ymm5
  vfmadd213ps  160(%rbx,%rax), %ymm3, %ymm5
  vmovaps  %ymm5, 160(%r11,%rax)
  vmovups  192(%rdx,%rax), %ymm1
  vfmadd213ps  192(%rbx,%rax), %ymm3, %ymm1
  vmovaps  %ymm1, 192(%r11,%rax)
  vmovups  224(%rdx,%rax), %ymm2
  vfmadd213ps  224(%rbx,%rax), %ymm3, %ymm2
  vmovaps  %ymm2, 224(%r11,%rax)
  addq  $256, %rax
  cmpl  %r10d, 24(%rsp)
  ja  .L9

The mmul_blocked routine is compiled to quite convoluted assembly. It has a huge problem with the expression right[k * n + j], which requires a gather and is almost guaranteed to create 8 cache misses per block for large matrices. Moreover, this inefficiency gets much worse with problem size.

static void mmul_blocked(const int n, const float* left, const float* right, float* result) {
    int BLOCK_SIZE = 8;
    for (int kk = 0; kk < n; kk += BLOCK_SIZE) {
        for (int jj = 0; jj < n; jj += BLOCK_SIZE) {
            for (int i = 0; i < n; i++) {
                for (int j = jj; j < jj + BLOCK_SIZE; ++j) {
                    float sum = result[i * n + j];
                    for (int k = kk; k < kk + BLOCK_SIZE; ++k) {
                        sum += left[i * n + k] * right[k * n + j]; // second read here requires a gather
                    }
                    result[i * n + j] = sum;
                }
            }
        }
    }
}

This compiles to assembly with the unrolled vectorised loop below:

.L114:
  cmpq  %r10, %r9
  setbe  %cl
  cmpq  56(%rsp), %r8
  setnb  %dl
  orl  %ecx, %edx
  cmpq  %r14, %r9
  setbe  %cl
  cmpq  64(%rsp), %r8
  setnb  %r15b
  orl  %ecx, %r15d
  andl  %edx, %r15d
  cmpq  %r11, %r9
  setbe  %cl
  cmpq  48(%rsp), %r8
  setnb  %dl
  orl  %ecx, %edx
  andl  %r15d, %edx
  cmpq  %rbx, %r9
  setbe  %cl
  cmpq  40(%rsp), %r8
  setnb  %r15b
  orl  %ecx, %r15d
  andl  %edx, %r15d
  cmpq  %rsi, %r9
  setbe  %cl
  cmpq  32(%rsp), %r8
  setnb  %dl
  orl  %ecx, %edx
  andl  %r15d, %edx
  cmpq  %rdi, %r9
  setbe  %cl
  cmpq  24(%rsp), %r8
  setnb  %r15b
  orl  %ecx, %r15d
  andl  %edx, %r15d
  cmpq  %rbp, %r9
  setbe  %cl
  cmpq  16(%rsp), %r8
  setnb  %dl
  orl  %ecx, %edx
  andl  %r15d, %edx
  cmpq  %r12, %r9
  setbe  %cl
  cmpq  8(%rsp), %r8
  setnb  %r15b
  orl  %r15d, %ecx
  testb  %cl, %dl
  je  .L111
  leaq  32(%rax), %rdx
  cmpq  %rdx, %r8
  setnb  %cl
  cmpq  %rax, %r9
  setbe  %r15b
  orb  %r15b, %cl
  je  .L111
  vmovups  (%r8), %ymm2
  vbroadcastss  (%rax), %ymm0
  vfmadd132ps  (%r14), %ymm2, %ymm0
  vbroadcastss  4(%rax), %ymm1
  vfmadd231ps  (%r10), %ymm1, %ymm0
  vbroadcastss  8(%rax), %ymm3
  vfmadd231ps  (%r11), %ymm3, %ymm0
  vbroadcastss  12(%rax), %ymm4
  vfmadd231ps  (%rbx), %ymm4, %ymm0
  vbroadcastss  16(%rax), %ymm5
  vfmadd231ps  (%rsi), %ymm5, %ymm0
  vbroadcastss  20(%rax), %ymm2
  vfmadd231ps  (%rdi), %ymm2, %ymm0
  vbroadcastss  24(%rax), %ymm1
  vfmadd231ps  0(%rbp), %ymm1, %ymm0
  vbroadcastss  28(%rax), %ymm3
  vfmadd231ps  (%r12), %ymm3, %ymm0
  vmovups  %ymm0, (%r8)

Benchmarks

I implemented a suite of benchmarks to compare the implementations. You can run them, but since they measure throughput and intensity averaged over hundreds of iterations per matrix size, the full run will take several hours.

g++ -mavx2 -mfma -march=native -funroll-loops -O3 mmul.cpp -o mmul.exe && ./mmul.exe > results.csv

The saxpy routine wins, with blocked fading fast after a middling start.

name size throughput (ops/s) flops/cycle
blocked 64 22770.2 4.5916
saxpy 64 25638.4 5.16997
blocked 128 2736.9 4.41515
saxpy 128 4108.52 6.62783
blocked 192 788.132 4.29101
saxpy 192 1262.45 6.87346
blocked 256 291.728 3.76492
saxpy 256 521.515 6.73044
blocked 320 147.979 3.72997
saxpy 320 244.528 6.16362
blocked 384 76.986 3.35322
saxpy 384 150.441 6.55264
blocked 448 50.4686 3.4907
saxpy 448 95.0752 6.57594
blocked 512 30.0085 3.09821
saxpy 512 65.1842 6.72991
blocked 576 22.8301 3.35608
saxpy 576 44.871 6.59614
blocked 640 15.5007 3.12571
saxpy 640 32.3709 6.52757
blocked 704 12.2478 3.28726
saxpy 704 25.3047 6.79166
blocked 768 8.69277 3.02899
saxpy 768 19.8011 6.8997
blocked 832 7.29356 3.23122
saxpy 832 15.3437 6.7976
blocked 896 4.95207 2.74011
saxpy 896 11.9611 6.61836
blocked 960 3.4467 2.34571
saxpy 960 9.25535 6.29888
blocked 1024 2.02289 1.67082
saxpy 1024 6.87039 5.67463

With GCC autovectorisation, saxpy performs well, maintaining intensity as size increases, albeit well below the theoretical capacity. It would be nice if similar code could be JIT compiled in Java.

Intel Intrinsics

To understand the problem space a bit better, I find out how fast matrix multiplication can get without domain expertise by handcrafting an algorithm with intrinsics. My laptop’s Skylake chip (turbo boost and hyperthreading disabled) is capable of 32 SP flops per cycle per core – Java and the LMS implementation previously fell a long way short of that. It was difficult getting beyond 4f/c with Java, and LMS peaked at almost 6f/c before quickly tailing off. GCC autovectorisation achieved and maintained 7f/c.

To start, I’ll take full advantage of the facility to align the matrices on 64 byte intervals, since I have 64B cache lines, though this might just be voodoo. I take the saxpy routine and replace its kernel with intrinsics. Because of the -funroll-loops option, this will get unrolled without effort.

static void mmul_saxpy_avx(const int n, const float* left, const float* right, float* result) {
    int in = 0;
    for (int i = 0; i < n; ++i) {
        int kn = 0;
        for (int k = 0; k < n; ++k) {
            __m256 aik = _mm256_set1_ps(left[in + k]);
            int j = 0;
            for (; j < n; j += 8) {
                _mm256_store_ps(result + in + j, _mm256_fmadd_ps(aik, _mm256_load_ps(right + kn + j), _mm256_load_ps(result + in + j)));
            }
            for (; j < n; ++j) {
                result[in + j] += left[in + k] * right[kn + j];
            }
            kn += n;
        }
        in += n;
    }
}

This code is actually not a lot faster, if at all, than the basic saxpy above: a lot of aggressive optimisations have already been applied.

Combining Blocked and SAXPY

What makes blocked so poor is the gather and the cache miss, not the concept of blocking itself. A limiting factor for saxpy performance is that the ratio of loads to floating point operations is too high. With this in mind, I tried combining the blocking idea with saxpy, by implementing saxpy multiplications for smaller sub-matrices. This results in a different algorithm with fewer loads per floating point operation, and the inner two loops are swapped. It avoids the gather and the cache miss in blocked. Because the matrices are in row major format, I make the width of the blocks much larger than the height. Also, different heights and widths make sense depending on the size of the matrix, so I choose them dynamically. The design constraints are to avoid gathers and horizontal reduction.

static void mmul_tiled_avx(const int n, const float *left, const float *right, float *result) {
    const int block_width = n >= 256 ? 512 : 256;
    const int block_height = n >= 512 ? 8 : n >= 256 ? 16 : 32;
    for (int row_offset = 0; row_offset < n; row_offset += block_height) {
        for (int column_offset = 0; column_offset < n; column_offset += block_width) {
            for (int i = 0; i < n; ++i) {
                for (int j = column_offset; j < column_offset + block_width && j < n; j += 8) {
                    __m256 sum = _mm256_load_ps(result + i * n + j);
                    for (int k = row_offset; k < row_offset + block_height && k < n; ++k) {
                        sum = _mm256_fmadd_ps(_mm256_set1_ps(left[i * n + k]), _mm256_load_ps(right + k * n + j), sum);
                    }
                    _mm256_store_ps(result + i * n + j, sum);
                }
            }
        }
    }
}

You will see in the benchmark results that this routine really doesn’t do very well compared to saxpy. Finally, I unroll it, which is profitable despite setting -funroll-loops because there is slightly more to this than an unroll. This is a sequence of vertical reductions which have no data dependencies.

static void mmul_tiled_avx_unrolled(const int n, const float *left, const float *right, float *result) {
    const int block_width = n >= 256 ? 512 : 256;
    const int block_height = n >= 512 ? 8 : n >= 256 ? 16 : 32;
    for (int column_offset = 0; column_offset < n; column_offset += block_width) {
        for (int row_offset = 0; row_offset < n; row_offset += block_height) {
            for (int i = 0; i < n; ++i) {
                for (int j = column_offset; j < column_offset + block_width && j < n; j += 64) {
                    __m256 sum1 = _mm256_load_ps(result + i * n + j);
                    __m256 sum2 = _mm256_load_ps(result + i * n + j + 8);
                    __m256 sum3 = _mm256_load_ps(result + i * n + j + 16);
                    __m256 sum4 = _mm256_load_ps(result + i * n + j + 24);
                    __m256 sum5 = _mm256_load_ps(result + i * n + j + 32);
                    __m256 sum6 = _mm256_load_ps(result + i * n + j + 40);
                    __m256 sum7 = _mm256_load_ps(result + i * n + j + 48);
                    __m256 sum8 = _mm256_load_ps(result + i * n + j + 56);
                    for (int k = row_offset; k < row_offset + block_height && k < n; ++k) {
                        __m256 multiplier = _mm256_set1_ps(left[i * n + k]);
                        sum1 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j), sum1);
                        sum2 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 8), sum2);
                        sum3 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 16), sum3);
                        sum4 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 24), sum4);
                        sum5 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 32), sum5);
                        sum6 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 40), sum6);
                        sum7 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 48), sum7);
                        sum8 = _mm256_fmadd_ps(multiplier, _mm256_load_ps(right + k * n + j + 56), sum8);
                    }
                    _mm256_store_ps(result + i * n + j, sum1);
                    _mm256_store_ps(result + i * n + j + 8, sum2);
                    _mm256_store_ps(result + i * n + j + 16, sum3);
                    _mm256_store_ps(result + i * n + j + 24, sum4);
                    _mm256_store_ps(result + i * n + j + 32, sum5);
                    _mm256_store_ps(result + i * n + j + 40, sum6);
                    _mm256_store_ps(result + i * n + j + 48, sum7);
                    _mm256_store_ps(result + i * n + j + 56, sum8);
                }
            }
        }
    }
}

This final implementation is fast, and is probably as good as I am going to manage, without reading papers. This should be a CPU bound problem because the algorithm is O(n^3) whereas the problem size is O(n^2). But the flops/cycle decreases with problem size in all of these implementations. It’s possible that this could be amelioarated by a better dynamic tiling policy. I’m unlikely to be able to fix that.

It does make a huge difference being able to go very low level – handwritten intrinsics with GCC unlock awesome throughput – but it’s quite hard to actually get to the point where you can beat a good optimising compiler. Mind you, there are harder problems to solve this, and you may well be a domain expert.

The benchmark results summarise this best:

name size throughput (ops/s) flops/cycle
saxpy_avx 64 49225.7 9.92632
tiled_avx 64 33680.5 6.79165
tiled_avx_unrolled 64 127936 25.7981
saxpy_avx 128 5871.02 9.47109
tiled_avx 128 4210.07 6.79166
tiled_avx_unrolled 128 15997.6 25.8072
saxpy_avx 192 1603.84 8.73214
tiled_avx 192 1203.33 6.55159
tiled_avx_unrolled 192 4383.09 23.8638
saxpy_avx 256 633.595 8.17689
tiled_avx 256 626.157 8.0809
tiled_avx_unrolled 256 1792.52 23.1335
saxpy_avx 320 284.161 7.1626
tiled_avx 320 323.197 8.14656
tiled_avx_unrolled 320 935.571 23.5822
saxpy_avx 384 161.517 7.03508
tiled_avx 384 188.215 8.19794
tiled_avx_unrolled 384 543.235 23.6613
saxpy_avx 448 99.1987 6.86115
tiled_avx 448 118.588 8.2022
tiled_avx_unrolled 448 314 21.718
saxpy_avx 512 70.0296 7.23017
tiled_avx 512 73.2019 7.55769
tiled_avx_unrolled 512 197.815 20.4233
saxpy_avx 576 46.1944 6.79068
tiled_avx 576 50.6315 7.44294
tiled_avx_unrolled 576 126.045 18.5289
saxpy_avx 640 33.8209 6.81996
tiled_avx 640 37.0288 7.46682
tiled_avx_unrolled 640 92.784 18.7098
saxpy_avx 704 24.9096 6.68561
tiled_avx 704 27.7543 7.44912
tiled_avx_unrolled 704 69.0399 18.53
saxpy_avx 768 19.5158 6.80027
tiled_avx 768 21.532 7.50282
tiled_avx_unrolled 768 54.1763 18.8777
saxpy_avx 832 12.8635 5.69882
tiled_avx 832 14.6666 6.49766
tiled_avx_unrolled 832 37.9592 16.8168
saxpy_avx 896 12.0526 6.66899
tiled_avx 896 13.3799 7.40346
tiled_avx_unrolled 896 34.0838 18.8595
saxpy_avx 960 8.97193 6.10599
tiled_avx 960 10.1052 6.87725
tiled_avx_unrolled 960 21.0263 14.3098
saxpy_avx 1024 6.73081 5.55935
tiled_avx 1024 7.21214 5.9569
tiled_avx_unrolled 1024 12.7768 10.5531

Can we do better in Java?

Writing genuinely fast code gives an indication of how little of the processor Java actually utilises, but is it possible to bring this knowledge over to Java? The saxpy based implementations in my previous post performed well for small to medium sized matrices. Once the matrices grow, however, they become too big to be allowed to pass through cache multiple times: we need hot, small cached data to be replenished from the larger matrix. Ideally we wouldn’t need to make any copies, but it seems that the autovectoriser doesn’t like offsets: System.arraycopy is a reasonably fast compromise. The basic sequential read pattern is validated: even native code requiring a gather does not perform well for this problem. The best effort C++ code translates almost verbatim into this Java code, which is quite fast for large matrices.


public void tiled(float[] a, float[] b, float[] c, int n) {
    final int bufferSize = 512;
    final int width = Math.min(n, bufferSize);
    final int height = Math.min(n, n >= 512 ? 8 : n >= 256 ? 16 : 32);
    float[] sum = new float[bufferSize];
    float[] vector = new float[bufferSize];
    for (int rowOffset = 0; rowOffset < n; rowOffset += height) {
      for (int columnOffset = 0; columnOffset < n; columnOffset += width) {
        for (int i = 0; i < n; ++i) {
          for (int j = columnOffset; j < columnOffset + width && j < n; j += width) {
            int stride = Math.min(n - columnOffset, bufferSize);
            // copy to give autovectorisation a hint
            System.arraycopy(c, i * n + j, sum, 0, stride);
            for (int k = rowOffset; k < rowOffset + height && k < n; ++k) {
              float multiplier = a[i * n + k];
              System.arraycopy(b, k * n  + j, vector, 0, stride);
              for (int l = 0; l < stride; ++l) {
                sum[l] = Math.fma(multiplier, vector[l], sum[l]);
              }
            }
            System.arraycopy(sum, 0, c, i * n + j, stride);
          }
        }
      }
    }
  }

Benchmarking it using the same harness used in the previous post, the performance is ~10% higher for large arrays than my previous best effort. Still, the reality is that this is too slow to be useful. If you need to do linear algebra, use C/C++ for the time being!

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size flops/cycle
fastBuffered thrpt 1 10 53.331195 0.270526 ops/s 448 3.688688696
fastBuffered thrpt 1 10 34.365765 0.16641 ops/s 512 3.548072999
fastBuffered thrpt 1 10 26.128264 0.239719 ops/s 576 3.840914622
fastBuffered thrpt 1 10 19.044509 0.139197 ops/s 640 3.84031059
fastBuffered thrpt 1 10 14.312154 1.045093 ops/s 704 3.841312378
fastBuffered thrpt 1 10 7.772745 0.074598 ops/s 768 2.708411991
fastBuffered thrpt 1 10 6.276182 0.067338 ops/s 832 2.780495238
fastBuffered thrpt 1 10 4.8784 0.067368 ops/s 896 2.699343067
fastBuffered thrpt 1 10 4.068907 0.038677 ops/s 960 2.769160387
fastBuffered thrpt 1 10 2.568101 0.108612 ops/s 1024 2.121136502
tiled thrpt 1 10 56.495366 0.584872 ops/s 448 3.907540754
tiled thrpt 1 10 30.884954 3.221017 ops/s 512 3.188698735
tiled thrpt 1 10 15.580581 0.412654 ops/s 576 2.290381075
tiled thrpt 1 10 9.178969 0.841178 ops/s 640 1.850932038
tiled thrpt 1 10 12.229763 0.350233 ops/s 704 3.282408783
tiled thrpt 1 10 9.371032 0.330742 ops/s 768 3.265334889
tiled thrpt 1 10 7.727068 0.277969 ops/s 832 3.423271628
tiled thrpt 1 10 6.076451 0.30305 ops/s 896 3.362255222
tiled thrpt 1 10 4.916811 0.2823 ops/s 960 3.346215151
tiled thrpt 1 10 3.722623 0.26486 ops/s 1024 3.074720008

Posted on

Incidental Similarity

I recently saw an interesting class, BitVector, in Apache Arrow, which represents a column of bits, providing minimal or zero copy distribution. The implementation is similar to a bitset but backed by a byte[] rather than a long[]. Given the coincidental similarity in implementation, it’s tempting to look at this, extend its interface and try to use it as a general purpose, distributed bitset. Could this work? Why not just implement some extra methods? Fork it on Github!

This post details the caveats of trying to adapt an abstraction beyond its intended purpose; in a scenario where generic bitset capabilities are added to BitVector without due consideration, examined through the lens of performance. This runs into the observable effect of word widening on throughput, given the constraints imposed by JLS 15.22. In the end, the only remedy is to use a long[], sacrificing the original zero copy design goal. I hope this is a fairly self-contained example of how uncontrolled adaptation can be hostile to the original design goals: having the source code isn’t enough reason to modify it.

Checking bits

How fast is it to check if the bit at index i is set or not? BitVector implements this functionality, and was designed for it. This can be measured by JMH by generating a random long[] and creating a byte[] 8x longer with identical bits. The throughput of checking the value of the bit at random indices can be measured. It turns out that if all you want to do is access bits, byte[] isn’t such a bad choice, and if those bytes are coming directly from the network, it could even be a great choice. I ran the benchmark below and saw that the two operations are similar (within measurement error).


@OutputTimeUnit(TimeUnit.MICROSECONDS)
@State(Scope.Thread)
public class BitSet {

    @Param({"1024", "2048", "4096", "8192"})
    int size;

    private long[] leftLongs;
    private long[] rightLongs;
    private long[] differenceLongs;
    private byte[] leftBytes;
    private byte[] rightBytes;
    private byte[] differenceBytes;

    @Setup(Level.Trial)
    public void init() {
        this.leftLongs = createLongArray(size);
        this.rightLongs = createLongArray(size);
        this.differenceLongs = new long[size];
        this.leftBytes = makeBytesFromLongs(leftLongs);
        this.rightBytes = makeBytesFromLongs(rightLongs);
        this.differenceBytes = new byte[size * 8];
    }

    @Benchmark
    public boolean CheckBit_LongArray() {
        int index = index();
        return (leftLongs[index >>> 6] & (1L << index)) != 0;
    }

    @Benchmark
    public boolean CheckBit_ByteArray() {
        int index = index();
        return ((leftBytes[index >>> 3] & 0xFF) & (1 << (index & 7))) != 0;
    }

    private int index() {
        return ThreadLocalRandom.current().nextInt(size * 64);
    }

    private static byte[] makeBytesFromLongs(long[] array) {
        byte[] bytes = new byte[8 * array.length];
        for (int i = 0; i < array.length; ++i) {
            long word = array[i];
            bytes[8 * i + 7] = (byte) word;
            bytes[8 * i + 6] = (byte) (word >>> 8);
            bytes[8 * i + 5] = (byte) (word >>> 16);
            bytes[8 * i + 4] = (byte) (word >>> 24);
            bytes[8 * i + 3] = (byte) (word >>> 32);
            bytes[8 * i + 2] = (byte) (word >>> 40);
            bytes[8 * i + 1] = (byte) (word >>> 48);
            bytes[8 * i]     = (byte) (word >>> 56);
        }
        return bytes;
    }
}

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
CheckBit_ByteArray thrpt 1 10 174.421170 1.583275 ops/us 1024
CheckBit_ByteArray thrpt 1 10 173.938408 1.445796 ops/us 2048
CheckBit_ByteArray thrpt 1 10 172.522190 0.815596 ops/us 4096
CheckBit_ByteArray thrpt 1 10 167.550530 1.677091 ops/us 8192
CheckBit_LongArray thrpt 1 10 171.639695 0.934494 ops/us 1024
CheckBit_LongArray thrpt 1 10 169.703960 2.427244 ops/us 2048
CheckBit_LongArray thrpt 1 10 169.333360 1.649654 ops/us 4096
CheckBit_LongArray thrpt 1 10 166.518375 0.815433 ops/us 8192

To support this functionality, there’s no reason to choose either way, and it must be very appealing to use bytes as they are delivered from the network, avoiding copying costs. Given that for a database column, this is the only operation needed, and Apache Arrow has a stated aim to copy data as little as possible, this seems like quite a good decision.

Logical Conjugations

But what happens if you try to add a logical operation to BitVector, such as an XOR? We need to handle the fact that bytes are signed and their sign bit must be preserved in promotion, according to the JLS. This would break the bitset, so extra operations are required to keep the 8th bit in its right place. With the widening and its associated workarounds, suddenly the byte[] is a much poorer choice than a long[], and it shows in benchmarks.


    @Benchmark
    public void Difference_ByteArray(Blackhole bh) {
        for (int i = 0; i < leftBytes.length && i < rightBytes.length; ++i) {
            differenceBytes[i] = (byte)((leftBytes[i] & 0xFF) ^ (rightBytes[i] & 0xFF));
        }
        bh.consume(differenceBytes);
    }

    @Benchmark
    public void Difference_LongArray(Blackhole bh) {
        for (int i = 0; i < leftLongs.length && i < rightLongs.length; ++i) {
            differenceLongs[i] = leftLongs[i] ^ rightLongs[i];
        }
        bh.consume(differenceLongs);
    }

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
Difference_ByteArray thrpt 1 10 0.805872 0.038644 ops/us 1024
Difference_ByteArray thrpt 1 10 0.391705 0.017453 ops/us 2048
Difference_ByteArray thrpt 1 10 0.190102 0.008580 ops/us 4096
Difference_ByteArray thrpt 1 10 0.169104 0.015086 ops/us 8192
Difference_LongArray thrpt 1 10 2.450659 0.094590 ops/us 1024
Difference_LongArray thrpt 1 10 1.047330 0.016898 ops/us 2048
Difference_LongArray thrpt 1 10 0.546286 0.014211 ops/us 4096
Difference_LongArray thrpt 1 10 0.277378 0.015663 ops/us 8192

This is a fairly crazy slow down. Why? You need to look at the assembly generated in each case. For long[] it’s demonstrable that logical operations do vectorise. The JLS, specifically section 15.22, doesn’t really give the byte[] implementation a chance. It states that for logical operations, sub dword primitive types must be promoted or widened before the operation. This means that if one were to try to implement this operation with, say AVX2, using 256 bit ymmwords each consisting of 16 bytes, then each ymmword would have to be inflated by a factor of four: it gets complicated quickly, given this constraint. Despite that complexity, I was surprised to see that C2 does use 128 bit xmmwords, but it’s not as fast as using the full 256 bit registers available. This can be seen by printing out the emitted assembly like normal.

movsxd  r10,ebx     

vmovq   xmm2,mmword ptr [rsi+r10+10h]

vpxor   xmm2,xmm2,xmmword ptr [r8+r10+10h]

vmovq   mmword ptr [rax+r10+10h],xmm2

Posted on

Vectorised Logical Operations in Java 9

This is a short post for my own reference, since I feel I have already done the topic of does Java 9 use AVX for this? to death. Cutting to the chase, Java 9 autovectorises loops to compute logical ANDs, XORs, ORs and ANDNOTs between arrays, making use of the instructions VPXOR, VPOR and VPAND. You can replicate this by running the code at github.

XOR


    @Benchmark
    public long[] xor(LongData state) {
        long[] result = new long[state.data1.length];
        long[] data1 = state.data1;
        long[] data2 = state.data2;
        for (int i = 0; i < data1.length && i < data2.length; ++i) {
            result[i] = data1[i] ^ data2[i];
        }
        return result;
    }

vmovdqu ymm0,ymmword ptr [r10+r13*8+10h]

vpxor   ymm0,ymm0,ymmword ptr [rbx+r13*8+10h]

vmovdqu ymmword ptr [rax+r13*8+10h],ymm0

OR


    @Benchmark
    public long[] or(LongData state) {
        long[] result = new long[state.data1.length];
        long[] data1 = state.data1;
        long[] data2 = state.data2;
        for (int i = 0; i < data1.length && i < data2.length; ++i) {
            result[i] = data1[i] | data2[i];
        }
        return result;
    }

vmovdqu ymm0,ymmword ptr [r10+rsi*8+30h]
 
vpor    ymm0,ymm0,ymmword ptr [rbx+rsi*8+30h]

vmovdqu ymmword ptr [rax+rsi*8+30h],ymm0

AND


    @Benchmark
    public long[] and(LongData state) {
        long[] result = new long[state.data1.length];
        long[] data1 = state.data1;
        long[] data2 = state.data2;
        for (int i = 0; i < data1.length && i < data2.length; ++i) {
            result[i] = data1[i] & data2[i];
        }
        return result;
    }

vmovdqu ymm0,ymmword ptr [r10+r13*8+10h]

vpand   ymm0,ymm0,ymmword ptr [rbx+r13*8+10h]

vmovdqu ymmword ptr [rax+r13*8+10h],ymm0

ANDNOT


    @Benchmark
    public long[] andNot(LongData state) {
        long[] result = new long[state.data1.length];
        long[] data1 = state.data1;
        long[] data2 = state.data2;
        for (int i = 0; i < data1.length && i < data2.length; ++i) {
            result[i] = data1[i] & ~data2[i];
        }
        return result;
    }

vpunpcklqdq xmm0,xmm0,xmm0

vinserti128 ymm0,ymm0,xmm0,1h

vmovdqu ymm1,ymmword ptr [rbx+r13*8+10h]

vpxor   ymm1,ymm1,ymm0

vpand   ymm1,ymm1,ymmword ptr [r10+r13*8+10h]

vmovdqu ymmword ptr [rax+r13*8+10h],ymm1

Posted on

How much Algebra does C2 Know? Part 2: Distributivity

In part one of this series of posts, I looked at how important associativity and independence are for fast loops. C2 seems to utilise these properties to generate unrolled and pipelined machine code for loops, achieving higher throughput even in cases where the kernel of the loop is 3x slower according to vendor advertised instruction throughputs. C2 has a weird and wonderful relationship with distributivity, and hints from the programmer can both and help hinder the generation of good quality machine code.

Viability and Correctness

Distributivity is the simple notion of factoring out brackets. Is this, in general, a viable loop rewrite strategy? This can be utilised to transform the method Scale into FactoredScale, both of which perform floating point arithmetic:


    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public double Scale(DoubleData state) {
        double value = 0D;
        double[] data = state.data1;
        for (int i = 0; i < data.length; ++i) {
            value += 3.14159 * data[i];
        }
        return value;
    }

    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public double FactoredScale(DoubleData state) {
        double value = 0D;
        double[] data = state.data1;
        for (int i = 0; i < data.length; ++i) {
            value += data[i];
        }
        return 3.14159 * value;
    }

Running the project at github with the argument --include .*scale.*, there may be a performance gain to be had from this rewrite, but it isn’t clear cut:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale thrpt 1 10 7.011606 0.274742 ops/ms 100000
FactoredScale thrpt 1 10 0.621515 0.026853 ops/ms 1000000
Scale thrpt 1 10 6.962434 0.240180 ops/ms 100000
Scale thrpt 1 10 0.671042 0.011686 ops/ms 1000000

With the real numbers it would be completely valid, but floating point arithmetic is not associative. Joseph Darcy explains why in this deep dive on floating point semantics. Broken associativity of addition entails broken distributivity of any operation over it, so the two loops are not equivalent, and they give different outputs (e.g. 15662.513298516365 vs 15662.51329851632 for one sample input). The rewrite isn’t correct even for floating point data, so it isn’t an optimisation that could be applied in good faith, except in a very small number of cases. You have to rewrite the loop yourself and figure out if the small but inevitable differences are acceptable.

Counterintuitive Performance

Integer multiplication is distributive over addition, and we can check if C2 does this rewrite by running the same code with 32 bit integer values, for now fixing a scale factor of 10 (which seems like an innocuous value, no?)


    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public int Scale_Int(IntData state) {
        int value = 0;
        int[] data = state.data1;
        for (int i = 0; i < data.length; ++i) {
            value += 10 * data[i];
        }
        return value;
    }

    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public int FactoredScale_Int(IntData state) {
        int value = 0;
        int[] data = state.data1;
        for (int i = 0; i < data.length; ++i) {
            value += data[i];
        }
        return 10 * value;
    }

The results are fascinating:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale_Int thrpt 1 10 28.339699 0.608075 ops/ms 100000
FactoredScale_Int thrpt 1 10 2.392579 0.506413 ops/ms 1000000
Scale_Int thrpt 1 10 33.335721 0.295334 ops/ms 100000
Scale_Int thrpt 1 10 2.838242 0.448213 ops/ms 1000000

The code is doing thousands more multiplications in less time when the multiplication is not factored out of the loop. So what the devil is going on? Inspecting the assembly for the faster loop is revealing

com/openkappa/simd/scale/Scale.Scale_Int(Lcom/openkappa/simd/state/IntData;)I  [0x000001c89e499320, 0x000001c89e4996f8]  984 bytes
Argument 0 is unknown.RIP: 0x1c89e499320 Code size: 0x000003d8
[Entry Point]
[Verified Entry Point]
[Constants]
  # {method} {0x000001c8b3701b10} 'Scale_Int' '(Lcom/openkappa/simd/state/IntData;)I' in 'com/openkappa/simd/scale/Scale'
  0x000001c89e499320: int3                      ;...cc

  0x000001c89e499321: nop     word ptr [rax+rax+0h]  ;...66
                                                ;...66
                                                ;...66
                                                ;...0f
                                                ;...1f
                                                ;...84
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e49932c: nop                       ;...66
                                                ;...66
                                                ;...66
                                                ;...90

  0x000001c89e499330: mov     dword ptr [rsp+0ffffffffffff9000h],eax
                                                ;...89
                                                ;...84
                                                ;...24
                                                ;...00
                                                ;...90
                                                ;...ff
                                                ;...ff

  0x000001c89e499337: push    rbp               ;...55

  0x000001c89e499338: sub     rsp,40h           ;...48
                                                ;...83
                                                ;...ec
                                                ;...40

  0x000001c89e49933c: mov     rbp,qword ptr [rdx+8h]  ;...48
                                                ;...8b
                                                ;...6a
                                                ;...08

  0x000001c89e499340: mov     ebx,dword ptr [rdx+10h]  ;...8b
                                                ;...5a
                                                ;...10

  0x000001c89e499343: mov     r13d,dword ptr [rdx]  ;...44
                                                ;...8b
                                                ;...2a

  0x000001c89e499346: mov     rcx,rdx           ;...48
                                                ;...8b
                                                ;...ca

  0x000001c89e499349: vzeroupper                ;...c5
                                                ;...f8
                                                ;...77

  0x000001c89e49934c: mov     r10,51da8d20h     ;...49
                                                ;...ba
                                                ;...20
                                                ;...8d
                                                ;...da
                                                ;...51
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e499356: call indirect r10         ;...41
                                                ;...ff
                                                ;...d2

  0x000001c89e499359: mov     r11d,dword ptr [rbp+8h]  ;...44
                                                ;...8b
                                                ;...5d
                                                ;...08
                                                ; implicit exception: dispatches to 0x000001c89e4996c1
  0x000001c89e49935d: cmp     r11d,0f800016dh   ;...41
                                                ;...81
                                                ;...fb
                                                ;...6d
                                                ;...01
                                                ;...00
                                                ;...f8
                                                ;   {metadata({type array int})}
  0x000001c89e499364: jne     1c89e4996a9h      ;...0f
                                                ;...85
                                                ;...3f
                                                ;...03
                                                ;...00
                                                ;...00
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@10 (line 41)

  0x000001c89e49936a: mov     edi,dword ptr [rbp+0ch]  ;...8b
                                                ;...7d
                                                ;...0c
                                                ;*arraylength {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@13 (line 41)

  0x000001c89e49936d: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e499370: jnl     1c89e49967ah      ;...0f
                                                ;...8d
                                                ;...04
                                                ;...03
                                                ;...00
                                                ;...00
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@14 (line 41)

  0x000001c89e499376: mov     r10d,ebp          ;...44
                                                ;...8b
                                                ;...d5

  0x000001c89e499379: mov     r8d,r13d          ;...45
                                                ;...8b
                                                ;...c5

  0x000001c89e49937c: inc     r8d               ;...41
                                                ;...ff
                                                ;...c0

  0x000001c89e49937f: shr     r10d,2h           ;...41
                                                ;...c1
                                                ;...ea
                                                ;...02

  0x000001c89e499383: and     r10d,7h           ;...41
                                                ;...83
                                                ;...e2
                                                ;...07

  0x000001c89e499387: xor     r9d,r9d           ;...45
                                                ;...33
                                                ;...c9

  0x000001c89e49938a: cmp     r8d,r9d           ;...45
                                                ;...3b
                                                ;...c1

  0x000001c89e49938d: cmovl   r8d,r9d           ;...45
                                                ;...0f
                                                ;...4c
                                                ;...c1

  0x000001c89e499391: cmp     r8d,edi           ;...44
                                                ;...3b
                                                ;...c7

  0x000001c89e499394: cmovnle r8d,edi           ;...44
                                                ;...0f
                                                ;...4f
                                                ;...c7

  0x000001c89e499398: add     r10d,r8d          ;...45
                                                ;...03
                                                ;...d0

  0x000001c89e49939b: mov     r11d,4h           ;...41
                                                ;...bb
                                                ;...04
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e4993a1: sub     r11d,r10d         ;...45
                                                ;...2b
                                                ;...da

  0x000001c89e4993a4: and     r11d,7h           ;...41
                                                ;...83
                                                ;...e3
                                                ;...07

  0x000001c89e4993a8: add     r11d,r8d          ;...45
                                                ;...03
                                                ;...d8

  0x000001c89e4993ab: cmp     r11d,edi          ;...44
                                                ;...3b
                                                ;...df

  0x000001c89e4993ae: cmovnle r11d,edi          ;...44
                                                ;...0f
                                                ;...4f
                                                ;...df

  0x000001c89e4993b2: nop                       ;...66
                                                ;...90

  0x000001c89e4993b4: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e4993b7: jnb     1c89e49968bh      ;...0f
                                                ;...83
                                                ;...ce
                                                ;...02
                                                ;...00
                                                ;...00

  0x000001c89e4993bd: mov     r10d,dword ptr [rbp+r13*4+10h]
                                                ;...46
                                                ;...8b
                                                ;...54
                                                ;...ad
                                                ;...10
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@23 (line 42)

  0x000001c89e4993c2: mov     r9d,r10d          ;...45
                                                ;...8b
                                                ;...ca

  0x000001c89e4993c5: shl     r9d,3h            ;...41
                                                ;...c1
                                                ;...e1
                                                ;...03

  0x000001c89e4993c9: shl     r10d,1h           ;...41
                                                ;...d1
                                                ;...e2

  0x000001c89e4993cc: add     r9d,r10d          ;...45
                                                ;...03
                                                ;...ca

  0x000001c89e4993cf: add     ebx,r9d           ;...41
                                                ;...03
                                                ;...d9
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@25 (line 42)

  0x000001c89e4993d2: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@27 (line 41)

  0x000001c89e4993d5: cmp     r13d,r11d         ;...45
                                                ;...3b
                                                ;...eb

  0x000001c89e4993d8: jl      1c89e4993b4h      ;...7c
                                                ;...da
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@14 (line 41)

  0x000001c89e4993da: mov     r8d,edi           ;...44
                                                ;...8b
                                                ;...c7

  0x000001c89e4993dd: add     r8d,0ffffffc1h    ;...41
                                                ;...83
                                                ;...c0
                                                ;...c1

  0x000001c89e4993e1: mov     ecx,80000000h     ;...b9
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...80

  0x000001c89e4993e6: cmp     edi,r8d           ;...41
                                                ;...3b
                                                ;...f8

  0x000001c89e4993e9: cmovl   r8d,ecx           ;...44
                                                ;...0f
                                                ;...4c
                                                ;...c1

  0x000001c89e4993ed: cmp     r13d,r8d          ;...45
                                                ;...3b
                                                ;...e8

  0x000001c89e4993f0: jnl     1c89e499651h      ;...0f
                                                ;...8d
                                                ;...5b
                                                ;...02
                                                ;...00
                                                ;...00

  0x000001c89e4993f6: nop     word ptr [rax+rax+0h]  ;...66
                                                ;...66
                                                ;...0f
                                                ;...1f
                                                ;...84
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e499400: vmovdqu ymm8,ymmword ptr [rbp+r13*4+10h]
                                                ;...c4
                                                ;...21
                                                ;...7e
                                                ;...6f
                                                ;...44
                                                ;...ad
                                                ;...10

  0x000001c89e499407: movsxd  r10,r13d          ;...4d
                                                ;...63
                                                ;...d5

  0x000001c89e49940a: vmovdqu ymm9,ymmword ptr [rbp+r10*4+30h]
                                                ;...c4
                                                ;...21
                                                ;...7e
                                                ;...6f
                                                ;...4c
                                                ;...95
                                                ;...30

  0x000001c89e499411: vmovdqu ymm13,ymmword ptr [rbp+r10*4+0f0h]
                                                ;...c4
                                                ;...21
                                                ;...7e
                                                ;...6f
                                                ;...ac
                                                ;...95
                                                ;...f0
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e49941b: vmovdqu ymm12,ymmword ptr [rbp+r10*4+50h]
                                                ;...c4
                                                ;...21
                                                ;...7e
                                                ;...6f
                                                ;...64
                                                ;...95
                                                ;...50

  0x000001c89e499422: vmovdqu ymm4,ymmword ptr [rbp+r10*4+70h]
                                                ;...c4
                                                ;...a1
                                                ;...7e
                                                ;...6f
                                                ;...64
                                                ;...95
                                                ;...70

  0x000001c89e499429: vmovdqu ymm3,ymmword ptr [rbp+r10*4+90h]
                                                ;...c4
                                                ;...a1
                                                ;...7e
                                                ;...6f
                                                ;...9c
                                                ;...95
                                                ;...90
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e499433: vmovdqu ymm2,ymmword ptr [rbp+r10*4+0b0h]
                                                ;...c4
                                                ;...a1
                                                ;...7e
                                                ;...6f
                                                ;...94
                                                ;...95
                                                ;...b0
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001c89e49943d: vmovdqu ymm0,ymmword ptr [rbp+r10*4+0d0h]
                                                ;...c4
                                                ;...a1
                                                ;...7e
                                                ;...6f
                                                ;...84
                                                ;...95
                                                ;...d0
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@23 (line 42)

  0x000001c89e499447: vpslld  ymm11,ymm8,1h     ;...c4
                                                ;...c1
                                                ;...25
                                                ;...72
                                                ;...f0
                                                ;...01

  0x000001c89e49944d: vpslld  ymm1,ymm0,1h      ;...c5
                                                ;...f5
                                                ;...72
                                                ;...f0
                                                ;...01

  0x000001c89e499452: vpslld  ymm0,ymm0,3h      ;...c5
                                                ;...fd
                                                ;...72
                                                ;...f0
                                                ;...03

  0x000001c89e499457: vpaddd  ymm5,ymm0,ymm1    ;...c5
                                                ;...fd
                                                ;...fe
                                                ;...e9

  0x000001c89e49945b: vpslld  ymm0,ymm2,3h      ;...c5
                                                ;...fd
                                                ;...72
                                                ;...f2
                                                ;...03

  0x000001c89e499460: vpslld  ymm7,ymm3,3h      ;...c5
                                                ;...c5
                                                ;...72
                                                ;...f3
                                                ;...03

  0x000001c89e499465: vpslld  ymm10,ymm4,3h     ;...c5
                                                ;...ad
                                                ;...72
                                                ;...f4
                                                ;...03

  0x000001c89e49946a: vpslld  ymm15,ymm12,3h    ;...c4
                                                ;...c1
                                                ;...05
                                                ;...72
                                                ;...f4
                                                ;...03

  0x000001c89e499470: vpslld  ymm14,ymm13,3h    ;...c4
                                                ;...c1
                                                ;...0d
                                                ;...72
                                                ;...f5
                                                ;...03

  0x000001c89e499476: vpslld  ymm1,ymm9,3h      ;...c4
                                                ;...c1
                                                ;...75
                                                ;...72
                                                ;...f1
                                                ;...03

  0x000001c89e49947c: vpslld  ymm2,ymm2,1h      ;...c5
                                                ;...ed
                                                ;...72
                                                ;...f2
                                                ;...01

  0x000001c89e499481: vpaddd  ymm6,ymm0,ymm2    ;...c5
                                                ;...fd
                                                ;...fe
                                                ;...f2

  0x000001c89e499485: vpslld  ymm0,ymm3,1h      ;...c5
                                                ;...fd
                                                ;...72
                                                ;...f3
                                                ;...01

  0x000001c89e49948a: vpaddd  ymm7,ymm7,ymm0    ;...c5
                                                ;...c5
                                                ;...fe
                                                ;...f8

  0x000001c89e49948e: vpslld  ymm0,ymm4,1h      ;...c5
                                                ;...fd
                                                ;...72
                                                ;...f4
                                                ;...01

  0x000001c89e499493: vpaddd  ymm10,ymm10,ymm0  ;...c5
                                                ;...2d
                                                ;...fe
                                                ;...d0

  0x000001c89e499497: vpslld  ymm0,ymm12,1h     ;...c4
                                                ;...c1
                                                ;...7d
                                                ;...72
                                                ;...f4
                                                ;...01

  0x000001c89e49949d: vpaddd  ymm12,ymm15,ymm0  ;...c5
                                                ;...05
                                                ;...fe
                                                ;...e0

  0x000001c89e4994a1: vpslld  ymm0,ymm13,1h     ;...c4
                                                ;...c1
                                                ;...7d
                                                ;...72
                                                ;...f5
                                                ;...01

  0x000001c89e4994a7: vpaddd  ymm4,ymm14,ymm0   ;...c5
                                                ;...8d
                                                ;...fe
                                                ;...e0

  0x000001c89e4994ab: vpslld  ymm0,ymm9,1h      ;...c4
                                                ;...c1
                                                ;...7d
                                                ;...72
                                                ;...f1
                                                ;...01

  0x000001c89e4994b1: vpaddd  ymm2,ymm1,ymm0    ;...c5
                                                ;...f5
                                                ;...fe
                                                ;...d0

  0x000001c89e4994b5: vpslld  ymm0,ymm8,3h      ;...c4
                                                ;...c1
                                                ;...7d
                                                ;...72
                                                ;...f0
                                                ;...03

  0x000001c89e4994bb: vpaddd  ymm8,ymm0,ymm11   ;...c4
                                                ;...41
                                                ;...7d
                                                ;...fe
                                                ;...c3

  0x000001c89e4994c0: vphaddd ymm0,ymm8,ymm8    ;...c4
                                                ;...c2
                                                ;...3d
                                                ;...02
                                                ;...c0

  0x000001c89e4994c5: vphaddd ymm0,ymm0,ymm3    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c3

  0x000001c89e4994ca: vextracti128 xmm3,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c3
                                                ;...01

  0x000001c89e4994d0: vpaddd  xmm0,xmm0,xmm3    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c3

  0x000001c89e4994d4: vmovd   xmm3,ebx          ;...c5
                                                ;...f9
                                                ;...6e
                                                ;...db

  0x000001c89e4994d8: vpaddd  xmm3,xmm3,xmm0    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d8

  0x000001c89e4994dc: vmovd   r10d,xmm3         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...da

  0x000001c89e4994e1: vphaddd ymm0,ymm2,ymm2    ;...c4
                                                ;...e2
                                                ;...6d
                                                ;...02
                                                ;...c2

  0x000001c89e4994e6: vphaddd ymm0,ymm0,ymm3    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c3

  0x000001c89e4994eb: vextracti128 xmm3,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c3
                                                ;...01

  0x000001c89e4994f1: vpaddd  xmm0,xmm0,xmm3    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c3

  0x000001c89e4994f5: vmovd   xmm3,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...da

  0x000001c89e4994fa: vpaddd  xmm3,xmm3,xmm0    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d8

  0x000001c89e4994fe: vmovd   r11d,xmm3         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...db

  0x000001c89e499503: vphaddd ymm2,ymm12,ymm12  ;...c4
                                                ;...c2
                                                ;...1d
                                                ;...02
                                                ;...d4

  0x000001c89e499508: vphaddd ymm2,ymm2,ymm0    ;...c4
                                                ;...e2
                                                ;...6d
                                                ;...02
                                                ;...d0

  0x000001c89e49950d: vextracti128 xmm0,ymm2,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d0
                                                ;...01

  0x000001c89e499513: vpaddd  xmm2,xmm2,xmm0    ;...c5
                                                ;...e9
                                                ;...fe
                                                ;...d0

  0x000001c89e499517: vmovd   xmm0,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c3

  0x000001c89e49951c: vpaddd  xmm0,xmm0,xmm2    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c2

  0x000001c89e499520: vmovd   r10d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c2

  0x000001c89e499525: vphaddd ymm0,ymm10,ymm10  ;...c4
                                                ;...c2
                                                ;...2d
                                                ;...02
                                                ;...c2

  0x000001c89e49952a: vphaddd ymm0,ymm0,ymm3    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c3

  0x000001c89e49952f: vextracti128 xmm3,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c3
                                                ;...01

  0x000001c89e499535: vpaddd  xmm0,xmm0,xmm3    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c3

  0x000001c89e499539: vmovd   xmm3,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...da

  0x000001c89e49953e: vpaddd  xmm3,xmm3,xmm0    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d8

  0x000001c89e499542: vmovd   r11d,xmm3         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...db

  0x000001c89e499547: vphaddd ymm2,ymm7,ymm7    ;...c4
                                                ;...e2
                                                ;...45
                                                ;...02
                                                ;...d7

  0x000001c89e49954c: vphaddd ymm2,ymm2,ymm0    ;...c4
                                                ;...e2
                                                ;...6d
                                                ;...02
                                                ;...d0

  0x000001c89e499551: vextracti128 xmm0,ymm2,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d0
                                                ;...01

  0x000001c89e499557: vpaddd  xmm2,xmm2,xmm0    ;...c5
                                                ;...e9
                                                ;...fe
                                                ;...d0

  0x000001c89e49955b: vmovd   xmm0,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c3

  0x000001c89e499560: vpaddd  xmm0,xmm0,xmm2    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c2

  0x000001c89e499564: vmovd   r10d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c2

  0x000001c89e499569: vphaddd ymm0,ymm6,ymm6    ;...c4
                                                ;...e2
                                                ;...4d
                                                ;...02
                                                ;...c6

  0x000001c89e49956e: vphaddd ymm0,ymm0,ymm3    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c3

  0x000001c89e499573: vextracti128 xmm3,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c3
                                                ;...01

  0x000001c89e499579: vpaddd  xmm0,xmm0,xmm3    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c3

  0x000001c89e49957d: vmovd   xmm3,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...da

  0x000001c89e499582: vpaddd  xmm3,xmm3,xmm0    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d8

  0x000001c89e499586: vmovd   r11d,xmm3         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...db

  0x000001c89e49958b: vphaddd ymm2,ymm5,ymm5    ;...c4
                                                ;...e2
                                                ;...55
                                                ;...02
                                                ;...d5

  0x000001c89e499590: vphaddd ymm2,ymm2,ymm0    ;...c4
                                                ;...e2
                                                ;...6d
                                                ;...02
                                                ;...d0

  0x000001c89e499595: vextracti128 xmm0,ymm2,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d0
                                                ;...01

  0x000001c89e49959b: vpaddd  xmm2,xmm2,xmm0    ;...c5
                                                ;...e9
                                                ;...fe
                                                ;...d0

  0x000001c89e49959f: vmovd   xmm0,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c3

  0x000001c89e4995a4: vpaddd  xmm0,xmm0,xmm2    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c2

  0x000001c89e4995a8: vmovd   r10d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c2

  0x000001c89e4995ad: vphaddd ymm2,ymm4,ymm4    ;...c4
                                                ;...e2
                                                ;...5d
                                                ;...02
                                                ;...d4

  0x000001c89e4995b2: vphaddd ymm2,ymm2,ymm1    ;...c4
                                                ;...e2
                                                ;...6d
                                                ;...02
                                                ;...d1

  0x000001c89e4995b7: vextracti128 xmm1,ymm2,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d1
                                                ;...01

  0x000001c89e4995bd: vpaddd  xmm2,xmm2,xmm1    ;...c5
                                                ;...e9
                                                ;...fe
                                                ;...d1

  0x000001c89e4995c1: vmovd   xmm1,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...ca

  0x000001c89e4995c6: vpaddd  xmm1,xmm1,xmm2    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...ca

  0x000001c89e4995ca: vmovd   ebx,xmm1          ;...c5
                                                ;...f9
                                                ;...7e
                                                ;...cb
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@25 (line 42)

  0x000001c89e4995ce: add     r13d,40h          ;...41
                                                ;...83
                                                ;...c5
                                                ;...40
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@27 (line 41)

  0x000001c89e4995d2: cmp     r13d,r8d          ;...45
                                                ;...3b
                                                ;...e8

  0x000001c89e4995d5: jl      1c89e499400h      ;...0f
                                                ;...8c
                                                ;...25
                                                ;...fe
                                                ;...ff
                                                ;...ff
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@14 (line 41)

  0x000001c89e4995db: mov     r10d,edi          ;...44
                                                ;...8b
                                                ;...d7

  0x000001c89e4995de: add     r10d,0fffffff9h   ;...41
                                                ;...83
                                                ;...c2
                                                ;...f9

  0x000001c89e4995e2: cmp     edi,r10d          ;...41
                                                ;...3b
                                                ;...fa

  0x000001c89e4995e5: cmovl   r10d,ecx          ;...44
                                                ;...0f
                                                ;...4c
                                                ;...d1

  0x000001c89e4995e9: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000001c89e4995ec: jnl     1c89e49962eh      ;...7d
                                                ;...40

  0x000001c89e4995ee: nop                       ;...66
                                                ;...90

  0x000001c89e4995f0: vmovdqu ymm0,ymmword ptr [rbp+r13*4+10h]
                                                ;...c4
                                                ;...a1
                                                ;...7e
                                                ;...6f
                                                ;...44
                                                ;...ad
                                                ;...10
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@23 (line 42)

  0x000001c89e4995f7: vpslld  ymm1,ymm0,3h      ;...c5
                                                ;...f5
                                                ;...72
                                                ;...f0
                                                ;...03

  0x000001c89e4995fc: vpslld  ymm0,ymm0,1h      ;...c5
                                                ;...fd
                                                ;...72
                                                ;...f0
                                                ;...01

  0x000001c89e499601: vpaddd  ymm0,ymm1,ymm0    ;...c5
                                                ;...f5
                                                ;...fe
                                                ;...c0

  0x000001c89e499605: vphaddd ymm3,ymm0,ymm0    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...d8

  0x000001c89e49960a: vphaddd ymm3,ymm3,ymm1    ;...c4
                                                ;...e2
                                                ;...65
                                                ;...02
                                                ;...d9

  0x000001c89e49960f: vextracti128 xmm1,ymm3,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d9
                                                ;...01

  0x000001c89e499615: vpaddd  xmm3,xmm3,xmm1    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d9

  0x000001c89e499619: vmovd   xmm1,ebx          ;...c5
                                                ;...f9
                                                ;...6e
                                                ;...cb

  0x000001c89e49961d: vpaddd  xmm1,xmm1,xmm3    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...cb

  0x000001c89e499621: vmovd   ebx,xmm1          ;...c5
                                                ;...f9
                                                ;...7e
                                                ;...cb
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@25 (line 42)

  0x000001c89e499625: add     r13d,8h           ;...41
                                                ;...83
                                                ;...c5
                                                ;...08
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@27 (line 41)

  0x000001c89e499629: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000001c89e49962c: jl      1c89e4995f0h      ;...7c
                                                ;...c2
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@14 (line 41)

  0x000001c89e49962e: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e499631: jnl     1c89e499651h      ;...7d
                                                ;...1e

  0x000001c89e499633: nop                       ;...90

  0x000001c89e499634: mov     r11d,dword ptr [rbp+r13*4+10h]
                                                ;...46
                                                ;...8b
                                                ;...5c
                                                ;...ad
                                                ;...10
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@23 (line 42)

  0x000001c89e499639: mov     r10d,r11d         ;...45
                                                ;...8b
                                                ;...d3

  0x000001c89e49963c: shl     r10d,3h           ;...41
                                                ;...c1
                                                ;...e2
                                                ;...03

  0x000001c89e499640: shl     r11d,1h           ;...41
                                                ;...d1
                                                ;...e3

  0x000001c89e499643: add     r10d,r11d         ;...45
                                                ;...03
                                                ;...d3

  0x000001c89e499646: add     ebx,r10d          ;...41
                                                ;...03
                                                ;...da
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@25 (line 42)

  0x000001c89e499649: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@27 (line 41)

  0x000001c89e49964c: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e49964f: jl      1c89e499634h      ;...7c
                                                ;...e3
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@14 (line 41)

  0x000001c89e499651: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e499654: jnl     1c89e49967ah      ;...7d
                                                ;...24

  0x000001c89e499656: nop                       ;...66
                                                ;...90

  0x000001c89e499658: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e49965b: jnb     1c89e499691h      ;...73
                                                ;...34

  0x000001c89e49965d: mov     r11d,dword ptr [rbp+r13*4+10h]
                                                ;...46
                                                ;...8b
                                                ;...5c
                                                ;...ad
                                                ;...10
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@23 (line 42)

  0x000001c89e499662: mov     r10d,r11d         ;...45
                                                ;...8b
                                                ;...d3

  0x000001c89e499665: shl     r10d,3h           ;...41
                                                ;...c1
                                                ;...e2
                                                ;...03

  0x000001c89e499669: shl     r11d,1h           ;...41
                                                ;...d1
                                                ;...e3

  0x000001c89e49966c: add     r10d,r11d         ;...45
                                                ;...03
                                                ;...d3

  0x000001c89e49966f: add     ebx,r10d          ;...41
                                                ;...03
                                                ;...da
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@25 (line 42)

  0x000001c89e499672: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@27 (line 41)

  0x000001c89e499675: cmp     r13d,edi          ;...44
                                                ;...3b
                                                ;...ef

  0x000001c89e499678: jl      1c89e499658h      ;...7c
                                                ;...de
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int@10 (line 41)

  0x000001c89e49967a: mov     eax,ebx           ;...8b
                                                ;...c3

  0x000001c89e49967c: vzeroupper                ;...c5
                                                ;...f8
                                                ;...77

  0x000001c89e49967f: add     rsp,40h           ;...48
                                                ;...83
                                                ;...c4
                                                ;...40

  0x000001c89e499683: pop     rbp               ;...5d

  0x000001c89e499684: test    dword ptr [1c88a9e0000h],eax
                                                ;...85
                                                ;...05
                                                ;...76
                                                ;...69
                                                ;...54
                                                ;...ec
                                                ;   {poll_return}
  0x000001c89e49968a: ret 

The loop is aggressively unrolled, pipelined, and vectorised. Moreover, the multiplication by ten results not in a multiplication but two left shifts (see VPSLLD) and an addition. Note that x << 1 + x << 3 = x * 10 and C2 seems to know it; this rewrite can be applied because it can be proven statically that the factor is always 10. The “optimised” loop doesn’t vectorise at all (and I have no idea why not – isn’t this a bug? Yes it is.)

com/openkappa/simd/scale/Scale.FactoredScale_Int(Lcom/openkappa/simd/state/IntData;)I  [0x000002bbebeda320, 0x000002bbebeda4b8]  408 bytes
Argument 0 is unknown.RIP: 0x2bbebeda320 Code size: 0x00000198
[Entry Point]
[Verified Entry Point]
[Constants]
  # {method} {0x000002bb81241148} 'FactoredScale_Int' '(Lcom/openkappa/simd/state/IntData;)I' in 'com/openkappa/simd/scale/Scale'
  0x000002bbebeda320: int3                      ;...cc

  0x000002bbebeda321: nop     word ptr [rax+rax+0h]  ;...66
                                                ;...66
                                                ;...66
                                                ;...0f
                                                ;...1f
                                                ;...84
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002bbebeda32c: nop                       ;...66
                                                ;...66
                                                ;...66
                                                ;...90

  0x000002bbebeda330: mov     dword ptr [rsp+0ffffffffffff9000h],eax
                                                ;...89
                                                ;...84
                                                ;...24
                                                ;...00
                                                ;...90
                                                ;...ff
                                                ;...ff

  0x000002bbebeda337: push    rbp               ;...55

  0x000002bbebeda338: sub     rsp,40h           ;...48
                                                ;...83
                                                ;...ec
                                                ;...40

  0x000002bbebeda33c: mov     rbp,qword ptr [rdx+8h]  ;...48
                                                ;...8b
                                                ;...6a
                                                ;...08

  0x000002bbebeda340: mov     ebx,dword ptr [rdx+10h]  ;...8b
                                                ;...5a
                                                ;...10

  0x000002bbebeda343: mov     r13d,dword ptr [rdx]  ;...44
                                                ;...8b
                                                ;...2a

  0x000002bbebeda346: mov     rcx,rdx           ;...48
                                                ;...8b
                                                ;...ca

  0x000002bbebeda349: mov     r10,51da8d20h     ;...49
                                                ;...ba
                                                ;...20
                                                ;...8d
                                                ;...da
                                                ;...51
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002bbebeda353: call indirect r10         ;...41
                                                ;...ff
                                                ;...d2

  0x000002bbebeda356: mov     r10d,dword ptr [rbp+8h]  ;...44
                                                ;...8b
                                                ;...55
                                                ;...08
                                                ; implicit exception: dispatches to 0x000002bbebeda46d
  0x000002bbebeda35a: cmp     r10d,0f800016dh   ;...41
                                                ;...81
                                                ;...fa
                                                ;...6d
                                                ;...01
                                                ;...00
                                                ;...f8
                                                ;   {metadata({type array int})}
  0x000002bbebeda361: jne     2bbebeda459h      ;...0f
                                                ;...85
                                                ;...f2
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@10 (line 52)

  0x000002bbebeda367: mov     r10d,dword ptr [rbp+0ch]
                                                ;...44
                                                ;...8b
                                                ;...55
                                                ;...0c
                                                ;*arraylength {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@13 (line 52)

  0x000002bbebeda36b: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda36e: jnl     2bbebeda422h      ;...0f
                                                ;...8d
                                                ;...ae
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@14 (line 52)

  0x000002bbebeda374: mov     r11d,r13d         ;...45
                                                ;...8b
                                                ;...dd

  0x000002bbebeda377: inc     r11d              ;...41
                                                ;...ff
                                                ;...c3

  0x000002bbebeda37a: xor     r8d,r8d           ;...45
                                                ;...33
                                                ;...c0

  0x000002bbebeda37d: cmp     r11d,r8d          ;...45
                                                ;...3b
                                                ;...d8

  0x000002bbebeda380: cmovl   r11d,r8d          ;...45
                                                ;...0f
                                                ;...4c
                                                ;...d8

  0x000002bbebeda384: cmp     r11d,r10d         ;...45
                                                ;...3b
                                                ;...da

  0x000002bbebeda387: cmovnle r11d,r10d         ;...45
                                                ;...0f
                                                ;...4f
                                                ;...da

  0x000002bbebeda38b: nop                       ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@21 (line 53)

  0x000002bbebeda38c: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda38f: jnb     2bbebeda43ch      ;...0f
                                                ;...83
                                                ;...a7
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002bbebeda395: add     ebx,dword ptr [rbp+r13*4+10h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...ad
                                                ;...10
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@22 (line 53)

  0x000002bbebeda39a: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@24 (line 52)

  0x000002bbebeda39d: cmp     r13d,r11d         ;...45
                                                ;...3b
                                                ;...eb

  0x000002bbebeda3a0: jl      2bbebeda38ch      ;...7c
                                                ;...ea
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@14 (line 52)

  0x000002bbebeda3a2: mov     r11d,r10d         ;...45
                                                ;...8b
                                                ;...da

  0x000002bbebeda3a5: add     r11d,0fffffff9h   ;...41
                                                ;...83
                                                ;...c3
                                                ;...f9

  0x000002bbebeda3a9: mov     r8d,80000000h     ;...41
                                                ;...b8
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...80

  0x000002bbebeda3af: cmp     r10d,r11d         ;...45
                                                ;...3b
                                                ;...d3

  0x000002bbebeda3b2: cmovl   r11d,r8d          ;...45
                                                ;...0f
                                                ;...4c
                                                ;...d8

  0x000002bbebeda3b6: cmp     r13d,r11d         ;...45
                                                ;...3b
                                                ;...eb

  0x000002bbebeda3b9: jnl     2bbebeda409h      ;...7d
                                                ;...4e

  0x000002bbebeda3bb: nop     dword ptr [rax+rax+0h]  ;...0f
                                                ;...1f
                                                ;...44
                                                ;...00
                                                ;...00
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@21 (line 53)

  0x000002bbebeda3c0: add     ebx,dword ptr [rbp+r13*4+10h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...ad
                                                ;...10

  0x000002bbebeda3c5: movsxd  r8,r13d           ;...4d
                                                ;...63
                                                ;...c5

  0x000002bbebeda3c8: add     ebx,dword ptr [rbp+r8*4+14h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...14

  0x000002bbebeda3cd: add     ebx,dword ptr [rbp+r8*4+18h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...18

  0x000002bbebeda3d2: add     ebx,dword ptr [rbp+r8*4+1ch]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...1c

  0x000002bbebeda3d7: add     ebx,dword ptr [rbp+r8*4+20h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...20

  0x000002bbebeda3dc: add     ebx,dword ptr [rbp+r8*4+24h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...24

  0x000002bbebeda3e1: add     ebx,dword ptr [rbp+r8*4+28h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...28

  0x000002bbebeda3e6: add     ebx,dword ptr [rbp+r8*4+2ch]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...85
                                                ;...2c
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@22 (line 53)

  0x000002bbebeda3eb: add     r13d,8h           ;...41
                                                ;...83
                                                ;...c5
                                                ;...08
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@24 (line 52)

  0x000002bbebeda3ef: cmp     r13d,r11d         ;...45
                                                ;...3b
                                                ;...eb

  0x000002bbebeda3f2: jl      2bbebeda3c0h      ;...7c
                                                ;...cc
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@14 (line 52)

  0x000002bbebeda3f4: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda3f7: jnl     2bbebeda409h      ;...7d
                                                ;...10

  0x000002bbebeda3f9: nop                       ;...66
                                                ;...66
                                                ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@21 (line 53)

  0x000002bbebeda3fc: add     ebx,dword ptr [rbp+r13*4+10h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...ad
                                                ;...10
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@22 (line 53)

  0x000002bbebeda401: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@24 (line 52)

  0x000002bbebeda404: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda407: jl      2bbebeda3fch      ;...7c
                                                ;...f3
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@14 (line 52)

  0x000002bbebeda409: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda40c: jnl     2bbebeda422h      ;...7d
                                                ;...14

  0x000002bbebeda40e: nop                       ;...66
                                                ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@21 (line 53)

  0x000002bbebeda410: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda413: jnb     2bbebeda442h      ;...73
                                                ;...2d

  0x000002bbebeda415: add     ebx,dword ptr [rbp+r13*4+10h]
                                                ;...42
                                                ;...03
                                                ;...5c
                                                ;...ad
                                                ;...10
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@22 (line 53)

  0x000002bbebeda41a: inc     r13d              ;...41
                                                ;...ff
                                                ;...c5
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@24 (line 52)

  0x000002bbebeda41d: cmp     r13d,r10d         ;...45
                                                ;...3b
                                                ;...ea

  0x000002bbebeda420: jl      2bbebeda410h      ;...7c
                                                ;...ee
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@10 (line 52)

  0x000002bbebeda422: mov     r11d,ebx          ;...44
                                                ;...8b
                                                ;...db

  0x000002bbebeda425: shl     r11d,3h           ;...41
                                                ;...c1
                                                ;...e3
                                                ;...03

  0x000002bbebeda429: shl     ebx,1h            ;...d1
                                                ;...e3

  0x000002bbebeda42b: add     ebx,r11d          ;...41
                                                ;...03
                                                ;...db
                                                ;*imul {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::FactoredScale_Int@33 (line 55)

  0x000002bbebeda42e: mov     eax,ebx           ;...8b
                                                ;...c3

  0x000002bbebeda430: add     rsp,40h           ;...48
                                                ;...83
                                                ;...c4
                                                ;...40

  0x000002bbebeda434: pop     rbp               ;...5d

  0x000002bbebeda435: test    dword ptr [2bbd8590000h],eax
                                                ;...85
                                                ;...05
                                                ;...c5
                                                ;...5b
                                                ;...6b
                                                ;...ec
                                                ;   {poll_return}
  0x000002bbebeda43b: ret                       ;...c3

This is a special case: data is usually dynamic and variable, so the loop cannot always be proven to be equivalent to a linear combination of bit shifts. The routine is compiled for all possible parameters, not just statically contrived cases like the one above, so you may never see this assembly in the wild. However, even with random factors, the slow looking loop is aggressively optimised in a way the hand “optimised” code is not:


    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public int Scale_Int_Dynamic(ScaleState state) {
        int value = 0;
        int[] data = state.data;
        int factor = state.randomFactor();
        for (int i = 0; i < data.length; ++i) {
            value += factor * data[i];
        }
        return value;
    }

    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    @Benchmark
    public int FactoredScale_Int_Dynamic(ScaleState state) {
        int value = 0;
        int[] data = state.data;
        int factor = state.randomFactor();
        for (int i = 0; i < data.length; ++i) {
            value += data[i];
        }
        return factor * value;
    }

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit Param: size
FactoredScale_Int_Dynamic thrpt 1 10 26.100439 0.340069 ops/ms 100000
FactoredScale_Int_Dynamic thrpt 1 10 1.918011 0.297925 ops/ms 1000000
Scale_Int_Dynamic thrpt 1 10 30.219809 2.977389 ops/ms 100000
Scale_Int_Dynamic thrpt 1 10 2.314159 0.378442 ops/ms 1000000

Far from seeking to exploit distributivity to reduce the number of multiplication instructions, it seems to almost embrace the extraneous operations as metadata to drive optimisations. The assembly for Scale_Int_Dynamic confirms this (it shows vectorised multiplication, not shifts, within the loop):

com/openkappa/simd/scale/Scale.Scale_Int_Dynamic(Lcom/openkappa/simd/scale/ScaleState;)I  [0x000001f5ca2fa120, 0x000001f5ca2fa498]  888 bytes
Argument 0 is unknown.RIP: 0x1f5ca2fa120 Code size: 0x00000378
[Entry Point]
[Verified Entry Point]
[Constants]
  # {method} {0x000001f5df561a20} 'Scale_Int_Dynamic' '(Lcom/openkappa/simd/scale/ScaleState;)I' in 'com/openkappa/simd/scale/Scale'
  0x000001f5ca2fa120: int3                      ;...cc

  0x000001f5ca2fa121: nop     word ptr [rax+rax+0h]  ;...66
                                                ;...66
                                                ;...66
                                                ;...0f
                                                ;...1f
                                                ;...84
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa12c: nop                       ;...66
                                                ;...66
                                                ;...66
                                                ;...90

  0x000001f5ca2fa130: mov     dword ptr [rsp+0ffffffffffff9000h],eax
                                                ;...89
                                                ;...84
                                                ;...24
                                                ;...00
                                                ;...90
                                                ;...ff
                                                ;...ff

  0x000001f5ca2fa137: push    rbp               ;...55

  0x000001f5ca2fa138: sub     rsp,50h           ;...48
                                                ;...83
                                                ;...ec
                                                ;...50

  0x000001f5ca2fa13c: mov     r13,qword ptr [rdx+10h]  ;...4c
                                                ;...8b
                                                ;...6a
                                                ;...10

  0x000001f5ca2fa140: mov     ebx,dword ptr [rdx+18h]  ;...8b
                                                ;...5a
                                                ;...18

  0x000001f5ca2fa143: mov     ebp,dword ptr [rdx+8h]  ;...8b
                                                ;...6a
                                                ;...08

  0x000001f5ca2fa146: mov     r14d,dword ptr [rdx]  ;...44
                                                ;...8b
                                                ;...32

  0x000001f5ca2fa149: mov     rcx,rdx           ;...48
                                                ;...8b
                                                ;...ca

  0x000001f5ca2fa14c: vzeroupper                ;...c5
                                                ;...f8
                                                ;...77

  0x000001f5ca2fa14f: mov     r10,51da8d20h     ;...49
                                                ;...ba
                                                ;...20
                                                ;...8d
                                                ;...da
                                                ;...51
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa159: call indirect r10         ;...41
                                                ;...ff
                                                ;...d2

  0x000001f5ca2fa15c: mov     r10d,dword ptr [r13+8h]  ;...45
                                                ;...8b
                                                ;...55
                                                ;...08
                                                ; implicit exception: dispatches to 0x000001f5ca2fa461
  0x000001f5ca2fa160: cmp     r10d,0f800016dh   ;...41
                                                ;...81
                                                ;...fa
                                                ;...6d
                                                ;...01
                                                ;...00
                                                ;...f8
                                                ;   {metadata({type array int})}
  0x000001f5ca2fa167: jne     1f5ca2fa445h      ;...0f
                                                ;...85
                                                ;...d8
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@16 (line 64)

  0x000001f5ca2fa16d: mov     edi,dword ptr [r13+0ch]  ;...41
                                                ;...8b
                                                ;...7d
                                                ;...0c
                                                ;*arraylength {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@19 (line 64)

  0x000001f5ca2fa171: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa174: jnl     1f5ca2fa411h      ;...0f
                                                ;...8d
                                                ;...97
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@20 (line 64)

  0x000001f5ca2fa17a: mov     r11d,r13d         ;...45
                                                ;...8b
                                                ;...dd

  0x000001f5ca2fa17d: mov     r10d,r14d         ;...45
                                                ;...8b
                                                ;...d6

  0x000001f5ca2fa180: inc     r10d              ;...41
                                                ;...ff
                                                ;...c2

  0x000001f5ca2fa183: shr     r11d,2h           ;...41
                                                ;...c1
                                                ;...eb
                                                ;...02

  0x000001f5ca2fa187: and     r11d,7h           ;...41
                                                ;...83
                                                ;...e3
                                                ;...07

  0x000001f5ca2fa18b: xor     r8d,r8d           ;...45
                                                ;...33
                                                ;...c0

  0x000001f5ca2fa18e: cmp     r10d,r8d          ;...45
                                                ;...3b
                                                ;...d0

  0x000001f5ca2fa191: cmovl   r10d,r8d          ;...45
                                                ;...0f
                                                ;...4c
                                                ;...d0

  0x000001f5ca2fa195: cmp     r10d,edi          ;...44
                                                ;...3b
                                                ;...d7

  0x000001f5ca2fa198: cmovnle r10d,edi          ;...44
                                                ;...0f
                                                ;...4f
                                                ;...d7

  0x000001f5ca2fa19c: add     r11d,r10d         ;...45
                                                ;...03
                                                ;...da

  0x000001f5ca2fa19f: mov     r9d,4h            ;...41
                                                ;...b9
                                                ;...04
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa1a5: sub     r9d,r11d          ;...45
                                                ;...2b
                                                ;...cb

  0x000001f5ca2fa1a8: and     r9d,7h            ;...41
                                                ;...83
                                                ;...e1
                                                ;...07

  0x000001f5ca2fa1ac: add     r9d,r10d          ;...45
                                                ;...03
                                                ;...ca

  0x000001f5ca2fa1af: cmp     r9d,edi           ;...44
                                                ;...3b
                                                ;...cf

  0x000001f5ca2fa1b2: cmovnle r9d,edi           ;...44
                                                ;...0f
                                                ;...4f
                                                ;...cf
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@29 (line 65)

  0x000001f5ca2fa1b6: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa1b9: jnb     1f5ca2fa422h      ;...0f
                                                ;...83
                                                ;...63
                                                ;...02
                                                ;...00
                                                ;...00

  0x000001f5ca2fa1bf: mov     r11d,ebp          ;...44
                                                ;...8b
                                                ;...dd

  0x000001f5ca2fa1c2: imul    r11d,dword ptr [r13+r14*4+10h]
                                                ;...47
                                                ;...0f
                                                ;...af
                                                ;...5c
                                                ;...b5
                                                ;...10

  0x000001f5ca2fa1c8: add     ebx,r11d          ;...41
                                                ;...03
                                                ;...db
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@31 (line 65)

  0x000001f5ca2fa1cb: inc     r14d              ;...41
                                                ;...ff
                                                ;...c6
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@33 (line 64)

  0x000001f5ca2fa1ce: cmp     r14d,r9d          ;...45
                                                ;...3b
                                                ;...f1

  0x000001f5ca2fa1d1: jl      1f5ca2fa1b6h      ;...7c
                                                ;...e3
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@20 (line 64)

  0x000001f5ca2fa1d3: mov     r9d,edi           ;...44
                                                ;...8b
                                                ;...cf

  0x000001f5ca2fa1d6: add     r9d,0ffffffc1h    ;...41
                                                ;...83
                                                ;...c1
                                                ;...c1

  0x000001f5ca2fa1da: mov     r8d,80000000h     ;...41
                                                ;...b8
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...80

  0x000001f5ca2fa1e0: cmp     edi,r9d           ;...41
                                                ;...3b
                                                ;...f9

  0x000001f5ca2fa1e3: cmovl   r9d,r8d           ;...45
                                                ;...0f
                                                ;...4c
                                                ;...c8

  0x000001f5ca2fa1e7: cmp     r14d,r9d          ;...45
                                                ;...3b
                                                ;...f1

  0x000001f5ca2fa1ea: jnl     1f5ca2fa3f0h      ;...0f
                                                ;...8d
                                                ;...00
                                                ;...02
                                                ;...00
                                                ;...00

  0x000001f5ca2fa1f0: vmovd   xmm2,ebp          ;...c5
                                                ;...f9
                                                ;...6e
                                                ;...d5

  0x000001f5ca2fa1f4: vpshufd xmm2,xmm2,0h      ;...c5
                                                ;...f9
                                                ;...70
                                                ;...d2
                                                ;...00

  0x000001f5ca2fa1f9: vinserti128 ymm2,ymm2,xmm2,1h  ;...c4
                                                ;...e3
                                                ;...6d
                                                ;...38
                                                ;...d2
                                                ;...01

  0x000001f5ca2fa1ff: nop                       ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@29 (line 65)

  0x000001f5ca2fa200: vmovdqu ymm0,ymmword ptr [r13+r14*4+10h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...44
                                                ;...b5
                                                ;...10

  0x000001f5ca2fa207: vpmulld ymm11,ymm0,ymm2   ;...c4
                                                ;...62
                                                ;...7d
                                                ;...40
                                                ;...da

  0x000001f5ca2fa20c: movsxd  r10,r14d          ;...4d
                                                ;...63
                                                ;...d6

  0x000001f5ca2fa20f: vmovdqu ymm0,ymmword ptr [r13+r10*4+30h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...44
                                                ;...95
                                                ;...30

  0x000001f5ca2fa216: vmovdqu ymm1,ymmword ptr [r13+r10*4+0f0h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...8c
                                                ;...95
                                                ;...f0
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa220: vmovdqu ymm3,ymmword ptr [r13+r10*4+50h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...5c
                                                ;...95
                                                ;...50

  0x000001f5ca2fa227: vmovdqu ymm7,ymmword ptr [r13+r10*4+70h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...7c
                                                ;...95
                                                ;...70

  0x000001f5ca2fa22e: vmovdqu ymm6,ymmword ptr [r13+r10*4+90h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...b4
                                                ;...95
                                                ;...90
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa238: vmovdqu ymm5,ymmword ptr [r13+r10*4+0b0h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...ac
                                                ;...95
                                                ;...b0
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa242: vmovdqu ymm4,ymmword ptr [r13+r10*4+0d0h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...a4
                                                ;...95
                                                ;...d0
                                                ;...00
                                                ;...00
                                                ;...00

  0x000001f5ca2fa24c: vpmulld ymm9,ymm0,ymm2    ;...c4
                                                ;...62
                                                ;...7d
                                                ;...40
                                                ;...ca

  0x000001f5ca2fa251: vpmulld ymm4,ymm4,ymm2    ;...c4
                                                ;...e2
                                                ;...5d
                                                ;...40
                                                ;...e2

  0x000001f5ca2fa256: vpmulld ymm5,ymm5,ymm2    ;...c4
                                                ;...e2
                                                ;...55
                                                ;...40
                                                ;...ea

  0x000001f5ca2fa25b: vpmulld ymm6,ymm6,ymm2    ;...c4
                                                ;...e2
                                                ;...4d
                                                ;...40
                                                ;...f2

  0x000001f5ca2fa260: vpmulld ymm8,ymm7,ymm2    ;...c4
                                                ;...62
                                                ;...45
                                                ;...40
                                                ;...c2

  0x000001f5ca2fa265: vpmulld ymm10,ymm3,ymm2   ;...c4
                                                ;...62
                                                ;...65
                                                ;...40
                                                ;...d2

  0x000001f5ca2fa26a: vpmulld ymm3,ymm1,ymm2    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...40
                                                ;...da

  0x000001f5ca2fa26f: vphaddd ymm1,ymm11,ymm11  ;...c4
                                                ;...c2
                                                ;...25
                                                ;...02
                                                ;...cb

  0x000001f5ca2fa274: vphaddd ymm1,ymm1,ymm0    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...c8

  0x000001f5ca2fa279: vextracti128 xmm0,ymm1,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c8
                                                ;...01

  0x000001f5ca2fa27f: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa283: vmovd   xmm0,ebx          ;...c5
                                                ;...f9
                                                ;...6e
                                                ;...c3

  0x000001f5ca2fa287: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa28b: vmovd   r10d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c2

  0x000001f5ca2fa290: vphaddd ymm1,ymm9,ymm9    ;...c4
                                                ;...c2
                                                ;...35
                                                ;...02
                                                ;...c9

  0x000001f5ca2fa295: vphaddd ymm1,ymm1,ymm0    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...c8

  0x000001f5ca2fa29a: vextracti128 xmm0,ymm1,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c8
                                                ;...01

  0x000001f5ca2fa2a0: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa2a4: vmovd   xmm0,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c2

  0x000001f5ca2fa2a9: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa2ad: vmovd   r11d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c3

  0x000001f5ca2fa2b2: vphaddd ymm0,ymm10,ymm10  ;...c4
                                                ;...c2
                                                ;...2d
                                                ;...02
                                                ;...c2

  0x000001f5ca2fa2b7: vphaddd ymm0,ymm0,ymm1    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c1

  0x000001f5ca2fa2bc: vextracti128 xmm1,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c1
                                                ;...01

  0x000001f5ca2fa2c2: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa2c6: vmovd   xmm1,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...cb

  0x000001f5ca2fa2cb: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa2cf: vmovd   r10d,xmm1         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...ca

  0x000001f5ca2fa2d4: vphaddd ymm1,ymm8,ymm8    ;...c4
                                                ;...c2
                                                ;...3d
                                                ;...02
                                                ;...c8

  0x000001f5ca2fa2d9: vphaddd ymm1,ymm1,ymm0    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...c8

  0x000001f5ca2fa2de: vextracti128 xmm0,ymm1,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c8
                                                ;...01

  0x000001f5ca2fa2e4: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa2e8: vmovd   xmm0,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c2

  0x000001f5ca2fa2ed: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa2f1: vmovd   r11d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c3

  0x000001f5ca2fa2f6: vphaddd ymm0,ymm6,ymm6    ;...c4
                                                ;...e2
                                                ;...4d
                                                ;...02
                                                ;...c6

  0x000001f5ca2fa2fb: vphaddd ymm0,ymm0,ymm1    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c1

  0x000001f5ca2fa300: vextracti128 xmm1,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c1
                                                ;...01

  0x000001f5ca2fa306: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa30a: vmovd   xmm1,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...cb

  0x000001f5ca2fa30f: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa313: vmovd   r10d,xmm1         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...ca

  0x000001f5ca2fa318: vphaddd ymm1,ymm5,ymm5    ;...c4
                                                ;...e2
                                                ;...55
                                                ;...02
                                                ;...cd

  0x000001f5ca2fa31d: vphaddd ymm1,ymm1,ymm0    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...c8

  0x000001f5ca2fa322: vextracti128 xmm0,ymm1,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c8
                                                ;...01

  0x000001f5ca2fa328: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa32c: vmovd   xmm0,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...c2

  0x000001f5ca2fa331: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa335: vmovd   r11d,xmm0         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...c3

  0x000001f5ca2fa33a: vphaddd ymm0,ymm4,ymm4    ;...c4
                                                ;...e2
                                                ;...5d
                                                ;...02
                                                ;...c4

  0x000001f5ca2fa33f: vphaddd ymm0,ymm0,ymm1    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...02
                                                ;...c1

  0x000001f5ca2fa344: vextracti128 xmm1,ymm0,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...c1
                                                ;...01

  0x000001f5ca2fa34a: vpaddd  xmm0,xmm0,xmm1    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c1

  0x000001f5ca2fa34e: vmovd   xmm1,r11d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...cb

  0x000001f5ca2fa353: vpaddd  xmm1,xmm1,xmm0    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...c8

  0x000001f5ca2fa357: vmovd   r10d,xmm1         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...7e
                                                ;...ca

  0x000001f5ca2fa35c: vphaddd ymm1,ymm3,ymm3    ;...c4
                                                ;...e2
                                                ;...65
                                                ;...02
                                                ;...cb

  0x000001f5ca2fa361: vphaddd ymm1,ymm1,ymm7    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...cf

  0x000001f5ca2fa366: vextracti128 xmm7,ymm1,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...cf
                                                ;...01

  0x000001f5ca2fa36c: vpaddd  xmm1,xmm1,xmm7    ;...c5
                                                ;...f1
                                                ;...fe
                                                ;...cf

  0x000001f5ca2fa370: vmovd   xmm7,r10d         ;...c4
                                                ;...c1
                                                ;...79
                                                ;...6e
                                                ;...fa

  0x000001f5ca2fa375: vpaddd  xmm7,xmm7,xmm1    ;...c5
                                                ;...c1
                                                ;...fe
                                                ;...f9

  0x000001f5ca2fa379: vmovd   ebx,xmm7          ;...c5
                                                ;...f9
                                                ;...7e
                                                ;...fb
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@31 (line 65)

  0x000001f5ca2fa37d: add     r14d,40h          ;...41
                                                ;...83
                                                ;...c6
                                                ;...40
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@33 (line 64)

  0x000001f5ca2fa381: cmp     r14d,r9d          ;...45
                                                ;...3b
                                                ;...f1

  0x000001f5ca2fa384: jl      1f5ca2fa200h      ;...0f
                                                ;...8c
                                                ;...76
                                                ;...fe
                                                ;...ff
                                                ;...ff
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@20 (line 64)

  0x000001f5ca2fa38a: mov     r11d,edi          ;...44
                                                ;...8b
                                                ;...df

  0x000001f5ca2fa38d: add     r11d,0fffffff9h   ;...41
                                                ;...83
                                                ;...c3
                                                ;...f9

  0x000001f5ca2fa391: cmp     edi,r11d          ;...41
                                                ;...3b
                                                ;...fb

  0x000001f5ca2fa394: cmovl   r11d,r8d          ;...45
                                                ;...0f
                                                ;...4c
                                                ;...d8

  0x000001f5ca2fa398: cmp     r14d,r11d         ;...45
                                                ;...3b
                                                ;...f3

  0x000001f5ca2fa39b: jnl     1f5ca2fa3d5h      ;...7d
                                                ;...38

  0x000001f5ca2fa39d: nop                       ;...66
                                                ;...66
                                                ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@29 (line 65)

  0x000001f5ca2fa3a0: vmovdqu ymm0,ymmword ptr [r13+r14*4+10h]
                                                ;...c4
                                                ;...81
                                                ;...7e
                                                ;...6f
                                                ;...44
                                                ;...b5
                                                ;...10

  0x000001f5ca2fa3a7: vpmulld ymm1,ymm0,ymm2    ;...c4
                                                ;...e2
                                                ;...7d
                                                ;...40
                                                ;...ca

  0x000001f5ca2fa3ac: vphaddd ymm3,ymm1,ymm1    ;...c4
                                                ;...e2
                                                ;...75
                                                ;...02
                                                ;...d9

  0x000001f5ca2fa3b1: vphaddd ymm3,ymm3,ymm0    ;...c4
                                                ;...e2
                                                ;...65
                                                ;...02
                                                ;...d8

  0x000001f5ca2fa3b6: vextracti128 xmm0,ymm3,1h  ;...c4
                                                ;...e3
                                                ;...7d
                                                ;...39
                                                ;...d8
                                                ;...01

  0x000001f5ca2fa3bc: vpaddd  xmm3,xmm3,xmm0    ;...c5
                                                ;...e1
                                                ;...fe
                                                ;...d8

  0x000001f5ca2fa3c0: vmovd   xmm0,ebx          ;...c5
                                                ;...f9
                                                ;...6e
                                                ;...c3

  0x000001f5ca2fa3c4: vpaddd  xmm0,xmm0,xmm3    ;...c5
                                                ;...f9
                                                ;...fe
                                                ;...c3

  0x000001f5ca2fa3c8: vmovd   ebx,xmm0          ;...c5
                                                ;...f9
                                                ;...7e
                                                ;...c3
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@31 (line 65)

  0x000001f5ca2fa3cc: add     r14d,8h           ;...41
                                                ;...83
                                                ;...c6
                                                ;...08
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@33 (line 64)

  0x000001f5ca2fa3d0: cmp     r14d,r11d         ;...45
                                                ;...3b
                                                ;...f3

  0x000001f5ca2fa3d3: jl      1f5ca2fa3a0h      ;...7c
                                                ;...cb
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@20 (line 64)

  0x000001f5ca2fa3d5: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa3d8: jnl     1f5ca2fa3f0h      ;...7d
                                                ;...16

  0x000001f5ca2fa3da: nop                       ;...66
                                                ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@29 (line 65)

  0x000001f5ca2fa3dc: mov     r10d,ebp          ;...44
                                                ;...8b
                                                ;...d5

  0x000001f5ca2fa3df: imul    r10d,dword ptr [r13+r14*4+10h]
                                                ;...47
                                                ;...0f
                                                ;...af
                                                ;...54
                                                ;...b5
                                                ;...10

  0x000001f5ca2fa3e5: add     ebx,r10d          ;...41
                                                ;...03
                                                ;...da
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@31 (line 65)

  0x000001f5ca2fa3e8: inc     r14d              ;...41
                                                ;...ff
                                                ;...c6
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@33 (line 64)

  0x000001f5ca2fa3eb: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa3ee: jl      1f5ca2fa3dch      ;...7c
                                                ;...ec
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@20 (line 64)

  0x000001f5ca2fa3f0: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa3f3: jnl     1f5ca2fa411h      ;...7d
                                                ;...1c

  0x000001f5ca2fa3f5: nop                       ;...66
                                                ;...66
                                                ;...90
                                                ;*iaload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@29 (line 65)

  0x000001f5ca2fa3f8: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa3fb: jnb     1f5ca2fa428h      ;...73
                                                ;...2b

  0x000001f5ca2fa3fd: mov     r11d,ebp          ;...44
                                                ;...8b
                                                ;...dd

  0x000001f5ca2fa400: imul    r11d,dword ptr [r13+r14*4+10h]
                                                ;...47
                                                ;...0f
                                                ;...af
                                                ;...5c
                                                ;...b5
                                                ;...10

  0x000001f5ca2fa406: add     ebx,r11d          ;...41
                                                ;...03
                                                ;...db
                                                ;*iadd {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@31 (line 65)

  0x000001f5ca2fa409: inc     r14d              ;...41
                                                ;...ff
                                                ;...c6
                                                ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@33 (line 64)

  0x000001f5ca2fa40c: cmp     r14d,edi          ;...44
                                                ;...3b
                                                ;...f7

  0x000001f5ca2fa40f: jl      1f5ca2fa3f8h      ;...7c
                                                ;...e7
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.scale.Scale::Scale_Int_Dynamic@16 (line 64)

  0x000001f5ca2fa411: mov     eax,ebx           ;...8b
                                                ;...c3

  0x000001f5ca2fa413: vzeroupper                ;...c5
                                                ;...f8
                                                ;...77

  0x000001f5ca2fa416: add     rsp,50h           ;...48
                                                ;...83
                                                ;...c4
                                                ;...50

  0x000001f5ca2fa41a: pop     rbp               ;...5d

  0x000001f5ca2fa41b: test    dword ptr [1f5b68c0000h],eax
                                                ;...85
                                                ;...05
                                                ;...df
                                                ;...5b
                                                ;...5c
                                                ;...ec
                                                ;   {poll_return}
  0x000001f5ca2fa421: ret                       ;...c3

There are two lessons to be learnt here. The first is that what you see is not what you get. The second is about the correctness of asymptotic analysis. If hierarchical cache renders asymptotic analysis bullshit (linear time but cache friendly algorithms can, and do, outperform logarithmic algorithms with cache misses), optimising compilers render the field practically irrelevant.

Posted on

Zeroing Negative Values in Arrays Efficiently

Replacing negatives with zeroes in large arrays of values is a primitive function of several complex financial risk measures, including potential future exposure (PFE) and the liquidity coverage ratio (LCR). While this is not an interesting operation by any stretch of the imagination, it is useful and there is significant benefit in its performance. This is an operation that can be computed very efficiently using the instruction VMAXPD. For Intel Xeon processors, this instruction requires half a cycle to calculate and has a latency (how long before another instruction can use its result) of four cycles. There is currently no way to trick Java into using this instruction for this simple operation, though there is a placeholder implementation on the current DoubleVector prototype in Project Panama which may do so.

C++ Intel Intrinsics

It’s possible to target instructions from different processor vendors, in my case Intel, by using intrinsic functions which expose instructions as high level functions. The code looks incredibly ugly but it works. Here is a C++ function for 256 bit ymm registers:


void zero_negatives(const double* source, double* target, const size_t length) {
  for (size_t i = 0; i + 3 < length; i += 4) {
    __m256d vector = _mm256_load_pd(source + i);
    __m256d zeroed = _mm256_max_pd(vector, _mm256_setzero_pd());
    _mm256_storeu_pd(target + i, zeroed);
  }
}

The function loads doubles into 256 bit vectors, within each vector replaces the negative values with zero, and writes them back into an array. It generates the following assembly code (which, incidentally, is less of a shit show to access than in Java):


void zero_negatives(const double* source, double* target, const size_t length) {
00007FF746EE5110  mov         qword ptr [rsp+18h],r8  
00007FF746EE5115  mov         qword ptr [rsp+10h],rdx  
00007FF746EE511A  mov         qword ptr [rsp+8],rcx  
00007FF746EE511F  push        r13  
00007FF746EE5121  push        rbp  
00007FF746EE5122  push        rdi  
00007FF746EE5123  sub         rsp,250h  
00007FF746EE512A  mov         r13,rsp  
00007FF746EE512D  lea         rbp,[rsp+20h]  
00007FF746EE5132  and         rbp,0FFFFFFFFFFFFFFE0h  
00007FF746EE5136  mov         rdi,rsp  
00007FF746EE5139  mov         ecx,94h  
00007FF746EE513E  mov         eax,0CCCCCCCCh  
00007FF746EE5143  rep stos    dword ptr [rdi]  
00007FF746EE5145  mov         rcx,qword ptr [rsp+278h]  
  for (size_t i = 0; i + 3 < length; i += 4) {
00007FF746EE514D  mov         qword ptr [rbp+8],0  
00007FF746EE5155  jmp         zero_negatives+53h (07FF746EE5163h)  
00007FF746EE5157  mov         rax,qword ptr [rbp+8]  
00007FF746EE515B  add         rax,4  
00007FF746EE515F  mov         qword ptr [rbp+8],rax  
00007FF746EE5163  mov         rax,qword ptr [rbp+8]  
00007FF746EE5167  add         rax,3  
00007FF746EE516B  cmp         rax,qword ptr [length]  
00007FF746EE5172  jae         zero_negatives+0DDh (07FF746EE51EDh)  
    __m256d vector = _mm256_load_pd(source + i);
00007FF746EE5174  mov         rax,qword ptr [source]  
00007FF746EE517B  mov         rcx,qword ptr [rbp+8]  
00007FF746EE517F  lea         rax,[rax+rcx*8]  
00007FF746EE5183  vmovupd     ymm0,ymmword ptr [rax]  
00007FF746EE5187  vmovupd     ymmword ptr [rbp+180h],ymm0  
00007FF746EE518F  vmovupd     ymm0,ymmword ptr [rbp+180h]  
00007FF746EE5197  vmovupd     ymmword ptr [rbp+40h],ymm0  
    __m256d zeroed = _mm256_max_pd(vector, _mm256_setzero_pd());
00007FF746EE519C  vxorpd      xmm0,xmm0,xmm0  
00007FF746EE51A0  vmovupd     ymmword ptr [rbp+200h],ymm0  
00007FF746EE51A8  vmovupd     ymm0,ymmword ptr [rbp+40h]  
00007FF746EE51AD  vmaxpd      ymm0,ymm0,ymmword ptr [rbp+200h]  
00007FF746EE51B5  vmovupd     ymmword ptr [rbp+1C0h],ymm0  
00007FF746EE51BD  vmovupd     ymm0,ymmword ptr [rbp+1C0h]  
00007FF746EE51C5  vmovupd     ymmword ptr [rbp+80h],ymm0  
    _mm256_storeu_pd(target + i, zeroed);
00007FF746EE51CD  mov         rax,qword ptr [target]  
00007FF746EE51D4  mov         rcx,qword ptr [rbp+8]  
00007FF746EE51D8  lea         rax,[rax+rcx*8]  
00007FF746EE51DC  vmovupd     ymm0,ymmword ptr [rbp+80h]  
00007FF746EE51E4  vmovupd     ymmword ptr [rax],ymm0  
  }
00007FF746EE51E8  jmp         zero_negatives+47h (07FF746EE5157h)  
}
00007FF746EE51ED  lea         rsp,[r13+250h]  
00007FF746EE51F4  pop         rdi  
00007FF746EE51F5  pop         rbp  
00007FF746EE51F6  pop         r13  
00007FF746EE51F8  ret    

This code is noticeably fast. I measured the throughput averaged over 1000 iterations, with an array of 10 million doubles (800MB) uniformly distributed between +/- 1E7, to quantify the throughput in GB/s and iterations/s. This code does between 4.5 and 5 iterations per second, which translates to processing approximately 4GB/s. This seems high, and since I am unaware of best practices in C++, if the measurement is flawed, I would gratefully be educated in the comments.


void benchmark() {
  const size_t length = 1E8;
  double* values = new double[length];
  fill_array(values, length);
  double* zeroed = new double[length];
  auto start = std::chrono::high_resolution_clock::now();
  int iterations = 1000;
  for (int i = 0; i < iterations; ++i) {
    zero_negatives(values, zeroed, length);
  }
  auto end = std::chrono::high_resolution_clock::now();
  auto nanos = std::chrono::duration_cast<std::chrono::nanoseconds>(end - start).count();
  double thrpt_s = (iterations * 1E9) / nanos;
  double thrpt_gbps = (thrpt_s * sizeof(double) * length) / 1E9;
  std::cout << thrpt_s << "/s" << std::endl;
  std::cout << thrpt_gbps << "GB/s" << std::endl;
  delete[] values;
  delete[] zeroed;
}

While I am sure there are various ways an expert could tweak this for performance, this code can’t get much faster unless there are 512 bit zmm registers available, in which case it would be wasteful. While the code looks virtually the same for AVX512 (just replace “256” with “512”), portability and efficiency are at odds. Handling the mess of detecting the best instruction set for the deployed architecture is the main reason for using Java in performance sensitive (but not critical) applications. But this is not the code the JVM generates.

Java Auto-Vectorisation (Play Your Cards Right)

There is currently no abstraction modelling vectorisation in Java. The only access available is if the compiler engineers implement an intrinsic, or auto-vectorisation, which will try, and sometimes succeed admirably, to translate your code to a good vector implementation. There is currently a prototype project for explicit vectorisation in Project Panama. There are a few ways to skin this cat, and it’s worth looking at the code they generate and the throughput available from each approach.

There is a choice between copying the array and zeroing out the negatives, and allocating a new array and only writing the non-negative values. There is another choice between an if statement and branchless code using Math.max. This results in the following four implementations which I measure on comparable data to the C++ benchmark (10 million doubles, normally distributed with mean zero). To be fair to the Java code, as in the C++ benchmarks, the cost of allocation is isolated by writing into an array pre-allocated once per benchmark. This penalises the approaches where the array is copied first and then zeroed wherever the value is negative. The code is online at github.


    @Benchmark
    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    public double[] BranchyCopyAndMask(ArrayWithNegatives state) {
        double[] data = state.data;
        double[] result = state.target;
        System.arraycopy(data, 0, result, 0, data.length);
        for (int i = 0; i < result.length; ++i) {
            if (result[i] < 0D) {
                result[i] = 0D;
            }
        }
        return result;
    }

    @Benchmark
    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    public double[] BranchyNewArray(ArrayWithNegatives state) {
        double[] data = state.data;
        double[] result = state.target;
        for (int i = 0; i < result.length; ++i) {
            result[i] = data[i] < 0D ? 0D : data[i];
        }
        return result;
    }

    @Benchmark
    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    public double[] NewArray(ArrayWithNegatives state) {
        double[] data = state.data;
        double[] result = state.target;
        for (int i = 0; i < result.length; ++i) {
            result[i] = Math.max(data[i], 0D);
        }
        return result;
    }

    @Benchmark
    @CompilerControl(CompilerControl.Mode.DONT_INLINE)
    public double[] CopyAndMask(ArrayWithNegatives state) {
        double[] data = state.data;
        double[] result = state.target;
        System.arraycopy(data, 0, result, 0, data.length);
        for (int i = 0; i < result.length; ++i) {
            result[i] = Math.max(result[i], 0D);
        }
        return result;
    }

None of these implementations comes close to the native code above. The best implementation performs 1.8 iterations per second which equates to processing approximately 1.4GB/s, vastly inferior to the 4GB/s achieved with Intel intrinsics. The results are below:

Benchmark Mode Threads Samples Score Score Error (99.9%) Unit
BranchyCopyAndMask thrpt 1 10 1.314845 0.061662 ops/s
BranchyNewArray thrpt 1 10 1.802673 0.061835 ops/s
CopyAndMask thrpt 1 10 1.146630 0.018903 ops/s
NewArray thrpt 1 10 1.357020 0.116481 ops/s

As an aside, there is a very interesting observation to make, worthy of its own post: if the array consists only of positive values, the “branchy” implementations run very well, at speeds comparable to the zero_negatives (when it ran with 50% negatives). The ratio of branch hits to misses is an orthogonal explanatory variable, and the input data, while I often don’t think about it enough, is very important.

I only looked at the assembly emitted for the fastest version (BranchyNewArray) and it doesn’t look anything like zero_negatives, though it does use some vectorisation – as pointed out by Daniel Lemire in the comments, this code has probably not been vectorised and is probably using SSE2 (indeed only quad words are loaded into 128 bit registers):

com/openkappa/simd/positive/PositiveValues.BranchyNewArray(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D  [0x000002ae309c3ce0, 0x000002ae309c3ff8]  792 bytes
Argument 0 is unknown.RIP: 0x2ae309c3ce0 Code size: 0x00000318
[Entry Point]
[Constants]
  # {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues'
  0x000002ae309c3ce0: mov     r10d,dword ptr [rdx+8h]  ;...44
                                                ;...8b
                                                ;...52
                                                ;...08

  0x000002ae309c3ce4: shl     r10,3h            ;...49
                                                ;...c1
                                                ;...e2
                                                ;...03

  0x000002ae309c3ce8: cmp     r10,rax           ;...4c
                                                ;...3b
                                                ;...d0

  0x000002ae309c3ceb: jne     2ae3042c200h      ;...0f
                                                ;...85
                                                ;...0f
                                                ;...85
                                                ;...a6
                                                ;...ff
                                                ;   {runtime_call ic_miss_stub}
  0x000002ae309c3cf1: nop     word ptr [rax+rax+0h]  ;...66
                                                ;...66
                                                ;...66
                                                ;...0f
                                                ;...1f
                                                ;...84
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3cfc: nop                       ;...66
                                                ;...66
                                                ;...66
                                                ;...90

[Verified Entry Point]
  0x000002ae309c3d00: mov     dword ptr [rsp+0ffffffffffff9000h],eax
                                                ;...89
                                                ;...84
                                                ;...24
                                                ;...00
                                                ;...90
                                                ;...ff
                                                ;...ff

  0x000002ae309c3d07: push    rbp               ;...55

  0x000002ae309c3d08: sub     rsp,60h           ;...48
                                                ;...83
                                                ;...ec
                                                ;...60

  0x000002ae309c3d0c: mov     rcx,2ae4d163880h  ;...48
                                                ;...b9
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3d16: mov     esi,dword ptr [rcx+0fch]
                                                ;...8b
                                                ;...b1
                                                ;...fc
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d1c: add     esi,8h            ;...83
                                                ;...c6
                                                ;...08

  0x000002ae309c3d1f: mov     dword ptr [rcx+0fch],esi
                                                ;...89
                                                ;...b1
                                                ;...fc
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d25: and     esi,1ff8h         ;...81
                                                ;...e6
                                                ;...f8
                                                ;...1f
                                                ;...00
                                                ;...00

  0x000002ae309c3d2b: cmp     esi,0h            ;...83
                                                ;...fe
                                                ;...00

  0x000002ae309c3d2e: je      2ae309c3ec1h      ;...0f
                                                ;...84
                                                ;...8d
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;*aload_1 {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@0 (line 29)

  0x000002ae309c3d34: mov     edx,dword ptr [r8+0ch]  ;...41
                                                ;...8b
                                                ;...50
                                                ;...0c
                                                ; implicit exception: dispatches to 0x000002ae309c3ee2
  0x000002ae309c3d38: shl     rdx,3h            ;...48
                                                ;...c1
                                                ;...e2
                                                ;...03
                                                ;*getfield data {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@1 (line 29)

  0x000002ae309c3d3c: mov     ecx,dword ptr [r8+10h]  ;...41
                                                ;...8b
                                                ;...48
                                                ;...10

  0x000002ae309c3d40: shl     rcx,3h            ;...48
                                                ;...c1
                                                ;...e1
                                                ;...03
                                                ;*getfield target {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@6 (line 30)

  0x000002ae309c3d44: mov     esi,0h            ;...be
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d49: jmp     2ae309c3e27h      ;...e9
                                                ;...d9
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;*iload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@13 (line 31)

  0x000002ae309c3d4e: nop                       ;...66
                                                ;...90

  0x000002ae309c3d50: movsxd  rax,esi           ;...48
                                                ;...63
                                                ;...c6

  0x000002ae309c3d53: cmp     esi,dword ptr [rdx+0ch]  ;...3b
                                                ;...72
                                                ;...0c
                                                ; implicit exception: dispatches to 0x000002ae309c3ee7
  0x000002ae309c3d56: jnb     2ae309c3ef1h      ;...0f
                                                ;...83
                                                ;...95
                                                ;...01
                                                ;...00
                                                ;...00

  0x000002ae309c3d5c: vmovsd  xmm0,qword ptr [rdx+rax*8+10h]
                                                ;...c5
                                                ;...fb
                                                ;...10
                                                ;...44
                                                ;...c2
                                                ;...10
                                                ;*daload {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@26 (line 32)

  0x000002ae309c3d62: vxorpd  xmm1,xmm1,xmm1    ;...c5
                                                ;...f1
                                                ;...57
                                                ;...c9

  0x000002ae309c3d66: vucomisd xmm0,xmm1        ;...c5
                                                ;...f9
                                                ;...2e
                                                ;...c1

  0x000002ae309c3d6a: mov     eax,1h            ;...b8
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d6f: jp      2ae309c3d88h      ;...0f
                                                ;...8a
                                                ;...13
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d75: jnbe    2ae309c3d88h      ;...0f
                                                ;...87
                                                ;...0d
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d7b: mov     eax,0h            ;...b8
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d80: je      2ae309c3d88h      ;...0f
                                                ;...84
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d86: dec     eax               ;...ff
                                                ;...c8
                                                ;*dcmpg {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@28 (line 32)

  0x000002ae309c3d88: cmp     eax,0h            ;...83
                                                ;...f8
                                                ;...00

  0x000002ae309c3d8b: mov     rax,2ae4d163880h  ;...48
                                                ;...b8
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3d95: mov     rdi,158h          ;...48
                                                ;...bf
                                                ;...58
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3d9f: jnl     2ae309c3dafh      ;...0f
                                                ;...8d
                                                ;...0a
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3da5: mov     rdi,168h          ;...48
                                                ;...bf
                                                ;...68
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3daf: mov     rbx,qword ptr [rax+rdi]  ;...48
                                                ;...8b
                                                ;...1c
                                                ;...38

  0x000002ae309c3db3: lea     rbx,[rbx+1h]      ;...48
                                                ;...8d
                                                ;...5b
                                                ;...01

  0x000002ae309c3db7: mov     qword ptr [rax+rdi],rbx  ;...48
                                                ;...89
                                                ;...1c
                                                ;...38

  0x000002ae309c3dbb: jnl     2ae309c3dd5h      ;...0f
                                                ;...8d
                                                ;...14
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;*ifge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@29 (line 32)

  0x000002ae309c3dc1: mov     rax,2ae4d163880h  ;...48
                                                ;...b8
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3dcb: inc     dword ptr [rax+178h]  ;...ff
                                                ;...80
                                                ;...78
                                                ;...01
                                                ;...00
                                                ;...00

  0x000002ae309c3dd1: vxorpd  xmm0,xmm0,xmm0    ;...c5
                                                ;...f9
                                                ;...57
                                                ;...c0
                                                ;*goto {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@33 (line 32)

  0x000002ae309c3dd5: movsxd  rax,esi           ;...48
                                                ;...63
                                                ;...c6

  0x000002ae309c3dd8: cmp     esi,dword ptr [rcx+0ch]  ;...3b
                                                ;...71
                                                ;...0c

  0x000002ae309c3ddb: jnb     2ae309c3efah      ;...0f
                                                ;...83
                                                ;...19
                                                ;...01
                                                ;...00
                                                ;...00

  0x000002ae309c3de1: vmovsd  qword ptr [rcx+rax*8+10h],xmm0
                                                ;...c5
                                                ;...fb
                                                ;...11
                                                ;...44
                                                ;...c1
                                                ;...10
                                                ;*dastore {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@40 (line 32)

  0x000002ae309c3de7: inc     esi               ;...ff
                                                ;...c6

  0x000002ae309c3de9: mov     rax,2ae4d163880h  ;...48
                                                ;...b8
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3df3: mov     edi,dword ptr [rax+100h]
                                                ;...8b
                                                ;...b8
                                                ;...00
                                                ;...01
                                                ;...00
                                                ;...00

  0x000002ae309c3df9: add     edi,8h            ;...83
                                                ;...c7
                                                ;...08

  0x000002ae309c3dfc: mov     dword ptr [rax+100h],edi
                                                ;...89
                                                ;...b8
                                                ;...00
                                                ;...01
                                                ;...00
                                                ;...00

  0x000002ae309c3e02: and     edi,0fff8h        ;...81
                                                ;...e7
                                                ;...f8
                                                ;...ff
                                                ;...00
                                                ;...00

  0x000002ae309c3e08: cmp     edi,0h            ;...83
                                                ;...ff
                                                ;...00

  0x000002ae309c3e0b: je      2ae309c3f03h      ;...0f
                                                ;...84
                                                ;...f2
                                                ;...00
                                                ;...00
                                                ;...00
                                                ; ImmutableOopMap{rcx=Oop rdx=Oop }
                                                ;*goto {reexecute=1 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@44 (line 31)

  0x000002ae309c3e11: test    dword ptr [2ae24520000h],eax
                                                ;...85
                                                ;...05
                                                ;...e9
                                                ;...c1
                                                ;...b5
                                                ;...f3
                                                ;   {poll}
  0x000002ae309c3e17: mov     rax,2ae4d163880h  ;...48
                                                ;...b8
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3e21: inc     dword ptr [rax+190h]  ;...ff
                                                ;...80
                                                ;...90
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;*goto {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@44 (line 31)

  0x000002ae309c3e27: mov     eax,dword ptr [rcx+0ch]  ;...8b
                                                ;...41
                                                ;...0c
                                                ;*arraylength {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@16 (line 31)
                                                ; implicit exception: dispatches to 0x000002ae309c3f24
  0x000002ae309c3e2a: cmp     esi,eax           ;...3b
                                                ;...f0

  0x000002ae309c3e2c: mov     rax,2ae4d163880h  ;...48
                                                ;...b8
                                                ;...80
                                                ;...38
                                                ;...16
                                                ;...4d
                                                ;...ae
                                                ;...02
                                                ;...00
                                                ;...00
                                                ;   {metadata(method data for {method} {0x000002ae4d13ec70} 'BranchyNewArray' '(Lcom/openkappa/simd/positive/ArrayWithNegatives;)[D' in 'com/openkappa/simd/positive/PositiveValues')}
  0x000002ae309c3e36: mov     rdi,148h          ;...48
                                                ;...bf
                                                ;...48
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3e40: jl      2ae309c3e50h      ;...0f
                                                ;...8c
                                                ;...0a
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3e46: mov     rdi,138h          ;...48
                                                ;...bf
                                                ;...38
                                                ;...01
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00
                                                ;...00

  0x000002ae309c3e50: mov     rbx,qword ptr [rax+rdi]  ;...48
                                                ;...8b
                                                ;...1c
                                                ;...38

  0x000002ae309c3e54: lea     rbx,[rbx+1h]      ;...48
                                                ;...8d
                                                ;...5b
                                                ;...01

  0x000002ae309c3e58: mov     qword ptr [rax+rdi],rbx  ;...48
                                                ;...89
                                                ;...1c
                                                ;...38

  0x000002ae309c3e5c: jl      2ae309c3d50h      ;...0f
                                                ;...8c
                                                ;...ee
                                                ;...fe
                                                ;...ff
                                                ;...ff
                                                ;*if_icmpge {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@17 (line 31)

  0x000002ae309c3e62: mov     rax,rcx           ;...48
                                                ;...8b
                                                ;...c1

  0x000002ae309c3e65: add     rsp,60h           ;...48
                                                ;...83
                                                ;...c4
                                                ;...60

  0x000002ae309c3e69: pop     rbp               ;...5d

  0x000002ae309c3e6a: test    dword ptr [2ae24520000h],eax
                                                ;...85
                                                ;...05
                                                ;...90
                                                ;...c1
                                                ;...b5
                                                ;...f3
                                                ;   {poll_return}
  0x000002ae309c3e70: ret                       ;...c3
                                                ;*areturn {reexecute=0 rethrow=0 return_oop=0}
                                                ; - com.openkappa.simd.positive.PositiveValues::BranchyNewArray@48 (line 34)

I don’t really understand, and haven’t thought about, the intent of the emitted code, but it makes extensive use of the instruction VUCOMISD for comparisons with zero, which has a lower latency but lower throughput than VMAXPD. It would certainly be interesting to see how Project Panama does this. Perhaps this should just be made available as a fail-safe intrinsic like Arrays.mismatch?

Posted on