#### Confusing Sets and Lists

I have often seen the roles of lists and sets confused. An application can be brought to its knees – that is, cease to deliver commercial value – if `List.contains`

is called frequently enough on big enough lists. And then there is the workaround… When I moved over to Java from C++ several years ago, it seemed utterly crazy that there was even a `Collection`

interface – *exactly* what Scott Meier’s *Effective STL* said not to do. I still think it’s crazy. Sets and lists cannot be substituted, and when you add up the marginal costs, as well as the costs of any compensatory workarounds, confusing them is responsible for a lot of performance bugs. As an application developer, it is part of your job to choose. Here are a few simple examples of when to use each collection type.

### Contains

Is an element in the collection?

Never ever do this with a `List`

. This operation is often referred to as being O(n). Which means in your worst case will touch every element in the list (technically, at least once). You have a choice between `HashSet`

and a `TreeSet`

, and both have costs and benefits.

If you choose a `HashSet`

, your best case is O(1): you evaluate a hash code, take its modulus with respect to the size of an array, and look up a bucket containing only one element. The worst case occurs with a degenerate hash code which maps all elements to the same bucket. This is again O(n): you probe a linked list testing each element for equality. On average you get something between these two cases and it depends on the uniformity of your hash code implementation.

If you choose a `TreeSet`

you get a worst case O(log n): this is effectively just a binary search through the nodes in a red black tree. Performance is limited by the cost of the comparator, and suffers systematically from cache misses for large sets (like any kind of pointer chasing, branch prediction and prefetching is difficult if not impossible).

If you’re working with numbers, and small to medium collections, a sorted primitive array may be the best approach, so long as it fits in cache. If you’re working with integers, you can do this in constant time in the worst case by using a `BitSet`

.

### Select

What is the value of the element at a given index with respect to a sort order?

This is an obvious use case for a `List`

: it’s O(1) – this is just a lookup at an array offset.

You couldn’t even write the code with a `HashSet`

without copying the data into an intermediate ordered structure, at which point you would probably think again. You see this sort of thing done in code written by inexpensive programmers at large outsourcing consultancies, who were perhaps just under unreasonable pressure to deliver to arbitrary deadlines.

`SortedSet`

, and anything conceptually similar, is the wrong data structure for this operation. The only way to compute this is O(n): you iterate through the set incrementing a counter until you reach the index, and then return the element you’ve iterated to. If you reach the end of the set, you throw. If you do this a lot, you’ll notice.

### Rank

How many predecessors does an element have with respect to a sort order?

Another classic operation for `List`

, so long as you keep it sorted. Use `Collections.binarySearch`

to find the index of the element in the collection with complexity O(log n). This is its rank. If you can get away with it, primitive arrays will be much better here, especially if they are small enough to fit in cache.

Once again, there are creativity points available for the solution involving a `HashSet`

, and they do exist in the wild, but a clean looking solution is at least *possible* with a `SortedSet`

. However, it involves an iteration with another check against an incrementing counter. It’s O(n) and if you do it a lot, you’ll blow your performance profile, so use a sorted list instead.

### What if you had the source code?

Is this fundamental or just a limitation of the Collections framework? Maybe if you had the source code you could just make these data structures optimal for every use case, without having to choose the right one? Not without creating a Frankenstein, and not without a *lot* of memory. Optimisation isn’t free.

#### Choosing the Right Radix: Measurement or Mathematics?

I recently wrote a post about radix sorting, and found that for large arrays of unsigned integers a handwritten implementation beats `Arrays.sort`

. However, I paid no attention to the choice of radix and used a default of eight bits. It turns out this was a lucky choice: modifying my benchmark to parametrise the radix, I observed a maximum at one byte, regardless of the size of array.

Is this an algorithmic or technical phenomenon? Is this something that could have been predicted on the back of an envelope without running a benchmark?

### Extended Benchmark Results

Size | Radix | Score | Score Error (99.9%) | Unit |
---|---|---|---|---|

100 | 2 | 135.559923 | 7.72397 | ops/ms |

100 | 4 | 262.854579 | 37.372678 | ops/ms |

100 | 8 | 345.038234 | 30.954927 | ops/ms |

100 | 16 | 7.717496 | 1.144967 | ops/ms |

1000 | 2 | 13.892142 | 1.522749 | ops/ms |

1000 | 4 | 27.712719 | 4.057162 | ops/ms |

1000 | 8 | 52.253497 | 4.761172 | ops/ms |

1000 | 16 | 7.656033 | 0.499627 | ops/ms |

10000 | 2 | 1.627354 | 0.186948 | ops/ms |

10000 | 4 | 3.620869 | 0.029128 | ops/ms |

10000 | 8 | 6.435789 | 0.610848 | ops/ms |

10000 | 16 | 3.703248 | 0.45177 | ops/ms |

100000 | 2 | 0.144575 | 0.014348 | ops/ms |

100000 | 4 | 0.281837 | 0.025707 | ops/ms |

100000 | 8 | 0.543274 | 0.031553 | ops/ms |

100000 | 16 | 0.533998 | 0.126949 | ops/ms |

1000000 | 2 | 0.011293 | 0.001429 | ops/ms |

1000000 | 4 | 0.021128 | 0.003137 | ops/ms |

1000000 | 8 | 0.037376 | 0.005783 | ops/ms |

1000000 | 16 | 0.031053 | 0.007987 | ops/ms |

### Modeling

To model the execution time of the algorithm, we can write , where is the length of the input array, and is the size in bits of the radix. We can inspect if the model predicts non-monotonic execution time with a minimum (corresponding to maximal throughput), or if increases indefinitely as a function of . If we find a plausible model predicting a minimum, temporarily treating as continuous, we can solve to find the theoretically optimal radix. This pre-supposes we derive a non-monotonic model.

### Constructing a Model

We need to write down an equation before we can do any calculus, which requires two dangerous assumptions.

- Each operation has the same cost, making the execution time proportional to the number of operation.
- The costs of operations do not vary as a function of or .

This means all we need to do is find a formula for the number of operations, and then vary and . The usual pitfall in this approach relates to the first assumption, in that memory accesses are modelled as uniform cost; memory access can vary widely in cost ranging from registers to RAM on another socket. We are about to fall foul of both assumptions constructing an intuitive model of the algorithm’s loop.

```
while (shift < Integer.SIZE) {
Arrays.fill(histogram, 0);
for (int i = 0; i < data.length; ++i) {
++histogram[((data[i] & mask) >> shift) + 1];
}
for (int i = 0; i < 1 << radix; ++i) {
histogram[i + 1] += histogram[i];
}
for (int i = 0; i < data.length; ++i) {
copy[histogram[(data[i] & mask) >> shift]++] = data[i];
}
for (int i = 0; i < data.length; ++i) {
data[i] = copy[i];
}
shift += radix;
mask <<= radix;
}
```

The outer loop depends on the choice of radix while the inner loops depend on the size of the array and the choice of radix. There are five obvious aspects to capture:

- The first inner loop takes time proportional to
- The third and fourth inner loops take time proportional to
- We can factor the per-element costs of loops 1, 3 and 4 into a constant
- The second inner loop takes time proportional to , modeled with by the term
- The body of the loop executes times

This can be summarised as the formula:

It was claimed the algorithm had linear complexity in and it only has a linear term in . Good. However, the exponential term in the numerator dominates the linear term in the denominator, making the function monotonic in . The model fails to predict the observed throughput maximising radix. *There are clearly much more complicated mechanisms at play than can be captured counting operations.*

#### Sorting Unsigned Integers Faster in Java

I discovered a curious resource for audio-visualising sort algorithms, which is exciting for two reasons. The first is that I finally feel like I understand Alexander Scriabin: he was not a composer. He discovered Tim Sort 80 years before Tim Peters and called it Black Mass. (If you aren’t familiar with the piece, fast-forward to 1:40 to hear the congruence.)

The second reason was that I noticed Radix Sort (LSD). While it was an affront to my senses, it used a mere 800 array accesses and *no comparisons*! I was unaware of this algorithm so delved deeper and implemented it for integers, and benchmarked my code against `Arrays.sort`

.

### Radix Sort

It is taken as given by many (myself included, or am I just projecting my thoughts on to others?) that is the best you can do in a sort algorithm. But this is actually only true for sort algorithms which depend on comparison. If you can afford to restrict the data types your sort algorithm supports to types with a positional interpretation (`java.util`

can’t because it needs to be ubiquitous and maintainable), you can get away with a linear time algorithm.

Radix sort, along with the closely related counting sort, does not use comparisons. Instead, the data is interpreted as a fixed length string of symbols. For each position, the cumulative histogram of symbols is computed to calculate sort indices. While the data needs to be scanned several times, the algorithm scales linearly and the overhead of the multiple scans is amortised for large arrays.

As you can see on Wikipedia, there are two kinds of radix sort: *Least Significant Digit* and *Most Significant Digit*. This dichotomy relates to the order the (representational) string of symbols is traversed in. I implemented and benchmarked the LSD version for integers.

### Implementation

The implementation interprets an integer as the concatenation of n bit string symbols of fixed size size 32/n. It performs n passes over the array, starting with the least significant bits, which it modifies in place. For each pass the data is scanned three times, in order to:

- Compute the cumulative histogram over the symbols in their natural sort order
- Copy the value with symbol k to the mth position in a buffer, where m is defined by the cumulative density of k.
- Copy the buffer back into the original array

The implementation, which won’t work unless the chunks are proper divisors of 32, is below. The bonus (or caveat) is that it automatically supports unsigned integers. The code could be modified slightly to work with signed integers at a performance cost.

```
import java.util.Arrays;
public class RadixSort {
private final int radix;
public RadixSort() {
this(Byte.SIZE);
}
public RadixSort(int radix) {
assert 32 % radix== 0;
this.radix= radix;
}
public void sort(int[] data) {
int[] histogram = new int[(1 << radix) + 1];
int shift = 0;
int mask = (1 << radix) - 1;
int[] copy = new int[data.length];
while (shift < Integer.SIZE) {
Arrays.fill(histogram, 0);
for (int i = 0; i < data.length; ++i) {
++histogram[((data[i] & mask) >> shift) + 1];
}
for (int i = 0; i < 1 << radix; ++i) {
histogram[i + 1] += histogram[i];
}
for (int i = 0; i < data.length; ++i) {
copy[histogram[(data[i] & mask) >> shift]++] = data[i];
}
for (int i = 0; i < data.length; ++i) {
data[i] = copy[i];
}
shift += radix;
mask <<= radix;
}
}
}
```

The time complexity is obviously linear, a temporary buffer *is* allocated, but in comparison to `Arrays.sort`

it looks fairly spartan. Instinctively, cache locality looks fairly poor because the second inner loop of the three jumps all over the place. Will this implementation beat `Arrays.sort`

(for integers)?

### Benchmark

The algorithm is measured using arrays of random positive integers, for which both algorithms are equivalent, from a range of sizes. This isn’t always the best idea (the Tim Sort algorithm comes into its own on nearly sorted data), so take the result below with a pinch of salt. Care must be taken to copy the array in the benchmark since both algorithms are in-place.

```
public void launchBenchmark(String... jvmArgs) throws Exception {
Options opt = new OptionsBuilder()
.include(this.getClass().getName() + ".*")
.mode(Mode.SampleTime)
.mode(Mode.Throughput)
.timeUnit(TimeUnit.MILLISECONDS)
.measurementTime(TimeValue.seconds(10))
.warmupIterations(10)
.measurementIterations(10)
.forks(1)
.shouldFailOnError(true)
.shouldDoGC(true)
.jvmArgs(jvmArgs)
.resultFormat(ResultFormatType.CSV)
.build();
new Runner(opt).run();
}
@Benchmark
public void Arrays_Sort(Data data, Blackhole bh) {
int[] array = Arrays.copyOf(data.data, data.size);
Arrays.sort(array);
bh.consume(array);
}
@Benchmark
public void Radix_Sort(Data data, Blackhole bh) {
int[] array = Arrays.copyOf(data.data, data.size);
data.radixSort.sort(array);
bh.consume(array);
}
@State(Scope.Thread)
public static class Data {
@Param({"100", "1000", "10000", "100000", "1000000"})
int size;
int[] data;
RadixSort radixSort = new RadixSort();
@Setup(Level.Trial)
public void init() {
data = createArray(size);
}
}
public static int[] createArray(int size) {
int[] array = new int[size];
Random random = new Random(0);
for (int i = 0; i < size; ++i) {
array[i] = Math.abs(random.nextInt());
}
return array;
}
```

Benchmark | Mode | Threads | Samples | Score | Score Error (99.9%) | Unit | Param: size |
---|---|---|---|---|---|---|---|

Arrays_Sort | thrpt | 1 | 10 | 1304.687189 | 147.380334 | ops/ms | 100 |

Arrays_Sort | thrpt | 1 | 10 | 78.518664 | 9.339994 | ops/ms | 1000 |

Arrays_Sort | thrpt | 1 | 10 | 1.700208 | 0.091836 | ops/ms | 10000 |

Arrays_Sort | thrpt | 1 | 10 | 0.133835 | 0.007146 | ops/ms | 100000 |

Arrays_Sort | thrpt | 1 | 10 | 0.010560 | 0.000409 | ops/ms | 1000000 |

Radix_Sort | thrpt | 1 | 10 | 404.807772 | 24.930898 | ops/ms | 100 |

Radix_Sort | thrpt | 1 | 10 | 51.787409 | 4.881181 | ops/ms | 1000 |

Radix_Sort | thrpt | 1 | 10 | 6.065590 | 0.576709 | ops/ms | 10000 |

Radix_Sort | thrpt | 1 | 10 | 0.620338 | 0.068776 | ops/ms | 100000 |

Radix_Sort | thrpt | 1 | 10 | 0.043098 | 0.004481 | ops/ms | 1000000 |

Arrays_Sort | sample | 1 | 3088586 | 0.000902 | 0.000018 | ms/op | 100 |

Arrays_Sort·p0.00 | sample | 1 | 1 | 0.000394 | ms/op | 100 | |

Arrays_Sort·p0.50 | sample | 1 | 1 | 0.000790 | ms/op | 100 | |

Arrays_Sort·p0.90 | sample | 1 | 1 | 0.000791 | ms/op | 100 | |

Arrays_Sort·p0.95 | sample | 1 | 1 | 0.001186 | ms/op | 100 | |

Arrays_Sort·p0.99 | sample | 1 | 1 | 0.001974 | ms/op | 100 | |

Arrays_Sort·p0.999 | sample | 1 | 1 | 0.020128 | ms/op | 100 | |

Arrays_Sort·p0.9999 | sample | 1 | 1 | 0.084096 | ms/op | 100 | |

Arrays_Sort·p1.00 | sample | 1 | 1 | 4.096000 | ms/op | 100 | |

Arrays_Sort | sample | 1 | 2127794 | 0.011876 | 0.000037 | ms/op | 1000 |

Arrays_Sort·p0.00 | sample | 1 | 1 | 0.007896 | ms/op | 1000 | |

Arrays_Sort·p0.50 | sample | 1 | 1 | 0.009872 | ms/op | 1000 | |

Arrays_Sort·p0.90 | sample | 1 | 1 | 0.015408 | ms/op | 1000 | |

Arrays_Sort·p0.95 | sample | 1 | 1 | 0.024096 | ms/op | 1000 | |

Arrays_Sort·p0.99 | sample | 1 | 1 | 0.033920 | ms/op | 1000 | |

Arrays_Sort·p0.999 | sample | 1 | 1 | 0.061568 | ms/op | 1000 | |

Arrays_Sort·p0.9999 | sample | 1 | 1 | 0.894976 | ms/op | 1000 | |

Arrays_Sort·p1.00 | sample | 1 | 1 | 4.448256 | ms/op | 1000 | |

Arrays_Sort | sample | 1 | 168991 | 0.591169 | 0.001671 | ms/op | 10000 |

Arrays_Sort·p0.00 | sample | 1 | 1 | 0.483840 | ms/op | 10000 | |

Arrays_Sort·p0.50 | sample | 1 | 1 | 0.563200 | ms/op | 10000 | |

Arrays_Sort·p0.90 | sample | 1 | 1 | 0.707584 | ms/op | 10000 | |

Arrays_Sort·p0.95 | sample | 1 | 1 | 0.766976 | ms/op | 10000 | |

Arrays_Sort·p0.99 | sample | 1 | 1 | 0.942080 | ms/op | 10000 | |

Arrays_Sort·p0.999 | sample | 1 | 1 | 2.058273 | ms/op | 10000 | |

Arrays_Sort·p0.9999 | sample | 1 | 1 | 7.526102 | ms/op | 10000 | |

Arrays_Sort·p1.00 | sample | 1 | 1 | 46.333952 | ms/op | 10000 | |

Arrays_Sort | sample | 1 | 13027 | 7.670135 | 0.021512 | ms/op | 100000 |

Arrays_Sort·p0.00 | sample | 1 | 1 | 6.356992 | ms/op | 100000 | |

Arrays_Sort·p0.50 | sample | 1 | 1 | 7.634944 | ms/op | 100000 | |

Arrays_Sort·p0.90 | sample | 1 | 1 | 8.454144 | ms/op | 100000 | |

Arrays_Sort·p0.95 | sample | 1 | 1 | 8.742502 | ms/op | 100000 | |

Arrays_Sort·p0.99 | sample | 1 | 1 | 9.666560 | ms/op | 100000 | |

Arrays_Sort·p0.999 | sample | 1 | 1 | 12.916883 | ms/op | 100000 | |

Arrays_Sort·p0.9999 | sample | 1 | 1 | 28.037900 | ms/op | 100000 | |

Arrays_Sort·p1.00 | sample | 1 | 1 | 28.573696 | ms/op | 100000 | |

Arrays_Sort | sample | 1 | 1042 | 96.278673 | 0.603645 | ms/op | 1000000 |

Arrays_Sort·p0.00 | sample | 1 | 1 | 86.114304 | ms/op | 1000000 | |

Arrays_Sort·p0.50 | sample | 1 | 1 | 94.896128 | ms/op | 1000000 | |

Arrays_Sort·p0.90 | sample | 1 | 1 | 104.293990 | ms/op | 1000000 | |

Arrays_Sort·p0.95 | sample | 1 | 1 | 106.430464 | ms/op | 1000000 | |

Arrays_Sort·p0.99 | sample | 1 | 1 | 111.223767 | ms/op | 1000000 | |

Arrays_Sort·p0.999 | sample | 1 | 1 | 134.172770 | ms/op | 1000000 | |

Arrays_Sort·p0.9999 | sample | 1 | 1 | 134.742016 | ms/op | 1000000 | |

Arrays_Sort·p1.00 | sample | 1 | 1 | 134.742016 | ms/op | 1000000 | |

Radix_Sort | sample | 1 | 2240042 | 0.002941 | 0.000033 | ms/op | 100 |

Radix_Sort·p0.00 | sample | 1 | 1 | 0.001578 | ms/op | 100 | |

Radix_Sort·p0.50 | sample | 1 | 1 | 0.002368 | ms/op | 100 | |

Radix_Sort·p0.90 | sample | 1 | 1 | 0.003556 | ms/op | 100 | |

Radix_Sort·p0.95 | sample | 1 | 1 | 0.004344 | ms/op | 100 | |

Radix_Sort·p0.99 | sample | 1 | 1 | 0.011056 | ms/op | 100 | |

Radix_Sort·p0.999 | sample | 1 | 1 | 0.027232 | ms/op | 100 | |

Radix_Sort·p0.9999 | sample | 1 | 1 | 0.731127 | ms/op | 100 | |

Radix_Sort·p1.00 | sample | 1 | 1 | 5.660672 | ms/op | 100 | |

Radix_Sort | sample | 1 | 2695825 | 0.018553 | 0.000038 | ms/op | 1000 |

Radix_Sort·p0.00 | sample | 1 | 1 | 0.013424 | ms/op | 1000 | |

Radix_Sort·p0.50 | sample | 1 | 1 | 0.016576 | ms/op | 1000 | |

Radix_Sort·p0.90 | sample | 1 | 1 | 0.025280 | ms/op | 1000 | |

Radix_Sort·p0.95 | sample | 1 | 1 | 0.031200 | ms/op | 1000 | |

Radix_Sort·p0.99 | sample | 1 | 1 | 0.050944 | ms/op | 1000 | |

Radix_Sort·p0.999 | sample | 1 | 1 | 0.082944 | ms/op | 1000 | |

Radix_Sort·p0.9999 | sample | 1 | 1 | 0.830295 | ms/op | 1000 | |

Radix_Sort·p1.00 | sample | 1 | 1 | 6.660096 | ms/op | 1000 | |

Radix_Sort | sample | 1 | 685589 | 0.145695 | 0.000234 | ms/op | 10000 |

Radix_Sort·p0.00 | sample | 1 | 1 | 0.112512 | ms/op | 10000 | |

Radix_Sort·p0.50 | sample | 1 | 1 | 0.128000 | ms/op | 10000 | |

Radix_Sort·p0.90 | sample | 1 | 1 | 0.196608 | ms/op | 10000 | |

Radix_Sort·p0.95 | sample | 1 | 1 | 0.225792 | ms/op | 10000 | |

Radix_Sort·p0.99 | sample | 1 | 1 | 0.309248 | ms/op | 10000 | |

Radix_Sort·p0.999 | sample | 1 | 1 | 0.805888 | ms/op | 10000 | |

Radix_Sort·p0.9999 | sample | 1 | 1 | 1.818141 | ms/op | 10000 | |

Radix_Sort·p1.00 | sample | 1 | 1 | 14.401536 | ms/op | 10000 | |

Radix_Sort | sample | 1 | 60843 | 1.641961 | 0.005783 | ms/op | 100000 |

Radix_Sort·p0.00 | sample | 1 | 1 | 1.251328 | ms/op | 100000 | |

Radix_Sort·p0.50 | sample | 1 | 1 | 1.542144 | ms/op | 100000 | |

Radix_Sort·p0.90 | sample | 1 | 1 | 2.002944 | ms/op | 100000 | |

Radix_Sort·p0.95 | sample | 1 | 1 | 2.375680 | ms/op | 100000 | |

Radix_Sort·p0.99 | sample | 1 | 1 | 3.447030 | ms/op | 100000 | |

Radix_Sort·p0.999 | sample | 1 | 1 | 5.719294 | ms/op | 100000 | |

Radix_Sort·p0.9999 | sample | 1 | 1 | 8.724165 | ms/op | 100000 | |

Radix_Sort·p1.00 | sample | 1 | 1 | 13.074432 | ms/op | 100000 | |

Radix_Sort | sample | 1 | 4846 | 20.640787 | 0.260926 | ms/op | 1000000 |

Radix_Sort·p0.00 | sample | 1 | 1 | 14.893056 | ms/op | 1000000 | |

Radix_Sort·p0.50 | sample | 1 | 1 | 18.743296 | ms/op | 1000000 | |

Radix_Sort·p0.90 | sample | 1 | 1 | 26.673152 | ms/op | 1000000 | |

Radix_Sort·p0.95 | sample | 1 | 1 | 30.724915 | ms/op | 1000000 | |

Radix_Sort·p0.99 | sample | 1 | 1 | 40.470446 | ms/op | 1000000 | |

Radix_Sort·p0.999 | sample | 1 | 1 | 63.016600 | ms/op | 1000000 | |

Radix_Sort·p0.9999 | sample | 1 | 1 | 136.052736 | ms/op | 1000000 | |

Radix_Sort·p1.00 | sample | 1 | 1 | 136.052736 | ms/op | 1000000 |

The table tells an interesting story. `Arrays.sort`

is vastly superior for small arrays (the arrays most people have), but for large arrays the custom implementation comes into its own. Interestingly, this is consistent with the computer science. If you need to sort large arrays of (unsigned) integers and care about performance, think about implementing radix sort.

#### Microsecond Latency Rules Engine with RoaringBitmap

Implementing a rules engine can shorten development time and remove a lot of tedious if statements from your business logic. Unfortunately they are almost always slow and often bloated. Simple rules engines can be implemented by assigning integer salience to each line in a truth table, with rule resolution treated as an iterative intersection of ordered sets of integers. Implemented in terms of sorted sets, it would be remiss not to consider RoaringBitmap for the engine’s core. The code is at github.

### Classification Table and Syntax

This rules engine builds on the simple idea of a truth table usually used to teach predicate logic and computer hardware. Starting with a table and some attributes, interpreting one attribute as a classification, we get a list of rules. It is trivial to load such a table from a database. Since classifications can overlap, we prioritise by putting the rules we care about most – or the most salient rules – at the top of the table. When multiple rules match a fact, we take the last in the set ordered by salience. So we don’t always have to specify all of the attributes to get a classification, we can rank attributes by their importance left to right, where it’s required that all attributes to the left of a specified attribute are also specified when matching a fact against a rule set.

It’s possible to define rules containing wildcards. Wildcard rules will match any query (**warning**: if these are marked as high salience they will hide more specific rules with lower salience). It’s also possible to specify a prefix with a wildcard, which will match any query that matches at least the prefix.

Below is an example table consisting of rules for classification of regional English accents by phonetic feature.

thought | cloth | lot | palm | plant | bath | trap | accent |
---|---|---|---|---|---|---|---|

/ɔ/ | /ɒ/ | /ɑ/ | /ɑː/ | /ɑː/ | /ɑː/ | /æ/ | Received Pronunciation (UK) |

/ɔ/ | /ɔ/ | /ɑ/ | /ɑ/ | /æ/ | /æ/ | /æ/ | Georgian (US) |

/ɑ/ | /ɑ/ | /ɑ/ | /ɑ/ | /æ/ | /æ/ | /æ/ | Canadian |

* | * | /ɑ/ | /ɑ/ | /æ/ | /æ/ | /æ/ | North American |

* | * | * | * | * | * | /æ/ | Non Native |

* | * | * | * | * | * | * | French |

In the example above, the vowel sounds used in words differentiating speakers of several English accents are configured as a classification table. The accent column is the classification of any speaker exhibiting the properties specified in the six leftmost columns. UK Received Pronunciation is the most specific rule and has high salience, whereas various North American accents differ from RP in their use of short *A* vowels. A catch all for North American accents would wild card the sounds in *thought* and *caught* (contrast Boston pronunciations with Texas). So long as *trap* has been pronounced with a short *A* (which all English speakers do), and no other rule would recognise the sounds used in the first six words, the rule engine would conclude the speaker is using English as a second language. If not even the word trap is recognisable, then the speaker is probably unintelligible, or could be French.

### Implementation

A rule with a given salience can be represented by creating a bitmap index on salience by the attribute values of the rules. For instance, to store the rule `{foo, bar} -> 42`

, with salience 10, create a bitmap index on the first attribute of the rule, and set the 10th bit of the “foo” bitmap; likewise for the “bar” bitmap of the second index. Finding rules which match both attributes is a bitwise intersection, and since we rank by salience, the rule that wins is the first in the set. An obvious choice for fast ordered sets is RoaringBitmap.

RoaringBitmap consists of containers, which are fast, cache-friendly sorted sets of integers, and can contain up to 2^16 shorts. In RoaringBitmap, containers are indexed by keys consisting of the most significant 16 bits of the integer. For a rules engine, if you have more than 2^16 rules you have a much bigger problem anyway, so a container could index all the rules you could ever need, so RoaringBitmap itself would be overkill. While RoaringBitmap indexes containers by shorts (it does so for the sake of compression), we can implement wildcard and prefix matching by associating containers with Strings rather than shorts. As the core data structure of the rules engine, a RoaringBitmap *container* is placed at each node of an Apache commons PatriciaTrie. It’s really that simple – see the source at github.

When the rules engine is queried, a set consisting of all the rules that match is intersected with the container found at the node in the trie matching the value specified for each attribute. When more than one rule matches, the rule with the highest salience is accessed via the Container.first() method, one of the features I have contributed to RoaringBitmap. See example usage at github.

#### A Quick Look at RoaringBitmap

This article is an introduction to the data structures found in the RoaringBitmap library, which I have been making extensive use of recently. I wrote some time ago about the basic idea of bitmap indices, which are used in various databases and search engines, with the caveat that no traditional implementation is optimal across all data scenarios (in terms of size of the data set, sparsity, cardinalities of attributes and global sort orders of data sets with respect to specific attributes). RoaringBitmap is a dynamic data structure which aims to be that *one-size-fits-all* solution across all scenarios.

#### Containers

A RoaringBitmap should be thought of as a set of unsigned integers, consisting of containers which cover disjoint subsets. Each subset can contain values from a range of size 2^16, and the subset is indexed by a 16 bit key. This means that in the worst case it only takes 16 bits to represent a single 32 bit value, so unsigned 32 bit integers can be stored as Java shorts. The choice of container size also means that in the worst case, the container will still fit in L1 cache on a modern processor.

The implementation of the container covering a disjoint subset is free to vary between *RunContainer, BitmapContainer *and *ArrayContainer*, depending entirely on properties of the subset. When inserting data into a RoaringBitmap, it is decided whether to create a new container, or to mutate an existing container, depending on whether the values fit in the range covered by the container’s key. When performing a set operation, for instance by intersecting two bitmaps or computing their symmetric difference, a new RoaringBitmap is created by performing operations container by container, and it is decided dynamically which container implementation is best suited for the result. For cases where it is too difficult to determine the best implementation automatically, the method *runOptimize* is available to the programmer to make sure.

When querying a RoaringBitmap, the query can be executed container by container (which incidentally makes the query naturally parallelisable, but it hasn’t been done yet), and each pair from the cartesian product of combinations of container implementations must be implemented separately. This is manageable because there are only three implementations, and there won’t be any more. There is less work to do for symmetric operations, such as union and intersection, than with asymmetric operations such as contains.

#### RunContainer

When there are lots of clean words in a section of a bitmap, the best choice of container is run length encoding. The implementation of RunContainer is simple and very compact. It consists of an array of shorts (not ints, the most significant 16 bits are in the key) where the values at the even indices are the starts of runs, and the values at the odd indices are the lengths of the respective runs. Membership queries can be implemented simply using a binary search, and quantile queries can be implemented in constant time. Computing container cardinality requires a pass over the entire run array.

#### ArrayContainer

When data is sparse within a section of the bitmap, the best implementation is an array (*short[]). *For very sparse data, this isn’t theoretically optimal, but for most cases it is very good and the array for the container will fit in L1 cache for *mechanical sympathy*. Cardinality is very fast because it is precomputed, and operations would be fast in spite of their precise implementation by virtue of the small size of the set (that being said, the actual implementations *are* fast). Often when creating a new container, it is necessary to convert to a bitmap for better compression as the container fills up.

#### BitmapContainer

BitmapContainer is the classic implementation of a bitset. There is a fixed length *long[] *which should be interpreted bitwise, and a precomputed cardinality. Operations on BitmapContainers tend to be very fast, despite typically touching each element in the array, because they fit in L1 cache and make extensive use of Java intrinsics. If you find a method name in here and run your JVM on a reasonably modern processor, your code will quickly get optimised by the JVM, sometimes even to a single instruction. A much hackneyed example, explained better by Michael Barker *quite* *some time ago*, would be Long.bitCount, which translates to the single instruction *popcnt* and has various uses when operating on BitmapContainers. When intersecting with another container, the cardinality can only decrease or remain the same, so there is a chance an ArrayContainer will be produced.

#### Examples

There is a really nice Scala project on github which functions as a DSL for creating RoaringBitmaps – it allows you to create an *equality encoded* (see my previous bitmap index post) RoaringBitmap in a very fluid way. The project is here.

I have implemented bit slice indices, both equality and range encoded, in a data quality tool I am building. That project is hosted here. Below is an implementation of a range encoded bit slice index as an example of how to work with RoaringBitmaps.

```
public class RangeEncodedOptBitSliceIndex implements RoaringIndex {
private final int[] basis;
private final int[] cumulativeBasis;
private final RoaringBitmap[][] bitslice;
public RangeEncodedOptBitSliceIndex(ProjectionIndex projectionIndex, int[] basis) {
this.basis = basis;
this.cumulativeBasis = accumulateBasis(basis);
this.bitslice = BitSlices.createRangeEncodedBitSlice(projectionIndex, basis);
}
@Override
public RoaringBitmap whereEqual(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
int[] expansion = expand(code, cumulativeBasis);
for(int i = 0; i < cumulativeBasis.length; ++i) {
int component = expansion[i];
if(component == 0) {
result.and(bitslice[i][0]);
}
else if(component == basis[i] - 1) {
result.andNot(bitslice[i][basis[i] - 2]);
}
else {
result.and(FastAggregation.xor(bitslice[i][component], bitslice[i][component - 1]));
}
}
return result;
}
@Override
public RoaringBitmap whereNotEqual(int code, RoaringBitmap existence) {
RoaringBitmap inequality = existence.clone();
inequality.andNot(whereEqual(code, existence));
return inequality;
}
@Override
public RoaringBitmap whereLessThan(int code, RoaringBitmap existence) {
return whereLessThanOrEqual(code - 1, existence);
}
@Override
public RoaringBitmap whereLessThanOrEqual(int code, RoaringBitmap existence) {
final int[] expansion = expand(code, cumulativeBasis);
final int firstIndex = cumulativeBasis.length - 1;
int component = expansion[firstIndex];
int threshold = basis[firstIndex] - 1;
RoaringBitmap result = component < threshold ? bitslice[firstIndex][component].clone() : existence.clone(); for(int i = firstIndex - 1; i >= 0; --i) {
component = expansion[i];
threshold = basis[i] - 1;
if(component != threshold) {
result.and(bitslice[i][component]);
}
if(component != 0) {
result.or(bitslice[i][component - 1]);
}
}
return result;
}
@Override
public RoaringBitmap whereGreaterThan(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
result.andNot(whereLessThanOrEqual(code, existence));
return result;
}
@Override
public RoaringBitmap whereGreaterThanOrEqual(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
result.andNot(whereLessThan(code, existence));
return result;
}
}
```

#### Further Reading

The library has been implemented under an Apache License by several contributors, the most significant contributions coming from computer science researcher Daniel Lemire, who presented RoaringBitmap at Spark Summit 2017. The project site is here and the research paper behind the library is freely available.