Flashsort

Template:Short description Flashsort is a distribution sorting algorithm showing [[O notation|linear computational complexity Template:Mvar]] for uniformly distributed data sets and relatively little additional memory requirement. The original work was published in 1998 by Karl-Dietrich Neubert.

Concept

Flashsort is an efficient in-place implementation of histogram sort, itself a type of bucket sort. It assigns each of the Template:Mvar input elements to one of Template:Mvar buckets, efficiently rearranges the input to place the buckets in the correct order, then sorts each bucket. The original algorithm sorts an input array Template:Mvar as follows:

1. Using a first pass over the input or a priori knowledge, find the minimum and maximum sort keys.
2. Linearly divide the range [A_min, A_max] into Template:Mvar buckets.
3. Make one pass over the input, counting the number of elements Template:Mvar which fall into each bucket. (Neubert calls the buckets "classes" and the assignment of elements to their buckets "classification".)
4. Convert the counts of elements in each bucket to a prefix sum, where Template:Mvar is the number of elements Template:Mvar in bucket Template:Mvar or less. (L₀ = 0 and L_m = n.)
5. Rearrange the input so all elements of each bucket Template:Mvar are stored in positions Template:Mvar where L_b−1 < i ≤ L_b.
6. Sort each bucket using insertion sort.

Steps 1–3 and 6 are common to any bucket sort, and can be improved using techniques generic to bucket sorts. In particular, the goal is for the buckets to be of approximately equal size (n/m elements each), with the ideal being division into Template:Mvar quantiles. While the basic algorithm is a linear interpolation sort, if the input distribution is known to be non-uniform, a non-linear division will more closely approximate this ideal. Likewise, the final sort can use any of a number of techniques, including a recursive flash sort.

What distinguishes flash sort is step 5: an efficient O(n) in-place algorithm for collecting the elements of each bucket together in the correct relative order using only Template:Mvar words of additional memory.

Memory efficient implementation

The Flashsort rearrangement phase operates in cycles. Elements start out "unclassified", then are moved to the correct bucket and considered "classified". The basic procedure is to choose an unclassified element, find its correct bucket, exchange it with an unclassified element there (which must exist, because we counted the size of each bucket ahead of time), mark it as classified, and then repeat with the just-exchanged unclassified element. Eventually, the element is exchanged with itself and the cycle ends.

The details are easy to understand using two (word-sized) variables per bucket. The clever part is the elimination of one of those variables, allowing twice as many buckets to be used and therefore half as much time spent on the final O(n²) sorting.

To understand it with two variables per bucket, assume there are two arrays of Template:Mvar additional words: Template:Mvar is the (fixed) upper limit of bucket Template:Mvar (and K₀ = 0), while Template:Mvar is a (movable) index into bucket Template:Mvar, so K_b−1 ≤ L_b ≤ K_b.

We maintain the loop invariant that each bucket is divided by Template:Mvar into an unclassified prefix (Template:Mvar for K_b−1 < i ≤ L_b have yet to be moved to their target buckets) and a classified suffix (Template:Mvar for L_b < i ≤ K_b are all in the correct bucket and will not be moved again). Initially L_b = K_b and all elements are unclassified. As sorting proceeds, the Template:Mvar are decremented until L_b = K_b−1 for all Template:Mvar and all elements are classified into the correct bucket.

Each round begins by finding the first incompletely classified bucket Template:Mvar (which has K_c−1 < L_c) and taking the first unclassified element in that bucket Template:Mvar where i = K_c−1 + 1. (Neubert calls this the "cycle leader".) Copy Template:Mvar to a temporary variable Template:Mvar and repeat:

• Compute the bucket Template:Mvar to which Template:Mvar belongs.
• Let j = L_b be the location where Template:Mvar will be stored.
• Exchange Template:Mvar with Template:Mvar, i.e. store Template:Mvar in Template:Mvar while fetching the previous value Template:Mvar thereby displaced.
• Decrement Template:Mvar to reflect the fact that Template:Mvar is now correctly classified.
• If j ≠ i, restart this loop with the new Template:Mvar.
• If j = i, this round is over and find a new first unclassified element Template:Mvar.
• When there are no more unclassified elements, the distribution into buckets is complete.

When implemented with two variables per bucket in this way, the choice of each round's starting point Template:Mvar is in fact arbitrary; any unclassified element may be used as a cycle leader. The only requirement is that the cycle leaders can be found efficiently.

Although the preceding description uses Template:Mvar to find the cycle leaders, it is in fact possible to do without it, allowing the entire Template:Mvar-word array to be eliminated. (After the distribution is complete, the bucket boundaries can be found in Template:Mvar.)

Suppose that we have classified all elements up to i−1, and are considering Template:Mvar as a potential new cycle leader. It is easy to compute its target bucket Template:Mvar. By the loop invariant, it is classified if L_b < i ≤ K_b, and unclassified if Template:Mvar is outside that range. The first inequality is easy to test, but the second appears to require the value Template:Mvar.

It turns out that the induction hypothesis that all elements up to i−1 are classified implies that i ≤ K_b, so it is not necessary to test the second inequality.

Consider the bucket Template:Mvar which position Template:Mvar falls into. That is, K_c−1 < i ≤ K_c. By the induction hypothesis, all elements below Template:Mvar, which includes all buckets up to K_c−1 < i, are completely classified. I.e. no elements which belong in those buckets remain in the rest of the array. Therefore, it is not possible that b < c.

The only remaining case is b ≥ c, which implies K_b ≥ K_c ≥ i, Q.E.D.

Incorporating this, the flashsort distribution algorithm begins with Template:Mvar as described above and i = 1. Then proceed:

• If i > n, the distribution is complete.
• Given Template:Mvar, compute the bucket Template:Mvar to which it belongs.
• If i ≤ L_b, then Template:Mvar is unclassified. Copy it a temporary variable Template:Mvar and:
  • Let j = L_b be the location where Template:Mvar will be stored.
  • Exchange Template:Mvar with Template:Mvar, i.e. store Template:Mvar in Template:Mvar while fetching the previous value Template:Mvar thereby displaced.
  • Decrement Template:Mvar to reflect the fact that Template:Mvar is now correctly classified.
  • If j ≠ i, compute the bucket Template:Mvar to which Template:Mvar belongs and restart this (inner) loop with the new Template:Mvar.
• Template:Mvar is now correctly classified. Increment Template:Mvar and restart the (outer) loop.

While saving memory, Flashsort has the disadvantage that it recomputes the bucket for many already-classified elements. This is already done twice per element (once during the bucket-counting phase and a second time when moving each element), but searching for the first unclassified element requires a third computation for most elements. This could be expensive if buckets are assigned using a more complex formula than simple linear interpolation. A variant reduces the number of computations from almost 3n to at most 2n + m − 1 by taking the last unclassified element in an unfinished bucket as cycle leader:

• Maintain a variable Template:Mvar identifying the first incompletely-classified bucket. Let c = 1 to begin with, and when Template:Mvar, the distribution is complete.
• Let i = L_c. If i = L_c−1, increment Template:Mvar and restart this loop. (L₀ = 0.)
• Compute the bucket Template:Mvar to which Template:Mvar belongs.
• If b < c, then L_c = K_c−1 and we are done with bucket Template:Mvar. Increment Template:Mvar and restart this loop.
• If b = c, the classification is trivial. Decrement Template:Mvar and restart this loop.
• If b > c, then Template:Mvar is unclassified. Perform the same classification loop as the previous case, then restart this loop.

Most elements have their buckets computed only twice, except for the final element in each bucket, which is used to detect the completion of the following bucket. A small further reduction can be achieved by maintaining a count of unclassified elements and stopping when it reaches zero.

Performance

The only extra memory requirements are the auxiliary vector Template:Mvar for storing bucket bounds and the constant number of other variables used. Further, each element is moved (via a temporary buffer, so two move operations) only once. However, this memory efficiency comes with the disadvantage that the array is accessed randomly, so cannot take advantage of a data cache smaller than the whole array.

As with all bucket sorts, performance depends critically on the balance of the buckets. In the ideal case of a balanced data set, each bucket will be approximately the same size. If the number Template:Mvar of buckets is linear in the input size Template:Mvar, each bucket has a constant size, so sorting a single bucket with an O(n²) algorithm like insertion sort has complexity O(1²) = O(1). The running time of the final insertion sorts is therefore m ⋅ O(1) = O(m) = O(n).

Choosing a value for Template:Mvar, the number of buckets, trades off time spent classifying elements (high Template:Mvar) and time spent in the final insertion sort step (low Template:Mvar). For example, if Template:Mvar is chosen proportional to

1. REDIRECT Template:Radic

Template:Rcat shell, then the running time of the final insertion sorts is therefore m ⋅ O(

1. REDIRECT Template:Radic

Template:Rcat shell²) = O(n^3/2).

In the worst-case scenarios where almost all the elements are in a few buckets, the complexity of the algorithm is limited by the performance of the final bucket-sorting method, so degrades to O(n²). Variations of the algorithm improve worst-case performance by using better-performing sorts such as quicksort or recursive flashsort on buckets which exceed a certain size limit.

For m = 0.1 n with uniformly distributed random data, flashsort is faster than heapsort for all Template:Mvar and faster than quicksort for n > 80. It becomes about twice as fast as quicksort at n = 10000. Note that these measurements were taken in the late 1990s, when memory hierarchies were much less dependent on cacheing.

Due to the in situ permutation that flashsort performs in its classification process, flashsort is not stable. If stability is required, it is possible to use a second array so elements can be classified sequentially. However, in this case, the algorithm will require O(n) additional memory.

See also

• Interpolation search, using the distribution of items for searching rather than sorting

External links

• Implementations of Randomized Sorting on Large Parallel Machines (1992)
• Implementation of Parallel Algorithms (1992)
• Visualization of Flashsort Template:Webarchive