Complexity analysis and performance of double hashing sort algorithm

Sorting is a fundamental and serious problem in different fields of computer science such as databases [1], computational geometry [2, 3], graphs [4], and bioinformatics [5]. For example, to determine a minimum spanning tree for a weighted connected undirect graph, Kruskal designed a greedy algorithm and started the solution by sorting the edges according to weight in increasing order [6].

Additionally, there are several reasons why the sorting problem in the algorithm aspect is important. The first pertains to finding a solution for a problem in which data are sorted according to certain criteria; this routine should be more efficient, in running time, than when the data are unsorted. For example, searching for an element in an unsorted array requires O(n); the searching requires O(log n) time when the array is sorted [6]. The second reason is that the sorting problem has been solved by a lot of algorithms using different strategies such as brute-force, divide-and-conquer, randomized, distribution, and advanced data structures [6, 7]. Insertion sort, bubble sort, and selection sort are examples of sorting algorithms using brute-force; merge sort and quick sort algorithms are examples of the divide-and-conquer strategy. Radix sort and flash sort algorithms are examples of sorting using the distribution technique; heap sort is an example of using an advanced data structure method. Additionally, the randomization techniques have been used for many previous sorting algorithms such as randomized shell sort algorithm [8]. The third reason is the lower bound for the sorting problem which is equal to Ω(n log n) was determined based on a comparison model. Based on a determined lower bound, the sorting algorithms are classified into two groups: (1) optimal algorithms such as merge sort and heap sort algorithms and (2) non-optimal algorithms such as insertion sort and quick sort algorithms. The fourth reason is when the input data of sorting problem are taken from the domain of integers [1,m], different strategies are suggested to reduce the time of sorting from O(n log n) to linear, O(n). These strategies are not based on a comparison model. Examples for this kind of sorting are counting sort and bucket sort [6].

The sorting problem has been studied thoroughly, and many research papers have focused on designing fast and optimal algorithms [9–16]. Also, some studies have focused on implementing these algorithms to obtain an efficient sorting algorithm on different platforms [15–17]. Additionally, several measurements have been suggested to compare and evaluate these sorting algorithms according to the following criteria [6, 7, 18]: (1) Running time, which is equal to the total number of operations done by the algorithm and is computed for three cases: (i) best case, (ii) worst case, and (iii) average case; (2) the number of comparisons performed by the algorithm; (3) data movements, which are equal to the total number of swaps or shifts of elements in the array; (4) in place, which means that the extra memory required by the algorithm is constant; and (5) stable, which means that the order of equal data elements in the output array is similar in which they appear in the input data.

Recently, a new sorting algorithm has been designed and called the double hashing sort (DHS) algorithm [19]. This algorithm is based on using the hashing strategy in two steps; the hash method used in the first step is different than in the second step. Based on these functions, the elements of the input array are divided into two groups. The first group is already sorted, and the second group will be sorted using a quick sort algorithm. The authors in [19] studied the complexity of the algorithm and calculated three cases of running time and storage of the algorithm. In addition, the algorithm was implemented and compared with a quick sort algorithm experimentally. The results reveal that the DHS algorithm is faster than the quick sort algorithm.

In this paper, we study the DHS algorithm from three viewpoints. The first aspect involves reevaluating the complexity analysis of the DHS algorithm based on the relation between the size of the input array and the range of the input elements. Then, we prove that the time complexity is different than that is calculated in [19] for most cases. The second aspect involves proving that a previous algorithm, counting sort algorithm, exhibits a time complexity less than or equal to that of the DHS algorithm. The third aspect refers to proving that the DHS algorithm exhibits a lower level of performance than another certain algorithm from a practical point of view.

The results of these studies are as follows: (1) the previous complexity analysis of the DHS algorithm was not accurate; (2) we calculated the corrected analysis of the DHS algorithm; (3) we proved that the counting sort algorithm is faster than the DHS algorithm from theoretical and practical points of view. Additionally, the percentage of improvement was roughly 46% on the average for all cases studied.

The remainder of this work is organized into four sections. In the “Comments on DHS algorithm” section, we discuss briefly the DHS algorithm, its analysis, and then, we provide some commentary about the analysis of DHS algorithm that was introduced by [19]. In the “Complexity analysis of DHS algorithm” section, we analyze the DHS algorithm using different methods. Also, we show that a previous algorithm exhibits a time complexity less than that of the DHS algorithm in most cases. We prove experimentally that the DHS algorithm is not as fast as the previous algorithm in the “Performance evaluation” section. Finally, our conclusions are presented in the “Conclusions” section.

The aim of this section is to give some comments about DHS algorithm. So, we mention in this section shortly the main stages and complexity analysis of DHS algorithm. Then, we give some comments about the analysis of the algorithm.

Comments on DHS algorithm

In this part, we provide two main comments about the DHS algorithm. The first category of comments is related to the theoretical analysis of the DHS algorithm; the second category of comments is related to the data generated in the practical study.

For the first category, we found that the running times for the DHS algorithm have the following three notes.

The first note is that the running time calculated for the best case is correct when m is small; the running time calculation is not correct when n is small. When n is small and m is large, the number of repetitions in the input array is very small in general. Therefore, most of the elements belong to the GrAr array. This situation implies that the DHS algorithm uses the quick sort algorithm on the GrAr array. Therefore, the running time of the third phase is O(n log n), not O(n).

Example 1 Let n = 10, m = n² = 100, and the elements of A is well distributed as follows:

It is clear that the value of n is small compared with m. Therefore, in general, the number of elements that belong to the EqAr array is very small compared with the GrAr array that contains most of the input elements.

The second note is that the calculated running time for the worst case is not correct if the value of n is large and m is small. This situation implies that the number of repetitions in the input array is large because the n elements of the input array belong to a small range. Therefore, the maximum number of elements belonging to the GrAr array is less than m, say α. On the other hand, the array EqAr contains n − α elements. Therefore, the statement “the number of elements belong to the GrAr array is large” [19] is not accurate. It should be small since m is small. Therefore, the calculated running time for the worst case of the DHS algorithm, O(n + x log m), is not accurate in the general case.

Example 2 Let m = 4, n = m² = 16 and A is given as follows

It is clear that the number of non-repeated elements is 3 and the GrAr array contains only 3 different elements, 2, 3, and 4; the EqAr array contains 13 elements from 16.

The third note is that no determination when the average case occurs, which is why the running time is O(n + x log m).

For the second category, the data results in [19] for the three cases reveal how that the percentages of elements in the EqAr array, repeated elements, is at least 65%, which is too much and do not represents the general or average cases. This situation means that the input data used to measure the performance of DHS algorithm do not represent various types of input. For example, in the average case, n = 100,000 and the range of elements equals 100,000, the number of elements in the EqAr array is 70,030 [19].

In this section, we study the complexity of the DHS algorithm using another method of analysis. The DHS algorithm is based on dividing the elements of an input array into many slots; each slot contains elements in a specific range. Therefore, we mainly analyze the DHS algorithm based on the relation between the size of the array, n, and the domain of the elements in the array, m. There are three cases for the relation between n and m.

Case 1: O(m) < O(n). In this case, the range of values for the elements of the input array is small compared with the number of elements in A. This case can be formed as A = (a₁, a₂, …, a_n), where a_i < m and m < n. We use big Oh notation to illustrate that the difference between n and m is significant. For example, let \( m=\sqrt{n} \) and m =  log n and if n = 10,000, then m = 100 and 4, respectively.

Case 2: O(m) = O(n). In this case, the range of the values for the elements of the input array is equal to the number of elements. This case can be formed as A = (a₁, a₂, …, a_n), where a_i ≤ m, n ≈ m, and m = α n ± β such that α and β are constant. For example, let m = 2n and m = n + 25; if n = 1000, then m = 2000 and 1025, respectively.

Case 3: O(n) < O(m). In this case, the range of the values for the elements of the input array is greater than the number of elements. This case can be formed as A = (a₁, a₂, …, a_n), where a_i < m, m > n. For example, let m = n^k, where k > 1. If n = 100 and k= 3, then m = 1,000,000.

Now, we study the complexity of the DHS algorithm in terms of three cases.

Case 1: O(m) < O(n). The value of m is small compared with the input size n; the array contains many repeated elements. In this case, the maximum number of slots is m, and there is no need to map the elements of the input array to n slots such as mapping sort algorithm [20], where the index of the element a_i is calculated using the equation: ⌊((a_i − Min(A)) × n)/(Max(A) − Min(A))⌋.

The solution to this case can be found using an efficient previous sorting algorithm called counting sort (CS) algorithm [6]. Therefore, there is no need to use the insertion sort, quick sort, and merge sort algorithms as in [19, 20] to sort un-repeated elements. The main idea of the CS algorithm is to calculate the number of elements less than the integer i ∈ [1, m]. Then, we use this value to allocate the element a_j in a correct location in the array A, ∀ 1 ≤ j ≤ n. The CS algorithm consists of three steps. The first step of the CS algorithm starts with scanning the input array A and computing the number of repetitions each element occurs within the input array A. The second step of the CS algorithm is to calculate, for each i ∈ [1, m], the starting location in the output array by updating the array C using the prefix-sum algorithm. The prefix-sum of the array C is to compute \( C\left[i\right]={\sum}_{j=1}^iC\left[j\right] \). The final step of the CS algorithm allocates each i ∈ [1, m] and its repetition in the output array using the array C.

Additionally, the running time of the CS algorithm is O(n + m) = O(n), because O(m) < O(n). The running time of the CS algorithm does not depend on the distribution of the elements, uniform and non-uniform, over the range m. Also, the CS algorithm is independent of how many repeated and unrepeated elements are found in the input array.

The following example illustrates how to use the CS algorithm in this case; there is no need to distribute the input into two arrays, EqAr and GrAr, as in the DHS Algorithm.

Example 3 Let m = 5, n = m² = 25, and the elements of the input array A as in Fig. 1a. As a first step, we calculate the repetition array C, where C[i] represents the number of repetitions of the integer i ∈ [1, m] in the input array A as in Fig. 1b. It is clear that the number of repetition for the integer “1” is 6; the integer “4” has zero repetition. In the second step, we calculate the prefix-sum of C as in Fig. 1c, where the prefix-sum for C[i] is equal to \( {\sum}_{j=1}^iC\left[j\right] \). In the last step, the integer 1 is located from positions 1–6; the integer 2 is located from positions 7–14 and so on. Therefore, the output array is shown as in Fig. 1d.

Remark Sometimes the value of m cannot fit in memory because the storage of the machine is limited. Then, we can divide the input array into k (<m) buckets, where the bucket number i contains the elements in the range [(i − 1)m/k + 1, i m/k], 1 ≤ i ≤ k. For a uniform distribution, each bucket contains n/k elements approximately. Therefore, the running time to sort each bucket is O(n/k + k). Hence, the overall running time is O(k(n/k + k)) = O(n + k²) = O(n). For non-uniform distributions, the number of elements in each bucket i is n_i such that \( {\sum}_{i=1}^k{n}_i=n \). Therefore, the overall running time is O(n + k) = O(n).

Case 2: O(m) = O(n). The value of m is approximately equal to the input size n. If the elements of the array are distributed uniformly, then the number of repetitions for the elements of the array is constant. In this case, we have two comments about the DHS algorithm. The first comment is that there is no need to construct two different arrays, GrAr and EqAr. The second comment is that there is no need to use the quick sort algorithm in the sorting because we can sort the array using the CS algorithm.

If the distribution of the elements for the input array is non-uniform, then the number of repetitions for the elements of the array is varied. Let the total number of repetitions for all the elements of the input array be φ(n). Therefore, the array EqAr contains φ(n) elements; the array GrAr contains n − φ(n) elements. The running time for executing the DHS algorithm is O(n + (n − φ(n)) log (n − φ(n)) ), where the first term represents the running time for the first two stages and the second term represents applying the quick sort algorithm on the GrAr array. In the average case, we have n/2 repeated elements, so the running time of the DHS algorithm is O((n/2) log (n/2) ) = O(n log n). In this case, the CS algorithm is better than the DHS algorithm. On the other side, if φ(n) ≈ n, then the running time of the DHS algorithm is O(n).

Example 4 Let m = 30, n = 25. Fig. 2 shows how the CS algorithm can be used instead of the DHS algorithm in the case of a uniform distribution.

From the analysis of the DHS algorithm for the three cases based on the relation between m and n, there is a previous sorting algorithm that is associated with less time complexity than the DHS algorithm.

In this section, we studied the performance of the DHS and CS algorithms from a practical point of view based on the relation between m and n for the two cases, O(m) < O(n) and O(m) = O(n). Note that both algorithms are not suitable in the case of O(n) < O(m).

Experimental results

The results of implementing the methodology to measure the running time of the CS and DHS algorithms considering all parameters that affect the execution times are shown in Figs. 3 and 4. Each figure consists of four subfigures (a), (b), (c), and (d) for n = 10⁵, 10⁶, 10⁷, and 10⁸, respectively. Also, each subfigure consists of six pairs of bars. Each pair of bar represents the running times for the CS and DHS algorithms using a certain type of data distribution. Fig. 3 illustrates the running times for the CS and DHS algorithm in the case of O(m) < O(n) and shows that the running time for the CS algorithm is faster than for the DHS algorithm for all values of n and benchmarks. The difference in running time between the algorithms varies from one type of data distribution to another. For example, the running time for the CS algorithm using the six benchmarks are 7.7, 9.7, 8.2, 9.8, 8.2, and 6.1 milliseconds; while the running times for the DHS algorithm using the same benchmarks are 11.2, 12.4, 10.6, 15, 13.9, and 14.1 milliseconds in the case of n = 10⁶. In general, the maximum difference between the two algorithms occurs in the case of a Gaussian distribution

Similarly, Fig. 4 illustrates the running times for the CS and DHS algorithm in the case of O(m) = O(n) and shows that the running time for the CS algorithm is faster than for the DHS algorithm for all values of n and benchmarks.

Table 1 lists data pertaining to the performance improvements of the CS algorithm in two points of view (i) range of improvements, and (ii) mean of improvements. In the case of a range of improvements, we fix the size of the array n and calculate the percentage of improvement for each benchmark. Then, we record the range of improvements from the minimum to maximum values as in the second and fourth columns in Table 1 for O(m) < O(n) and O(m) = O(n), respectively. In the case of the mean of improvements, we take the mean value for the percentage of improvements for all data distributions, as in the third and fifth columns. The results of applying these measurements are as follows.

1. 1.

   In the case of O(m) < O(n), the CS algorithm performed 45–80%, 21.5–56.5%, 20–57%, and 21–47% faster than the DHS algorithm for n = 10⁵, 10⁶, 10⁷, and 10⁸, respectively. For example n = 10⁸, the percentage of improvements for CS algorithm for data distribution: [U], [D], [S], [RS], [NS], and [G], are 21%, 27.2%, 24.4%, 23.9%, 24.9%, and 47%, respectively. Therefore, the range of improvements for the CS algorithm is 21–47% when n = 10⁸. Additionally, based on the percentage of improvement calculated for each data distribution and a fixed value of n, we can calculate the mean of improvements which are equal to 62.5%, 34.7%, 28.3%, and 28.1% for n = 10⁵, 10⁶, 10⁷, and 10⁸, respectively. For example, n = 10⁸, the mean of improvements is 28%.

    
2. 2.

   In the case of O(m) = O(n), the CS algorithm performed 88.7–96.8%, 58.5–60.7%, 20–53.8%, and 24.8–43.5% faster than the DHS algorithm for n = 10⁵, 10⁶, 10⁷, and 10⁸, respectively. For example n = 10⁷, the percentage of improvements for CS algorithm for data distribution: [U], [D], [S], [RS], [NS], and [G], are 20.5%, 20%, 24.4%, 47.4%, 46.3%, and 53.8%, respectively. Therefore, the range of improvements for CS algorithm is 20–53.8% when n = 10⁷. Similarly, we can compute the mean of improvements which are equal to 91.9%, 59.9%, 35.4%, and 33.6% for n = 10⁵, 10⁶, 10⁷, and 10⁸, respectively.

    

n	O(m) < O(n)		O(m) = O(n)
	Range of improvements	Average improvements	Range of improvements	Average improvements
105	45–80%	62.5%	88.7–96.8%	91.9%
106	21.5–56.5%	34.7%	58.5–60.7%	59.9%
107	20–57%	28.3%	20–53.8%	35.4%
108	21–47%	28.1%	24.8–43.5%	33.6%

From previous results, the CS algorithm performed 38.4% and 55.2% faster than the DHS algorithm for O(m) < O(n) and O(m) = O(n), respectively. Therefore, the percentage of improvement for the CS algorithm was roughly 46% on the average for all cases studied.

The sorting problem is to rearrange the elements of a given array in increasing order. This problem is important in a variety of computer science applications, and it is used as a subroutine in many computer applications. In this work, we studied the complexity analysis and measured performance of the double hashing sort (DHS) algorithm. The results of this study are (1) the previous complexity analysis of the DHS algorithm was not accurate; (2) we calculated the corrected analysis of this algorithm based on the relation between size of the input array n and domain of the input elements m; (3) there is a previous sorting algorithm called counting sort algorithm that is faster than the DHS algorithm in the case of O(m) ≤ O(n) from theoretical and practical points of view; and (4) our experimental studies are based on six benchmarks; the percentage of improvement was roughly 46% on the average for all cases studied.