\section{Results} \label{sec:results} \begin{table*}[b] \centering \small \setlength{\tabcolsep}{0.2em} \makebox[\textwidth]{% \begin{tabular}{l l *{18}{c}} & & Ack & E & Fib & FSve & Ham & LQns & Mat & Perm & Psc & Rev & RevTw & RFib & SQns & Sve & STw & Tak & Tw & WSeq \\\hline \multirow{3}{*}{Size} & ARM (b) & 188 & 2388 & 164 & 1036 & 1576 & 2708 & 4452 & 1756 & 2940 & 796 & 820 & 248 & 1660 & 1200 & 340 & 232 & 360 & 2628 \\ & Thumb (b) & 164 & 1944 & 148 & 840 & 1200 & 2196 & 3392 & 1332 & 2452 & 636 & 632 & 248 & 1328 & 936 & 304 & 204 & 312 & 2020 \\ & Diff. ($-\%$) & 12.8 & 18.6 & 9.8 & 18.9 & 23.9 & 18.9 & 23.8 & 24.1 & 16.6 & 20.1 & 22.9 & 0.0 & 20.0 & 22.0 & 10.6 & 12.1 & 13.3 & 23.1 \\\hline \multirow{3}{*}{Time} & ARM (s) & 0.59 & 1.39 & 0.59 & 1.18 & --- & 1.64 & --- & --- & --- & 4.09 & 1.07 & 1.15 & 1.16 & 0.99 & --- & 0.87 & --- & --- \\ & Thumb (s) & 0.66 & 1.44 & 0.62 & 1.11 & --- & 1.70 & --- & --- & --- & 4.31 & 1.08 & 1.18 & 1.22 & 0.99 & --- & 0.94 & --- & --- \\ & Diff. (\%) & 11.9 & 3.6 & 5.1 & 5.9 & --- & 3.7 & --- & --- & --- & 5.4 & 0.9 & 2.6 & 5.2 & 0.0 & --- & 8.0 & --- & --- \\ \end{tabular} } \caption{Code size and running time comparison for ARM and Thumb-2.\label{tab:results}} \end{table*} % Program Code size Running time (+gc) Nr. runs % ARM Thumb ARM Thumb % acker 3 10 188 164 0.59s 0.66s 20 % e 2000 2388 1944 1.39s 1.44s 10 % fsieve 50000 1036 840 1.18s 1.11s 10 % hamming 300 1576 1200 negligible - % invperm 16nrs 1756 1332 negligible - % lqueen 11 2708 2196 1.64s 1.70s 10 % mulmat 6 4452 3392 negligible - % nfib 35 164 148 0.59s 0.62s 10 % pascal 18 40 2940 2452 negligible - % reverse 10000 796 636 4.09s 4.31s 10 % revtwice 1 500 820 632 1.07s 1.08s 10 (-h 500m) % rfib 35.0 248 248 1.15s 1.18s 10 % sieve 3000 1200 936 0.99s 0.99s 10 % squeen 11 1660 1328 1.16s 1.22s 10 % str_arit SEGFAULT % stwice 340 304 negligible - % tak 32 16 8 232 204 0.87s 0.94s 10 % twice 360 312 negligible - % war_seq 2628 2020 negligible - \begin{multicols}{2} \subsection{Test suite} \label{sec:results:testsuite} We use a standard set of Clean programs to compare ARM and Thumb performance. These programs are included in the \texttt{examples} directory of every Clean release~\parencite{clean}. They are: \begin{description}[itemsep=0pt] \item[Ack] $A(3,10)$, where $A$ is the Ackermann function. \item[E] The approximation of $e$ to 2000 digits. \item[Fib] Computing the 35\textsuperscript{th} Fibonacci number using naive recursion. \item[FSve] The sieve of Eratosthenes (optimised), computing the first 50,000 primes. \item[Ham] The Hamming function, computing the first 300 numbers with only 2, 3 and 5 as prime factors. \item[LQns] Computing the possibilities to place 11 queens on an $11\times11$ \enquote{chess} board (not optimised). \item[Mat] Multiplying two $6\times6$ matrices. \item[Perm] Inverting a permutation of \clean{[1..16]}. \item[Psc] Pretty-printing the first 18 rows of the triangle of Pascal. \item[Rev] Reversing a 10,000-elements list, 10,000 times. \item[RevTw] Reversing a list of 500 elements 65,536 times using the higher-order function \clean{twice} (with a heap size of 500M% \footnote{The large heap size is needed to prevent switching to another garbage collector, which had not yet been finished at the moment of writing.}). \item[RFib] Computing the 35\textsuperscript{th} Fibonacci number using naive recursion and \clean{Real}s instead of \clean{Int}s. \item[SQns] Computing the possibilities to place 11 queens on a $11\times11$ \enquote{chess} board (optimised). \item[Sve] The sieve of Eratosthenes (not optimised), computing the first 3,000 primes. \item[STw] Incrementing an integer 65,536 times using the higher-order function \clean{twice} (optimised). \item[Tak] $\tau(32,16,8)$, where $\tau$ is the Takeuchi function. \item[Tw] Incrementing an integer 65,536 times using the higher-order function \clean{twice} (not optimised). \item[WSeq] A sequential version of Warshall's shortest path algorithm on a $6\times6$ matrix. \end{description} \subsection{Code size} \label{sec:results:size} We compared the code size for the code generated by the ARM and Thumb backends for all programs in the test suite. This does not include the run-time system, but does include parts of Clean's standard library, StdEnv, that are necessary for the program. The results are in \cref{tab:results}. The Thumb backend produces on average 17.3\% smaller code, with a standard deviation of 6.3pp. If we ignore those programs for which the ARM code size is less than 1,000 bytes, this is 21.0\%, with a standard deviation of 2.6pp., comparable to the 19.3\% found for the Clean compiler in \cref{sec:reg-alloc:results}. Small programs with short function blocks skew the data, because they contain relatively many jumps which are longer in Thumb than in ARM (see \cref{sec:storing-pc}). \subsection{Running time} \label{sec:results:time} For the same programs, we compare the running time. Some of the examples as provided in the Clean releases have a too short running time to be able to compare them, so the parameters had to be tweaked. In this case, the new parameters are mentioned in \cref{sec:results:testsuite}. For some examples it was not possible to increase the running time enough. \Cref{tab:results} lists \enquote{---} for these programs. The increase in running time is relatively low compared to the decrease in code size: 4.8\% on average with a standard deviation of 3.1pp. For the larger programs, this is 3.7\% with a standard deviation of 2.04pp. (though this is based on only five programs). \end{multicols} % vim: nowrap