\documentclass{scrartcl}

\usepackage{cleveref}
\usepackage{enumerate}
\usepackage[hidelinks]{hyperref}
\usepackage{minted}
\usepackage{tikz}
	\usetikzlibrary{arrows, matrix, positioning}

\title{The {\tt chesshs} Haskell Package}
\subtitle{Test Plan, Manual Testing \& Automated Testing}
\author{
	\begin{tabular}{r l}
		Ren\'e den Hertog & {\tt 1007823} \\
		Camil Staps       & {\tt 4498062} \\
		Erin van der Veen & {\tt 4431200}
	\end{tabular}
}
% Remove the date from '\maketitle'.
\makeatletter
	\patchcmd
		{\@maketitle}
		{{\usekomafont{date}{\@date\par}}\vskip\z@\@plus 1em}
		{}
		{}
		{}
\makeatother

\begin{document}

\maketitle

\section{Test Plan}

\section{Test goals}
\begin{enumerate}
	\setcounter{enumi}{4}
	\item
		The documentation of the SUT is defined in two parts.
		The first part, the chess part, is defined by the rules of the FIDE.\footnote{\url{http://fide.com/fide/handbook.html}}
		%TODO: Summary
		The second part, the PGN notation, is defined by the PGN standard.\footnote{\url{http://saremba.de/chessgml/standards/pgn/pgn-complete.htm}}
		%TODO: Summary
	\item
		The parts of the SUT that will be tested will be the entire library and the individual components thereof.
		This implies that PGN parsing will be tested, as well as the verification part of the library that determines if a move is legal.
		The function that determines the checkmate position of the game will also be tested.
		This means, explicitly, that the timing rules, and any other rules of the game of Chess, will not be tested.
		%TODO: Manual/Automated testing
	\item
		Functionality and Reliability.
\end{enumerate}
\subsection*{System Under Test (SUT) Description}

The chosen SUT is the {\tt chesshs} Haskell package. \cite{SUT} It should be able to run in any software environment and hardware platform which supports Haskell. The latest version of the SUT, since the 13\textsuperscript{th} of March 2015, is {\tt 0.2.1}.

\paragraph{Executing/Running the SUT} \label{P:ERSUT}

It is required to be able to compile Haskell, typically through the Glasgow Haskell Compiler (GHC), and having the {\tt chesshs} package installed, normally through Cabal via the \mintinline{bash}{cabal install chesshs} command. \cite{GHC, C} Both the GHC and Cabal are part of the Haskell Platform. \cite{HP} Under our computer systems, compiling and starting the SUT is done with the \mintinline{bash}{make} and \mintinline{bash}{./runchess} commands, respectively.

The SUT is used by providing it with, via the standard input, a text file containing a game of Chess in the Portable Game Notation (PGN). \cite{PGN} After computation, via the standard output, some general information about the Chess match is displayed, including the end result. Followed by the state of the board after each move. In any other case, an error message is displayed. \\

As is common with everything software, the SUT is likely to contain faults. However, the risks surrounding the program are near zero due to the very low impact of the program faulting. As of our expectation, the most ``risky" scenario for the SUT is at a notable Chess competition. In the case the program does not behave as intended, it is almost certain that a human referee would take over its place. Furthermore, we expect such a system to be used as a supporting tool for match officials and not as a replacement for regulation.

\section{Test method}
\begin{enumerate}
	\setcounter{enumi}{7}
	\item
		Unit, Integration. The scale and the number of sub-components of the project limits us to these types of tests.
	\item
		Mostly Black box testing, for the input/output of entire games in pgn notation.
		%TODO: Fix sentence
		The functions of the library will also be tested using a White box model.
	\item
		\begin{enumerate}[a)]
			\item
				For test generation of the black box testing, we will use a mixture of error guessing and the parsing of PGN files in a database.
				White box testing will be done through statement coverage and branch/condition coverage.
			\item
				Equivalence partitioning and boundary value analysis is hard to implement given our SUT, given that all possible chess games are nearly impossible to enumerate.
				Use case testing is also very hard to test, given that Chess.hs is a library, no user will likely every interact with the library directly.
				Path coverage and condition coverage are not implemented in Quickcheck, making it substantially harder to implement.
		\end{enumerate}
\end{enumerate}

\begin{figure}
	\centering
	\begin{tikzpicture}[
			every node/.style={draw,rectangle},
			title/.style={draw=none},
			->,>=open triangle 60]
		\node (bbtest) at (4,0)  {Black-box tester (\texttt{test.sh})};
		\node[matrix,row sep=.5em] (prog) at (0,0) {
			\node[title] {Wrapper};\\
			\node (chesshs) {Chess.hs};\\
		};
		\node[left=2em of chesshs] (wbtest) {White-box tester (\texttt{test.hs})};
		\node[below of=wbtest] (quickcheck) {Quickcheck};
		\draw (bbtest) -- (prog);
		\draw (wbtest) -- (chesshs);
		\draw (quickcheck) -- (wbtest);
	\end{tikzpicture}
	\caption{A graphical overview of the testing setup.\label{fig:testing-setup}}
\end{figure}

\begin{listing}
	\inputminted[fontsize=\footnotesize,firstline=13]{haskell}{../src/runchess.hs}
	\caption{The basic wrapper.\label{lst:wrapper}}
\end{listing}

\begin{thebibliography}{8}
	\bibitem[C]    {C}    \url{http://haskell.org/cabal}
	\bibitem[GHC]  {GHC}  \url{http://haskell.org/ghc}
	\bibitem[HP]   {HP}   \url{http://haskell.org/platform}
	\bibitem[PGN]  {PGN}  \url{http://saremba.de/chessgml/standards/pgn/pgn-complete.htm}
	\bibitem[SUT]  {SUT}  \url{http://hackage.haskell.org/package/chesshs}
\end{thebibliography}

\section{Test Environment}
\begin{enumerate}
	\setcounter{enumi}{10}
	\item 
		A wrapper written by us will be used to interact with the SUT.
		See \cref{lst:wrapper} for the implementation.
	\item
		Tests will be automatically performed using a combination of bash and Quickcheck.
	\item
		\begin{enumerate}[a)]
			\item
				The software environment GHC 8, Quickcheck 2.10, Linux Debian (derived).
				The hardware environment is a set of three amd64 laptops.
			\item
				See \cref{fig:testing-setup}.
		\end{enumerate}
	\item
		A Git repository, the results will be stored in individual files.
\end{enumerate}
\end{document}