diff options
Diffstat (limited to 'assignments')
| -rw-r--r-- | assignments/assignment1-test-run.txt | 12 | ||||
| -rw-r--r-- | assignments/assignment1.tex | 79 |
2 files changed, 62 insertions, 29 deletions
diff --git a/assignments/assignment1-test-run.txt b/assignments/assignment1-test-run.txt index 06c460f..2ddcec4 100644 --- a/assignments/assignment1-test-run.txt +++ b/assignments/assignment1-test-run.txt @@ -2,6 +2,18 @@ Test: "Normal Legal Game" passed ################################################## Test: "Two Games, single file" passed ################################################## +Test: "Illegal move by Bishop" failed +InvalidMove | NoParse +################################################## +Test: "Illegal move by Knight" failed +InvalidMove | NoParse +################################################## +Test: "Illegal move by Queen" failed +InvalidMove | NoParse +################################################## +Test: "Double jump forward" failed +InvalidMove | NoParse +################################################## Test: "Double Castling on Kingside" passed ################################################## Test: "White Moving Black piece" failed diff --git a/assignments/assignment1.tex b/assignments/assignment1.tex index 6275e9b..a135402 100644 --- a/assignments/assignment1.tex +++ b/assignments/assignment1.tex @@ -5,7 +5,9 @@ \usepackage{enumerate} \usepackage[hidelinks]{hyperref} \usepackage{minted} -\setminted{fontsize=\small,breaklines,tabsize=4} +\setminted{fontsize=\small,breaklines,breakanywhere,tabsize=4} +\usepackage{caption} +\newenvironment{longlisting}{\captionsetup{type=listing}}{} \usepackage{skak} \usepackage{tikz} \usetikzlibrary{arrows, matrix, positioning} @@ -80,13 +82,6 @@ However, due to the competitive nature of the game, \paragraph{Portable Game Notation (PGN)} Many Chess matches are recorded in PGN~\cite{PGN}, an example of which is shown in \cref{L:PGNE}. -\begin{listing} - \inputminted[lastline=19]{text}{../test/database/m0.in} - \dots - \inputminted[firstline=42]{text}{../test/database/m0.in} - \caption{An example of Portable Game Notation (PGN).\label{L:PGNE}} -\end{listing} - PGN consists of two parts: the tag pairs and the movetext. The tag pairs encode properties of the game of Chess in question, @@ -229,11 +224,6 @@ Hence, it is substantially harder to implement these test generation techniques Since the SUT is in actuality the \texttt{chesshs} package, a user interface is clearly lacking. Thats why we have written a wrapper which makes the library interactive. See \cref{L:WI} for the full implementation. -\begin{listing} - \inputminted{haskell}{../src/runchess.hs} - \caption{Wrapper Implementation} \label{L:WI} -\end{listing} - Black box tests will be automatically performed using shell scripting. In our case, we will be using bash and Linux compatible scripts. Test automation for white box testing is achieved by functionalities available in the to be used testing tool QuickCheck~\cite{QC}. @@ -281,11 +271,6 @@ This position sequence is then compared to a user defined expected output, an ex The first two lines contain metadata, after which a list of board configurations is printed, either until the match is finished or an invalid move is detected. -\begin{listing} - \inputminted{text}{../test/database/m2.out} - \caption{An example output file.\label{L:OUTE}} -\end{listing} - Because we do not test an interactive program, our manual tests are in a sense already automated. If the wrapper were extended to support interactivity, @@ -294,7 +279,7 @@ If the wrapper were extended to support interactivity, \subsection*{Database} We have created 11 test cases for both valid and invalid games. As described above, the invalid games are derived from the FIDE rules - in that they attempt to find borderline cases where the library may deviate from those rules~\cite{FIDE}. + in that they attempt to find borderline cases where the library may deviate from those rules~\cite{FIDERULES}. The concrete tests we have used are shown in \cref{T:T}. \begin{table} @@ -312,6 +297,10 @@ The concrete tests we have used are shown in \cref{T:T}. 8 & Mixed PGN comment notations & Accepted\\ 9 & Continuing after insufficient material draw & \texttt{InvalidMove}\\ 10 & Two games in a single file & The outputs for both games\\ + 11 & Illegal move by Bishop & \texttt{InvalidMove}\\ + 12 & Illegal move by Knight & \texttt{InvalidMove}\\ + 13 & Illegal move by Queen & \texttt{InvalidMove}\\ + 14 & Double jump forward & \texttt{InvalidMove}\\ \end{tabular} \caption{Descriptions of the manual tests.\label{T:T}} \end{table} @@ -323,15 +312,7 @@ This script runs all tests and compares the output that is produced by the wrapp If there is no difference between the two, the test is considered \enquote{passed}. If there is a difference, the test is considered \enquote{failed} and the script shows the difference. -\begin{listing} - \inputminted{bash}{../test/test.sh} - \caption{Automated Test script\label{L:ATS}} -\end{listing} - -\begin{listing} - \inputminted[fontsize=\footnotesize]{text}{assignment1-test-run.txt} - \caption{Results of the test run.\label{L:TR}} -\end{listing} +\section{Results} The output of the test run on the tests in \cref{T:T} is given in \cref{L:TR}. We were able to find several failures: @@ -349,7 +330,18 @@ We were able to find several failures: This special pawn capture is only allowed in the move directly after the target pawn moved. The test attempts to capture the pawn one round later. Like the test above, we would expect an \texttt{InvalidMove} error here, - but \texttt{chesshs} gives a \texttt{NoParse} error. + but \texttt{chesshs} gives a \texttt{NoParse} error.% + \footnote{% + This is not a big issue. + Most important is that the move was rejected. + However, the \texttt{NoParse} error is not a very clear name for what it is used for.} + + \item[Illegal moves] + Likewise, the library produces \texttt{NoParse} when a move is attempted to an unreachable field, + whereas an \texttt{InvalidMove} error would be more appropriate. + Also the \textbf{\textsf{double jump forward}} test, + which has a pawn moving two ranks forward twice, + suffers from this issue. \item[Mate when impossible] This test includes the \texttt{\#} annotation in the PGN file to annotate a checkmate. @@ -369,6 +361,7 @@ We were able to find several failures: However, \texttt{chesshs} allows further moves. \end{description} +\clearpage \begin{thebibliography}{8} \bibitem[C] {C} \url{http://haskell.org/cabal} \bibitem[FIDE] {FIDE} \url{http://fide.com/fide/handbook.html} @@ -381,4 +374,32 @@ We were able to find several failures: \bibitem[SUT] {SUT} \url{http://hackage.haskell.org/package/chesshs} \end{thebibliography} +\appendix +\section{Listings} + +\begin{longlisting} + \inputminted{text}{../test/database/m0.in} + \caption{An example of Portable Game Notation (PGN).\label{L:PGNE}} +\end{longlisting} + +\begin{longlisting} + \inputminted{haskell}{../src/runchess.hs} + \caption{Wrapper Implementation} \label{L:WI} +\end{longlisting} + +\begin{longlisting} + \inputminted{text}{../test/database/m2.out} + \caption{An example output file.\label{L:OUTE}} +\end{longlisting} + +\begin{longlisting} + \inputminted{bash}{../test/test.sh} + \caption{Automated Test script\label{L:ATS}} +\end{longlisting} + +\begin{longlisting} + \inputminted{text}{assignment1-test-run.txt} + \caption{Results of the test run.\label{L:TR}} +\end{longlisting} + \end{document} |