diff options
| -rw-r--r-- | assignments/assignment1.tex | 93 |
1 files changed, 75 insertions, 18 deletions
diff --git a/assignments/assignment1.tex b/assignments/assignment1.tex index c9002c6..639635e 100644 --- a/assignments/assignment1.tex +++ b/assignments/assignment1.tex @@ -4,6 +4,7 @@ \usepackage{enumerate} \usepackage[hidelinks]{hyperref} \usepackage{minted} +\usepackage{skak} \usepackage{tikz} \usetikzlibrary{arrows, matrix, positioning} @@ -32,24 +33,6 @@ \section{Test Plan} -\section{Test goals} -\begin{enumerate} - \setcounter{enumi}{4} - \item - The documentation of the SUT is defined in two parts. - The first part, the chess part, is defined by the rules of the FIDE.\footnote{\url{http://fide.com/fide/handbook.html}} - %TODO: Summary - The second part, the PGN notation, is defined by the PGN standard.\footnote{\url{http://saremba.de/chessgml/standards/pgn/pgn-complete.htm}} - %TODO: Summary - \item - The parts of the SUT that will be tested will be the entire library and the individual components thereof. - This implies that PGN parsing will be tested, as well as the verification part of the library that determines if a move is legal. - The function that determines the checkmate position of the game will also be tested. - This means, explicitly, that the timing rules, and any other rules of the game of Chess, will not be tested. - %TODO: Manual/Automated testing - \item - Functionality and Reliability. -\end{enumerate} \subsection*{System Under Test (SUT) Description} The chosen SUT is the {\tt chesshs} Haskell package. \cite{SUT} It should be able to run in any software environment and hardware platform which supports Haskell. The latest version of the SUT, since the 13\textsuperscript{th} of March 2015, is {\tt 0.2.1}. @@ -63,6 +46,64 @@ The SUT is used by providing it with, via the standard input, a text file contai As is common with everything software, the SUT is likely to contain faults. However, the risks surrounding the program are near zero due to the very low impact of the program faulting. As of our expectation, the most ``risky" scenario for the SUT is at a notable Chess competition. In the case the program does not behave as intended, it is almost certain that a human referee would take over its place. Furthermore, we expect such a system to be used as a supporting tool for match officials and not as a replacement for regulation. \section{Test method} +\subsection*{Test Goals} + +\paragraph{Specification} + +The specification of the SUT is for the most part similar to the input-output behavior described in \S\ref{P:ERSUT}. Because the SUT is strongly related to Chess, many finer details of its specification can be derived from documentation of the world of Chess. + +\subparagraph{F\'ed\'eration Internationale Des Echecs (FIDE)} Due to the popularity of Chess, most people are familiar with how it is played. Hence, a summary of the rules would be excesive. However, due to the competitve nature of the game, the World Chess Federation describes the rules painfully precise. \cite{FIDE} + +\subparagraph{Portable Game Notation (PGN)} Many Chess matches are recorded with the PGN. \cite{PGN} As some of the test cases consist of text files describing Chess games in PGN, one of our test cases is included in listing \ref{L:PGNE} as an example for the notation. + +\begin{listing} + \inputminted{text}{../test/database/m0.in} + \caption{Portable Game Notation (PGN) Example} \label{L:PGNE} +\end{listing} + +PGN consists of two parts: first, the tag pairs and, second, the movetext. + +The tag pairs encode properties of the game of Chess in question, such as the name of the tournament or match event, the player of the white pieces and the player of the black pieces, and the result of the game. It is possible to add self defined tag pairs. During our testing, we have added the name of the test with a \mintinline{text}{TestName} tag pair. Tag pairs always follow the structure + +\begin{minted}{text} + ['Tag Name' "'Tag Value'"] +\end{minted} + +where \mintinline{text}{'Tag Name'} holds the name of the tag (\mintinline{text}{Event}, \mintinline{text}{White}, \mintinline{text}{Black}, \mintinline{text}{Result}, ...) and \mintinline{text}{'Tag Value'} the value of the tag (\mintinline{text}{Third Rosenwald Trophy}, \mintinline{text}{Donald Byrne}, \mintinline{text}{Robert James Fischer}, \mintinline{text}{0-1}, ...). + +The movetext encodes the actual game of Chess in question. It mainly consists of the moves made during the match, which are encode with + +\begin{minted}{text} + 'Round Number'. 'White's Move' 'Black's Move' +\end{minted} + +where each player's move is recorded in the Standard Algebraic Notation (SAN). + +\subparagraph{Standard Algebraic Notation (SAN)} + +Most moves are recorded with the SAN, which adheres to the following structure. \cite{SAN} + +\begin{minted}{text} + 'Piece Identifier'x?'End Position' +\end{minted} + +\mintinline{text}{'Piece Identifier'} denotes the piece being moved by a single capital letter: \mintinline{text}{B} (Bishop), \mintinline{text}{K} (King), \mintinline{text}{N} (Knight), \mintinline{text}{Q} (Queen) and \mintinline{text}{R} (Rook). Pawns either do not have an abbreviation or are denoted with \mintinline{text}{P}. In the case the abbreviation alone is not sufficient to identify the piece being moved, (partial) initial position information, according to the notation explained in figure \ref{F:SN}, is added to solve ambiguity. \mintinline{text}{'Piece Identifier'} is followed by \mintinline{text}{x} only if the move resulted in a capture. \mintinline{text}{'End Position'} records the final position of the moved piece using the notation explained in figure \ref{F:SN}. Additional notation exists to describe more specialized aspects of Chess moves. For example: \mintinline{text}{+} is appended to checking moves and \mintinline{text}{O-O} alone denotes King side Castling moves. + +\begin{figure} + \centering + \newgame + \showboard + \caption{Square Notation: Each square is noted by its file (column letter) and rank (row number).} \label{F:SN} +\end{figure} + +\paragraph{Testing Scope} + +The SUT consists of a wrapper and the {\tt chesshs} library which the wrapper uses. + +In order to transform the {\tt chesshs} package into a system with a command line interface, we have written a Haskell program which interacts with the library in such a way that the desired input-output behavior as described in \S\ref{P:ERSUT} of the SUT is achieved. In essence, it ``wraps" around the library to make it interactive. Hence the term `wrapper'. + +Due to its strong influence on the behavior of the SUT, we will also be testing the wrapper. Nonetheless, testing will still focus mainly on the {\tt chesshs} package and its individual components, as this is the (sub)system of interest. Specifically, we will be testing + \begin{enumerate} \setcounter{enumi}{7} \item @@ -81,8 +122,22 @@ As is common with everything software, the SUT is likely to contain faults. Howe Use case testing is also very hard to test, given that Chess.hs is a library, no user will likely every interact with the library directly. Path coverage and condition coverage are not implemented in Quickcheck, making it substantially harder to implement. \end{enumerate} + \item the wrapper, \label{EI:W} + \item PGN parsing, \label{EI:PGNP} + \item move legality verification and \label{EI:MLV} + \item checkmate position determination. \label{EI:CPD} \end{enumerate} +This means that any rule of the game of Chess not implemented in the {\tt chesshs} library, such as timing, can and will not be tested. + +The input-output specification in \S\ref{P:ERSUT} has effect on \ref{EI:W}, the PGN specification has effect on \ref{EI:PGNP} and the FIDE Chess rules have an effect on \ref{EI:MLV} and \ref{EI:CPD}. \cite{PGN, FIDE} + +Because every system component will be tested both manually and automatically, no special distinctions with regards to the system elements and test methods are necessary. \\ + +Functionality is the most relevant quality characteristic of the tests. We would like to have a reasonable amount of certainty that the SUT behaves as specified. Reliability is also relevant to the SUT. Because it is a library, many other systems will depend on its continuous availability, even after a ``crash" of some sort. For the same reason, usability is also relevant to the SUT. Packages are components of a programming language that will be \emph{used} by other systems, hence it is paramount for the library to have an \emph{``usable"} interface. In short, many of the quality characteristics that apply to libraries, will also apply neatly to the SUT. \\ + +White box and gray box testing would look somewhat similar when testing the SUT, due to the fact that the system is a package. Gray box testing would only look at the components available to other programs, that is, the exported elements. White box testing would also look at every component of the library including their implementation. Gray box testing would consider every component as a black box, that is, it will only focus on testing if each component adheres to its related specification. White box testing would consider every component by its literal code. In conclusion, white box testing would focus on the actual implementation of the library's functionality and gray box testing would only focus on the library's exported component's functionality without considering any actual code. + \begin{figure} \centering \begin{tikzpicture}[ @@ -110,9 +165,11 @@ As is common with everything software, the SUT is likely to contain faults. Howe \begin{thebibliography}{8} \bibitem[C] {C} \url{http://haskell.org/cabal} + \bibitem[FIDE] {FIDE} \url{http://fide.com/fide/handbook.html} \bibitem[GHC] {GHC} \url{http://haskell.org/ghc} \bibitem[HP] {HP} \url{http://haskell.org/platform} \bibitem[PGN] {PGN} \url{http://saremba.de/chessgml/standards/pgn/pgn-complete.htm} + \bibitem[SAN] {SAN} \url{http://cfajohnson.com/chess/SAN} \bibitem[SUT] {SUT} \url{http://hackage.haskell.org/package/chesshs} \end{thebibliography} |