1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
|
\documentclass{scrartcl}
\usepackage{listings}
\usepackage{enumerate}
\title{Testing Techniques\\Assignment 1}
\author{Ren\'e den Hertog - s1007823\\Camil Staps - s4498062\\Erin van der Veen - s4431200}
\begin{document}
\maketitle
\section{SUT Description}
\begin{enumerate}
\item
\begin{description}
\item[Name] Chess.hs
\item[Software Environment] Linux, and any other environment supported by Haskell
\item[Hardware Platform] Desktop PC, and any other platform supported by Haskell
\item[Latest Version Number] 0.2.1
\end{description}
\item
Starting the SUT requires being able to compile haskell, typically through the GHC, and having the Chess.hs dependency installed.
Installing Chess.hs can be done through cabal, using the \lstinline|cabal install chesshs| command.
Compiling and running is then done through \lstinline|make run|.
\item
\begin{enumerate}[a)]
\item The SUT needs to be provided with a PGN file via stdin.
\item Either the result of the game is output to stdout or the SUT outputs an error.
\end{enumerate}
\item
The SUT does have a decent chance of containing faults, but the risk is near zero due to the low impact of the of these faults.
\end{enumerate}
\section{Test goals}
\begin{enumerate}
\setcounter{enumi}{4}
\item
The documentation of the SUT is defined in two parts.
The first part, the chess part, is defined by the rules of the FIDE.\footnote{fide.com/fide/handbook.html}
%TODO: Summary
The second part, the PGN notation, is defined by the PGN standard.\footnote{saremba.de/chessgml/standards/pgn/pgn-complete.htm}
%TODO: Summary
\item
The parts of the SUT that will be tested will be the entire library and the individual components thereof.
This implies that PGN parsing will be tested, as well as the verification part of the library that determines if a move is legal.
The function that determines the checkmate position of the game will also be tested.
This means, explicitly, that the timing rules, and any other rules of the game of Chess, will not be tested.
%TODO: Manual/Automated testing
\item
Functionality and Reliability.
\end{enumerate}
\section{Test method}
\begin{enumerate}
\setcounter{enumi}{7}
\item
Unit, Integration. The scale and the number of sub-components of the project limits us to these types of tests.
\item
Mostly Black box testing, for the input/output of entire games in pgn notation.
%TODO: Fix sentence
The functions of the library will also be tested using a White box model.
\item
\begin{enumerate}[a)]
\item
For test generation of the black box testing, we will use a mixture of error guessing and the parsing of PGN files in a database.
White box testing will be done through statement coverage and branch/condition coverage.
\item
Equivalence partitioning and boundary value analysis is hard to implement given our SUT, given that all possible chess games are nearly impossible to enumerate.
Use case testing is also very hard to test, given that Chess.hs is a library, no user will likely every interact with the library directly.
Path coverage and condition coverage are not implemented in Quickcheck, making it substantially harder to implement.
\end{enumerate}
\end{enumerate}
\section{Test Environment}
\begin{enumerate}
\setcounter{enumi}{10}
\item
A wrapper written by us will be used to interact with the SUT.
\item
Tests will be automatically performed using a combination of bash and Quickcheck.
\item
\begin{enumerate}[a)]
\item
The software environment GHC 8, Quickcheck 2.10, Linux Debian (derived).
The hardware environment is a set of three amd64 laptops.
\item
A Git repository, the results will be stored in individual files.
\end{enumerate}
\end{enumerate}
\end{document}
|
