Added framework project 1

author: Camil Staps 2016-03-02 21:37:50 +0100
committer: Camil Staps 2016-03-02 21:39:52 +0100
commit: 436f26b4eb1b38089396374876908fdb06d3c015 (patch)
tree: d599191368ff027e98a704051dfa04a20d6645d1 /project1/proj1_s4498062
parent: Assignment 2 (diff)
15 files changed, 554 insertions, 0 deletions
diff --git a/project1/proj1_s4498062/README.md b/project1/proj1_s4498062/README.md
new file mode 100644
index 0000000..adc2a2d
--- /dev/null
+++ b/project1/proj1_s4498062/README.md
@@ -0,0 +1,63 @@
+# Project 1 HTTP Server Framework
+
+## Description
+
+This repo contains a framework for the first project.
+It provides a class structure and several utility wrappers for the Python Standard Library (so that you don't have to dig in the Python Docs).
+The project can be completed by filling in the empty methods
+
+## File Structure
+
+* proj_sn1_sn2
+    * content
+        * test
+            * index.html
+    * webhttp
+        * composer.py
+        * \_\_init\_\_.py
+        * message.py
+        * resource.py
+        * parser.py
+        * server.py
+    * webserver.py
+    * webtests.py
+
+The content directory contains the content for the website.
+The test subfolder is meant for resources that are used by the tests which are defined in webtests.py.
+
+The webhttp directory contains a package for HTTP.
+Most methods are unimplemented.
+The file message.py contains classes for HTTP messages.
+The base class Message implements a basic HTTP message, which is specialized by Request and Response, which are for HTTP requests and HTTP responses respectively.
+The file resource.py contains a handler class for resources.
+The file parser.py contains classes for parsing HTTP requests and responses.
+The file composer.py contains a class for composing responses to request.
+The file server.py contains the main HTTP server.
+
+The webserver.py file is Python file for starting the HTTP server, this file is fully implemented.
+
+Finally webtests.py contains the tests for your HTTP server.
+You manually have to start the HTTP server before running the tests.
+Currently only one of the tests is implemented, you will have to implement the other tests.
+
+## Suggested Implementation Order
+
+1. Implement "run" in the class "Server" in "webhttp/server.py". This method should listen for incoming connections and create to a "ConnectionHandler" to handle the connection.
+2. Implement "handle_connection" in the class "ConnectionHandler" in "webhttp/server.py". For now this method should receive a response from a client, send "Hello World" back and close the connection. Test that this works using the Python shell.
+3. Implement "parse_requests" in the class "RequestParser" in "webhttp/parser.py". You can test your implementation by using to parse a few Requests in the Python shell.
+4. Implement "\_\_str\_\_" for "Message", "Response" and "Request" in "webhttp/message.py". This function should return a string representation of the message according to the RFC. In order to test this you should create a few Responses and Requests in the Python shell and test if they comply with the RFC.
+5. Reimplement "handle_connection" using "Response", "Request", "RequestParser" and "ResponseComposer".
+6. Implement "parse_response" in "ResponseParser" in "webhttp/parser.py". At this point you should be able to pass the test in "webtests.py".
+7. Replace the stub code in "compose_response" in "ResponseComposer" in "webhttp/composer.py". The composer should now be able to create the correct response to a request. You can ignore persistent connections and client side caching for now, but the response should have the right code and body.
+8. Write additional tests in "webtests.py" for the following scenarios:
+    * GET for a single resource that does not exist
+    * GET for a directory with an existing index.html file
+    * GET for a directory with non-existing index.html file
+Your code should be able to pass these tests at this point.
+9. Implement client side caching using ETags in "compose_response" and "generate_etag" in the class "Resource" in "resource.py". The "os.stat" module might be useful for generating an ETag. You should also implement the following test (which your server should pass at this point):
+    * GET for an existing resource followed by a GET for that same resource, with caching utilized on the client/tester side.
+10. Implement persistent connections in "compose_response" and "handle_connection", and implement the following tests:
+    * multiple GETs over the same (persistent) connection with the last GET prompting closing the connection, the connection should be closed.
+    * multiple GETs over the same (persistent) connection, followed by a wait during which the connection times out, the connection should be closed.
+11. Implement content encoding. You may need to add extra methods to class "Resource" for this. You should also implement the following test:
+    * GET which requests an existing resource gzip encoding, which is accepted by the server.
diff --git a/project1/proj1_s4498062/content/test/index.html b/project1/proj1_s4498062/content/test/index.html
new file mode 100644
index 0000000..a5d8c3d
--- /dev/null
+++ b/project1/proj1_s4498062/content/test/index.html
@@ -0,0 +1,9 @@
+<html>
+    <head>
+        <title> Index Page </title>
+    </head>
+    <body>
+        <h1> Index Page </h1>
+        <p> Test Text </p>
+    </body>
+</html>
diff --git a/project1/proj1_s4498062/documentation.txt b/project1/proj1_s4498062/documentation.txt
new file mode 100644
index 0000000..630fc88
--- /dev/null
+++ b/project1/proj1_s4498062/documentation.txt
@@ -0,0 +1,5 @@
+Don't forget to document your implementation:
+-> language + external libraries used (if any)
+-> control flow with headers/status codes considered for each requirement (GET, persistent connections, ETag, encoding)
+-> concurrency, hashing, resource encoding
+-> challenges (if any)
+\ No newline at end of file
diff --git a/project1/proj1_s4498062/webhttp/__init__.py b/project1/proj1_s4498062/webhttp/__init__.py
new file mode 100644
index 0000000..d501a0e
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/__init__.py
@@ -0,0 +1,9 @@
+"""HTTP Package
+
+This package contains the following modules:
+    * message: Module for HTTP responses/requests
+    * composer: Module for composing responses to requests
+    * parser: Module for parsing HTTP responses/requests
+    * util: Module with utility functions
+    * server: Module which contains a HTTP server
+"""
diff --git a/project1/proj1_s4498062/webhttp/__init__.pyc b/project1/proj1_s4498062/webhttp/__init__.pyc
new file mode 100644
index 0000000..d86ce3e
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/__init__.pyc
diff --git a/project1/proj1_s4498062/webhttp/composer.py b/project1/proj1_s4498062/webhttp/composer.py
new file mode 100644
index 0000000..dfac7a1
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/composer.py
@@ -0,0 +1,50 @@
+""" Composer for HTTP responses
+
+This module contains a composer, which can compose responses to
+HTTP requests from a client.
+"""
+
+import time
+
+import webhttp.message
+import webhttp.resource
+
+
+class ResponseComposer:
+    """Class that composes a HTTP response to a HTTP request"""
+
+    def __init__(self, timeout):
+        """Initialize the ResponseComposer
+        
+        Args:
+            timeout (int): connection timeout
+        """
+        self.timeout = timeout
+    
+    def compose_response(self, request):
+        """Compose a response to a request
+        
+        Args:
+            request (webhttp.Request): request from client
+
+        Returns:
+            webhttp.Response: response to request
+
+        """
+        response = webhttp.message.Response()
+
+        # Stub code
+        response.code = 200
+        response.set_header("Content-Length", 4)
+        response.set_header("Connection", "close")
+        response.body = "Test"
+
+        return response
+
+    def make_date_string(self):
+        """Make string of date and time
+        
+        Returns:
+            str: formatted string of date and time
+        """
+        return time.strftime("%a, %d %b %Y %H:%M:%S", time.localtime())
diff --git a/project1/proj1_s4498062/webhttp/message.py b/project1/proj1_s4498062/webhttp/message.py
new file mode 100644
index 0000000..fb47f88
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/message.py
@@ -0,0 +1,90 @@
+"""HTTP Messages
+
+This modules contains classes for representing HTTP responses and requests.
+"""
+
+reasondict = {
+    # Dictionary for code reasons
+    # Format: code : "Reason"
+    500 : "Internal Server Error"
+}
+
+
+class Message(object):
+    """Class that stores a HTTP Message"""
+
+    def __init__(self):
+        """Initialize the Message"""
+        self.version = "HTTP/1.1"
+        self.startline = ""
+        self.body = ""
+        self.headerdict = dict()
+        
+    def set_header(self, name, value):
+        """Add a header and its value
+        
+        Args:
+            name (str): name of header
+            value (str): value of header
+        """
+        self.headerdict[name] = value
+        
+    def get_header(self, name):
+        """Get the value of a header
+        
+        Args:
+            name (str): name of header
+
+        Returns:
+            str: value of header, empty if header does not exist
+        """
+        if name in self.headerdict:
+            return self.headerdict[name]
+        else:
+            return ""
+        
+    def __str__(self):
+        """Convert the Message to a string
+        
+        Returns:
+            str: representation the can be sent over socket
+        """
+        message = ""
+        return message
+
+
+class Request(Message):
+    """Class that stores a HTTP request"""
+
+    def __init__(self):
+        """Initialize the Request"""
+        super(Request, self).__init__()
+        self.method = ""
+        self.uri = ""
+        
+    def __str__(self):
+        """Convert the Request to a string
+
+        Returns:
+            str: representation the can be sent over socket
+        """
+        self.startline = ""
+        return super(Request, self).__str__()
+        
+
+class Response(Message):
+    """Class that stores a HTTP Response"""
+
+    def __init__(self):
+        """Initialize the Response"""
+        super(Response, self).__init__()
+        self.code = 500
+    
+    def __str__(self):
+        """Convert the Response to a string
+
+        Returns:
+            str: representation the can be sent over socket
+        """
+        self.startline = ""                                      
+        return super(Response, self).__str__()
diff --git a/project1/proj1_s4498062/webhttp/message.pyc b/project1/proj1_s4498062/webhttp/message.pyc
new file mode 100644
index 0000000..432142c
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/message.pyc
diff --git a/project1/proj1_s4498062/webhttp/parser.py b/project1/proj1_s4498062/webhttp/parser.py
new file mode 100644
index 0000000..f3809c3
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/parser.py
@@ -0,0 +1,66 @@
+"""HTTP response and request parsers
+
+This module contains parses for HTTP response and HTTP requests.
+"""
+
+import webhttp.message
+
+
+class RequestParser:
+    """Class that parses a HTTP request"""
+
+    def __init__(self):
+        """Initialize the RequestParser"""
+        pass
+        
+    def parse_requests(self, buff):
+        """Parse requests in a buffer
+
+        Args:
+            buff (str): the buffer contents received from socket
+
+        Returns:
+            list of webhttp.Request
+        """
+        requests = split_requests(buff)
+        
+        http_requests = []
+        for request in requests:
+            http_request = webhttp.message.Request()
+            http_requests.append(http_request)
+        
+        return http_requests
+
+    def split_requests(self, buff):
+        """Split multiple requests
+        
+        Arguments:
+            buff (str): the buffer contents received from socket
+
+        Returns:
+            list of str
+        """
+        requests = buff.split('\r\n\r\n')
+        requests = filter(None, requests)
+        requests = [r + '\r\n\r\n' for r in requests]
+        requests = [r.lstrip() for r in requests]
+        return requests
+
+
+class ResponseParser:
+    """Class that parses a HTTP response"""
+    def __init__(self):
+        """Initialize the ResponseParser"""
+        pass
+
+    def parse_response(self, buff):
+        """Parse responses in buffer
+
+        Args:
+            buff (str): the buffer contents received from socket
+
+        Returns:
+            webhttp.Response
+        """
+        response = webhttp.message.Response()
+        return response
diff --git a/project1/proj1_s4498062/webhttp/parser.pyc b/project1/proj1_s4498062/webhttp/parser.pyc
new file mode 100644
index 0000000..c5d9ced
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/parser.pyc
diff --git a/project1/proj1_s4498062/webhttp/resource.py b/project1/proj1_s4498062/webhttp/resource.py
new file mode 100644
index 0000000..dc20067
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/resource.py
@@ -0,0 +1,86 @@
+"""Resources
+
+This module contains a handler class for resources.
+"""
+
+import os
+import mimetype
+import urlparse
+
+
+class FileExistError(Exception):
+    """Exception which is raised when file does not exist"""
+    pass
+
+
+class FileAccessError(Exception):
+    """Exception which is raised when file exists, but cannot be accessed"""
+    pass
+
+
+class Resource:
+    """Class for representing a Resource (file)"""
+
+    def __init__(self, uri):
+        """Initialize the resource"
+
+        Raises:
+            FileExistError: if resource does not exist
+            FileAccessError: if resource exists, but cannot be accessed
+
+        Args:
+            uri (str): Uniform Resource Identifier
+        """
+        self.uri = uri
+        out = urlparse.urlparse(uri)
+        self.path = os.path.join("content", out.path.lstrip("/"))
+        if os.path.isdir(self.path):
+            self.path = os.path.join(self.path, "index.html")
+        if not os.path.isfile(self.path):
+            raise FileExistError
+        if not os.access(self.path, os.R_OK):
+            raise FileAccessError
+
+    def generate_etag(self):
+        """Generate the ETag for the resource
+
+        Returns:
+            str: ETag for the resource
+        """
+        stat = os.stat(self.path)
+        etag = ""
+        return etag
+
+    def get_content(self):
+        """Get the contents of the resource
+        
+        Returns:
+            str: Contents of the resource
+        """
+        return open(self.path).read()
+
+    def get_content_type(self):
+        """Get the content type, i.e "text/html"
+
+        Returns:
+            str: type of content in the resource
+        """
+        mimetype = mimetypes.guess_type(self.path)
+        return mimetype[0]
+
+    def get_content_encoding(self):
+        """Get the content encoding, i.e "gzip"
+
+        Returns:
+            str: encoding used for the resource
+        """
+        mimetype = mimetypes.guess_type(self.path)
+        return mimetype[1]
+
+    def get_content_length(self):
+        """Get the length of the resource
+
+        Returns:
+            int: length of resource in bytes
+        """
+        return os.path.getsize(self.path)
diff --git a/project1/proj1_s4498062/webhttp/server.py b/project1/proj1_s4498062/webhttp/server.py
new file mode 100644
index 0000000..b540ac4
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/server.py
@@ -0,0 +1,59 @@
+"""HTTP Server
+
+This module contains a HTTP server
+"""
+
+import threading
+import socket
+
+
+class ConnectionHandler(threading.Thread):
+    """Connection Handler for HTTP Server"""
+
+    def __init__(self, conn_socket, addr, timeout):
+        """Initialize the HTTP Connection Handler
+        
+        Args:
+            conn_socket (socket): socket used for connection with client
+            addr (str): ip address of client
+            timeout (int): seconds until timeout
+        """
+        super(HTTPConnectionHandler, self).__init__()
+        self.daemon = True
+        self.conn_socket = conn_socket
+        self.addr = addr
+        self.timeout = timeout
+    
+    def handle_connection(self):
+        """Handle a new connection"""
+        pass
+        
+    def run(self):
+        """Run the thread of the connection handler"""
+        self.handle_connection()
+        
+
+class Server:
+    """HTTP Server"""
+
+    def __init__(self, hostname, server_port, timeout):
+        """Initialize the HTTP server
+        
+        Args:
+            hostname (str): hostname of the server
+            server_port (int): port that the server is listening on
+            timeout (int): seconds until timeout
+        """
+        self.hostname = hostname
+        self.server_port = server_port
+        self.timeout = timeout
+        self.done = False
+    
+    def run(self):
+        """Run the HTTP Server and start listening"""
+        while not self.done:
+            pass
+    
+    def shutdown(self):
+        """Safely shut down the HTTP server"""
+        self.done = True
diff --git a/project1/proj1_s4498062/webhttp/server.pyc b/project1/proj1_s4498062/webhttp/server.pyc
new file mode 100644
index 0000000..7981ff7
--- /dev/null
+++ b/project1/proj1_s4498062/webhttp/server.pyc
diff --git a/project1/proj1_s4498062/webserver.py b/project1/proj1_s4498062/webserver.py
new file mode 100644
index 0000000..e518810
--- /dev/null
+++ b/project1/proj1_s4498062/webserver.py
@@ -0,0 +1,23 @@
+import argparse
+import webhttp.server
+
+# Create and start the HTTP Server
+# Use `python webserver.py --help` to display command line options
+if __name__ == '__main__':
+    # Parse command line arguments
+    parser = argparse.ArgumentParser(description="HTTP Server")
+    parser.add_argument("-a", "--address", type=str, default="localhost")
+    parser.add_argument("-p", "--port", type=int, default=8001)
+    parser.add_argument("-t", "--timeout", type=int, default=15)
+    args = parser.parse_args()
+
+    # Start server
+    server = webhttp.server.Server(args.address, args.port, args.timeout)
+    try:
+        server.run()
+    except KeyboardInterrupt:
+        server.shutdown()
+        print ("")
+            
+
+
diff --git a/project1/proj1_s4498062/webtests.py b/project1/proj1_s4498062/webtests.py
new file mode 100644
index 0000000..9b0cdbf
--- /dev/null
+++ b/project1/proj1_s4498062/webtests.py
@@ -0,0 +1,94 @@
+import unittest
+import socket
+import sys
+
+import webhttp.message
+import webhttp.parser
+
+
+portnr = 8001
+
+
+class TestGetRequests(unittest.TestCase):
+    """Test cases for GET requests"""
+
+    def setUp(self):
+        """Prepare for testing"""
+        self.client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.client_socket.connect(("localhost", portnr))
+        self.parser = webhttp.parser.ResponseParser()
+
+    def tearDown(self):
+        """Clean up after testing"""
+        self.client_socket.shutdown(socket.SHUT_RDWR)
+        self.client_socket.close()
+
+    def test_existing_file(self):
+        """GET for a single resource that exists"""
+        # Send the request
+        request = webhttp.message.Request()
+        request.method = "GET"
+        request.uri = "/test/index.html"
+        request.set_header("Host", "localhost:{}".format(portnr))
+        request.set_header("Connection", "close")
+        self.client_socket.send(str(request))
+
+        # Test response
+        message = self.client_socket.recv(1024)
+        response = self.parser.parse_response(message)
+        self.assertEqual(response.code, 200)
+        self.assertTrue(response.body)
+
+    def test_nonexistant_file(self):
+        """GET for a single resource that does not exist"""
+        pass
+
+    def test_caching(self):
+        """GET for an existing single resource followed by a GET for that same
+        resource with caching utilized on the client/tester side
+        """
+        pass
+
+    def test_extisting_index_file(self):
+        """GET for a directory with an existing index.html file"""
+        pass
+
+    def test_nonexistant_index_file(self):
+        """GET for a directory with a non-existant index.html file"""
+        pass
+
+    def test_persistent_close(self):
+        """Multiple GETs over the same (persistent) connection with the last
+        GET prompting closing the connection, the connection should be closed.
+        """
+        pass
+
+    def test_persistent_timeout(self):
+        """Multiple GETs over the same (persistent) connection, followed by a
+        wait during which the connection times out, the connection should be
+        closed.
+        """
+        pass
+
+    def test_encoding(self):
+        """GET which requests an existing resource using gzip encodign, which
+        is accepted by the server.
+        """
+        pass
+
+
+if __name__ == "__main__":
+    # Parse command line arguments
+    import argparse
+    parser = argparse.ArgumentParser(description="HTTP Tests")
+    parser.add_argument("-p", "--port", type=int, default=8001)
+    
+    # Arguments for the unittest framework
+    parser.add_argument('unittest_args', nargs='*')
+    args = parser.parse_args()
+    
+    # Only pass the unittest arguments to unittest
+    sys.argv[1:] = args.unittest_args
+
+    # Start test suite
+    unittest.main()
author	Camil Staps	2016-03-02 21:37:50 +0100
committer	Camil Staps	2016-03-02 21:39:52 +0100
commit	436f26b4eb1b38089396374876908fdb06d3c015 (patch)
tree	d599191368ff027e98a704051dfa04a20d6645d1 /project1/proj1_s4498062
parent	Assignment 2 (diff)