--- /dev/null
+{
+ "metadata": {
+ "name": ""
+ },
+ "nbformat": 3,
+ "nbformat_minor": 0,
+ "worksheets": [
+ {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Testing Exercises\n",
+ "\n",
+ "The following exercises do not contain solutions. Yet. Instead, we will be\n",
+ "asking you to submit your solutions to these exercises and then we will post up\n",
+ "solutions at the start of next week. We encourage you to discuss your approaches\n",
+ "or solutions on the course forum!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Exercise 1: Mileage\n",
+ "\n",
+ "The function 'convert_mileage' converts miles per gallon (US style) to\u00c2 liters\n",
+ "per 100 km (metric style):"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "gal_to_litre = 3.78541178\n",
+ "mile_to_km = 1.609344\n",
+ " \n",
+ "def convert_mileage(mpg):\n",
+ " '''Converts miles per gallon to liters per 100 km'''\n",
+ " litres_per_100_km = 100 / mpg / mile_to_km * gal_to_litre\n",
+ " return litres_per_100_km"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Create a subdirectory in your version control directory called `testing-exercises`, then\n",
+ "copy this function into a file in that directory called `mileage.py`. Add more\n",
+ "code to that file to repeatedly ask the user for a mileage in miles per gallon,\n",
+ "and output the mileage in liters per 100 km, until the user enters the string\n",
+ "`q`. You will need to use the `float()` function to convert from string to a\n",
+ "floating point number. Use the '`if __name__ == \"__main__\":`' trick to ensure\n",
+ "that the module can be imported without executing your testing code.\n",
+ "\n",
+ "1. Copy `mileage.py` to create `tryexcept.py` Add a try/except block to the new\n",
+ "program to display a helpful message instead of crashing when users enter\n",
+ "invalid input (such as the number \"0\" or the name of their favorite hockey\n",
+ "team). \n",
+ "\n",
+ "2. Reading the function again, you realize that accepting 0 or negative values\n",
+ "make no sense and should be reported as an error. Look at the exceptions defined\n",
+ "in the `exceptions` module (use the built-in `help(...)` or `dir(...)`\n",
+ "functions) and decide which of Python's built-in exceptions is most appropriate\n",
+ "to use for invalid input. Create a copy of 'tryexcept.py' called 'raiser.py'\n",
+ "that raises this exception; modify the main body of your program to catch it;\n",
+ "and add a comment inside the file explaining why you chose the exception you\n",
+ "did. (Note: you have to call this file `raiser.py`, not `raise.py` because\n",
+ "'import raise' is an error. \u00c2 Can you see why?)\n",
+ "\n",
+ "3. [According to Google](http://www.google.ca/search?q=20+miles+per+gallon+in+litres+per+100+km&gbv=1),\n",
+ "20 miles per gallon are equivalent to 11.7607292 liters per 100 km. Use these\n",
+ "values to write a unit test. Keep in mind that these floating values are subject\n",
+ "to truncation and rounding errors. Save the test case in a file called\n",
+ "`test_mileage.py` and run it using the `nosetests` command. \u00c2 Note:\n",
+ "`test_mileage.py` should use '`from raiser import convert_mileage`' to get the\n",
+ "final version of your mileage conversion function.\n",
+ "\n",
+ "4. Now add a second test case, for 40 miles per gallon equivalent to 5.88036458\n",
+ "liters per 100 km and run the tests again. \u00c2 Unless you have already fixed the\n",
+ "error that was present in the initial function, your test should fail. \u00c2 Find\n",
+ "and fix the error; submit your new function in a file called 'final_mileage.py'. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Exercise 2: Testing Averages\n",
+ "\n",
+ "The results of a set of experiments are stored in a file, where the _i-th_ line\n",
+ "stores the results of the _i-th_ experiment as a comma-separated list of\n",
+ "integers. A student is assigned the task of finding the experiment with the\n",
+ "smallest average value. She writes the following code: "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def avg_line(line):\n",
+ " values = line.split(',')\n",
+ " count = 0\n",
+ " total = 0\n",
+ " for value in values:\n",
+ " total += int(value)\n",
+ " count += 1\n",
+ " return total / count\n",
+ " \n",
+ "def min_avg(file_name):\n",
+ " contents = open(file_name)\n",
+ " averages = []\n",
+ " for (i, line) in enumerate(contents):\n",
+ " averages.append((avg_line(line), i))\n",
+ " contents.close()\n",
+ " averages.sort()\n",
+ " min_avg, experiment_number = averages[0]\n",
+ " return experiment_number"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "1. Refactor `min_avg` so that it can be tested without depending on external\n",
+ "files. Submit your code in a file called `first_averages.py`.\n",
+ "\n",
+ "2. Write Nose test cases for both functions. Consider what should happen if the\n",
+ "file is empty. Submit your tests in a file called `test_first_averages.py`.\n",
+ "Note: you may assume for now that all input is well formatted, i.e., you do\n",
+ "_not_ have to worry about empty lines, lines containing the names of hockey\n",
+ "teams, etc.\n",
+ "\n",
+ "3. The given specification is ambiguous: what should the result be if two or\n",
+ "more experiments are tied for the minimum average? Copy 'first_averages.py' to\n",
+ "create a new file 'second_averages.py'; modify it to handle this case; add a\n",
+ "comment to the top explaining what rule you decided to use; and create a file\n",
+ "'test_second_averages.py' that tests your changes.\n",
+ "\n",
+ "4. Another student proposed an alternative implementation of the min_avg\n",
+ "function:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def min_avg(file_name):\n",
+ " contents = open(file_name).readlines()\n",
+ " min_avg = avg_line(contents[0])\n",
+ " min_index = 0\n",
+ " for (i,line) in enumerate(contents):\n",
+ " current_avg = avg_line(line)\n",
+ " if current_avg <= min_avg:\n",
+ " min_avg = current_avg\n",
+ " min_index = i\n",
+ " return min_index"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This implementation also finds an experiment with the smallest average, but\n",
+ "possibly a different one than the your function. Modify your test cases so that\n",
+ "both your implementation and this one will pass. (Hint: use the 'in' operator.)\n",
+ "\n",
+ "5. One way to avoid the ambiguity of this specification is to define a\n",
+ "'min_avg_all' function instead, which returns a list with all the experiments\n",
+ "with the smallest average, and let the caller select one. Write tests for the\n",
+ "'min_avg_all' function, considering the following situations: an empty file,\n",
+ "exactly one experiment with minimum average, and more than one experiment with\n",
+ "minimum average. Keep in mind that in the last case, implementations could\n",
+ "return the list in different order. Write the tests the file \"test_averages.py\".\n",
+ "Use the same data as for the previous tests, if possible. You should use\n",
+ "variables to avoid code duplication. You don't need to implement the\n",
+ "'min_avg_all' function, but your test cases should be comprehensive enough to\n",
+ "serve as a specification for it."
+ ]
+ }
+ ],
+ "metadata": {}
+ }
+ ]
+}
\ No newline at end of file
--- /dev/null
+{
+ "metadata": {
+ "name": ""
+ },
+ "nbformat": 3,
+ "nbformat_minor": 0,
+ "worksheets": [
+ {
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Testing\n",
+ "\n",
+ "**Based on materials by Katy Huff, Rachel Slaybaugh, and Anthony Scopatz**"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## What is testing?\n",
+ "\n",
+ "Software testing is a process by which one or more expected behaviors\n",
+ "and results from a piece of software are exercised and confirmed. Well\n",
+ "chosen tests will confirm expected code behavior for the extreme\n",
+ "boundaries of the input domains, output ranges, parametric combinations,\n",
+ "and other behavioral **edge cases**."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Why test software?\n",
+ "\n",
+ "Unless you write flawless, bug-free, perfectly accurate, fully precise,\n",
+ "and predictable code *every time*, you must test your code in order to\n",
+ "trust it enough to answer in the affirmative to at least a few of the\n",
+ "following questions:\n",
+ "\n",
+ "- Does your code work?\n",
+ "- **Always?**\n",
+ "- Does it do what you think it does? ([Patriot Missile Failure](http://www.ima.umn.edu/~arnold/disasters/patriot.html))\n",
+ "- Does it continue to work after changes are made?\n",
+ "- Does it continue to work after system configurations or libraries\n",
+ " are upgraded?\n",
+ "- Does it respond properly for a full range of input parameters?\n",
+ "- What about **edge and corner cases**?\n",
+ "- What's the limit on that input parameter?\n",
+ "- How will it affect your\n",
+ " [publications](http://www.nature.com/news/2010/101013/full/467775a.html)?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Verification\n",
+ "\n",
+ "*Verification* is the process of asking, \"Have we built the software\n",
+ "correctly?\" That is, is the code bug free, precise, accurate, and\n",
+ "repeatable?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Validation\n",
+ "\n",
+ "*Validation* is the process of asking, \"Have we built the right\n",
+ "software?\" That is, is the code designed in such a way as to produce the\n",
+ "answers we are interested in, data we want, etc."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Uncertainty Quantification\n",
+ "\n",
+ "*Uncertainty Quantification* is the process of asking, \"Given that our\n",
+ "algorithm may not be deterministic, was our execution within acceptable\n",
+ "error bounds?\" This is particularly important for anything which uses\n",
+ "random numbers, eg Monte Carlo methods."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Where are tests?\n",
+ "\n",
+ "Say we have an averaging function:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def mean(numlist):\n",
+ " total = sum(numlist)\n",
+ " length = len(numlist)\n",
+ " return total/length"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Tests could be implemented as runtime *exceptions in the function*:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def mean(numlist):\n",
+ " try:\n",
+ " total = sum(numlist)\n",
+ " length = len(numlist)\n",
+ " except TypeError:\n",
+ " raise TypeError(\"The number list was not a list of numbers.\")\n",
+ " except:\n",
+ " print \"There was a problem evaluating the number list.\"\n",
+ " return total/length"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Sometimes tests are functions alongside the function definitions\n",
+ "they are testing."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def mean(numlist):\n",
+ " try:\n",
+ " total = sum(numlist)\n",
+ " length = len(numlist)\n",
+ " except TypeError:\n",
+ " raise TypeError(\"The number list was not a list of numbers.\")\n",
+ " except:\n",
+ " print \"There was a problem evaluating the number list.\"\n",
+ " return total/length\n",
+ "\n",
+ "\n",
+ "def test_mean():\n",
+ " assert mean([0, 0, 0, 0]) == 0\n",
+ " assert mean([0, 200]) == 100\n",
+ " assert mean([0, -200]) == -100\n",
+ " assert mean([0]) == 0\n",
+ "\n",
+ "\n",
+ "def test_floating_mean():\n",
+ " assert mean([1, 2]) == 1.5"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Sometimes tests live in an executable independent of the main executable.\n",
+ "\n",
+ "**Implementation File:** `mean.py`"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def mean(numlist):\n",
+ " try:\n",
+ " total = sum(numlist)\n",
+ " length = len(numlist)\n",
+ " except TypeError:\n",
+ " raise TypeError(\"The number list was not a list of numbers.\")\n",
+ " except:\n",
+ " print \"There was a problem evaluating the number list.\"\n",
+ " return total/length"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Test File:** `test_mean.py`"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "from stats import mean\n",
+ "\n",
+ "def test_mean():\n",
+ " assert mean([0, 0, 0, 0]) == 0\n",
+ " assert mean([0, 200]) == 100\n",
+ " assert mean([0, -200]) == -100\n",
+ " assert mean([0]) == 0\n",
+ "\n",
+ "\n",
+ "def test_floating_mean():\n",
+ " assert mean([1, 2]) == 1.5"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## When should we test?\n",
+ "\n",
+ "The three right answers are:\n",
+ "\n",
+ "- **ALWAYS!**\n",
+ "- **EARLY!**\n",
+ "- **OFTEN!**\n",
+ "\n",
+ "The longer answer is that testing either before or after your software\n",
+ "is written will improve your code, but testing after your program is\n",
+ "used for something important is too late.\n",
+ "\n",
+ "If we have a robust set of tests, we can run them before adding\n",
+ "something new and after adding something new. If the tests give the same\n",
+ "results (as appropriate), we can have some assurance that we didn't\n",
+ "wreak anything. The same idea applies to making changes in your system\n",
+ "configuration, updating support codes, etc.\n",
+ "\n",
+ "Another important feature of testing is that it helps you remember what\n",
+ "all the parts of your code do. If you are working on a large project\n",
+ "over three years and you end up with 200 classes, it may be hard to\n",
+ "remember what the widget class does in detail. If you have a test that\n",
+ "checks all of the widget's functionality, you can look at the test to\n",
+ "remember what it's supposed to do."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Who should test?\n",
+ "\n",
+ "In a collaborative coding environment, where many developers contribute\n",
+ "to the same code base, developers should be responsible individually for\n",
+ "testing the functions they create and collectively for testing the code\n",
+ "as a whole.\n",
+ "\n",
+ "Professionals often test their code, and take pride in test coverage,\n",
+ "the percent of their functions that they feel confident are\n",
+ "comprehensively tested."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## How are tests written?\n",
+ "\n",
+ "The type of tests that are written is determined by the testing\n",
+ "framework you adopt. Don't worry, there are a lot of choices."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Types of Tests\n",
+ "\n",
+ "**Exceptions:** Exceptions can be thought of as type of runtime test.\n",
+ "They alert the user to exceptional behavior in the code. Often,\n",
+ "exceptions are related to functions that depend on input that is unknown\n",
+ "at compile time. Checks that occur within the code to handle exceptional\n",
+ "behavior that results from this type of input are called Exceptions.\n",
+ "\n",
+ "**Unit Tests:** Unit tests are a type of test which test the fundamental\n",
+ "units of a program's functionality. Often, this is on the class or\n",
+ "function level of detail. However what defines a *code unit* is not\n",
+ "formally defined.\n",
+ "\n",
+ "To test functions and classes, the interfaces (API) - rather than the\n",
+ "implementation - should be tested. Treating the implementation as a\n",
+ "black box, we can probe the expected behavior with boundary cases for\n",
+ "the inputs.\n",
+ "\n",
+ "**System Tests:** System level tests are intended to test the code as a\n",
+ "whole. As opposed to unit tests, system tests ask for the behavior as a\n",
+ "whole. This sort of testing involves comparison with other validated\n",
+ "codes, analytical solutions, etc.\n",
+ "\n",
+ "**Regression Tests:** A regression test ensures that new code does\n",
+ "change anything. If you change the default answer, for example, or add a\n",
+ "new question, you'll need to make sure that missing entries are still\n",
+ "found and fixed.\n",
+ "\n",
+ "**Integration Tests:** Integration tests query the ability of the code\n",
+ "to integrate well with the system configuration and third party\n",
+ "libraries and modules. This type of test is essential for codes that\n",
+ "depend on libraries which might be updated independently of your code or\n",
+ "when your code might be used by a number of users who may have various\n",
+ "versions of libraries.\n",
+ "\n",
+ "**Test Suites:** Putting a series of unit tests into a collection of\n",
+ "modules creates, a test suite. Typically the suite as a whole is\n",
+ "executed (rather than each test individually) when verifying that the\n",
+ "code base still functions after changes have been made."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Elements of a Test\n",
+ "\n",
+ "**Behavior:** The behavior you want to test. For example, you might want\n",
+ "to test the fun() function.\n",
+ "\n",
+ "**Expected Result:** This might be a single number, a range of numbers,\n",
+ "a new fully defined object, a system state, an exception, etc. When we\n",
+ "run the fun() function, we expect to generate some fun. If we don't\n",
+ "generate any fun, the fun() function should fail its test.\n",
+ "Alternatively, if it does create some fun, the fun() function should\n",
+ "pass this test. The the expected result should known *a priori*. For\n",
+ "numerical functions, this is result is ideally analytically determined\n",
+ "even if the function being tested isn't.\n",
+ "\n",
+ "**Assertions:** Require that some conditional be true. If the\n",
+ "conditional is false, the test fails.\n",
+ "\n",
+ "**Fixtures:** Sometimes you have to do some legwork to create the\n",
+ "objects that are necessary to run one or many tests. These objects are\n",
+ "called fixtures as they are not really part of the test themselves but\n",
+ "rather involve getting the computer into the appropriate state.\n",
+ "\n",
+ "For example, since fun varies a lot between people, the fun() function\n",
+ "is a method of the Person class. In order to check the fun function,\n",
+ "then, we need to create an appropriate Person object on which to run\n",
+ "fun().\n",
+ "\n",
+ "**Setup and teardown:** Creating fixtures is often done in a call to a\n",
+ "setup function. Deleting them and other cleanup is done in a teardown\n",
+ "function.\n",
+ "\n",
+ "**The Big Picture:** Putting all this together, the testing algorithm is\n",
+ "often:\n",
+ "\n",
+ " setup()\n",
+ " test()\n",
+ " teardown()\n",
+ "\n",
+ "But, sometimes it's the case that your tests change the fixtures. If so,\n",
+ "it's better for the setup() and teardown() functions to occur on either\n",
+ "side of each test. In that case, the testing algorithm should be:\n",
+ "\n",
+ " setup()\n",
+ " test1()\n",
+ " teardown()\n",
+ "\n",
+ " setup()\n",
+ " test2()\n",
+ " teardown()\n",
+ "\n",
+ " setup()\n",
+ " test3()\n",
+ " teardown()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Nose: A Python Testing Framework\n",
+ "\n",
+ "The testing framework we'll discuss today is called nose. However, there are\n",
+ "several other testing frameworks available in most language. Most notably there\n",
+ "is [JUnit](http://www.junit.org/) in Java which can arguably attributed to\n",
+ "inventing the testing framework. Google also provides a [test\n",
+ "framework](code.google.com/p/googletest/) for C++ applications (note, there's\n",
+ "also [CTest](http://cmake.org/Wiki/CMake/Testing_With_CTest)). There\n",
+ "is at least one testing framework for R:\n",
+ "[testthat](http://cran.r-project.org/web/packages/testthat/index.html)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Where do nose tests live?\n",
+ "\n",
+ "Nose tests are files that begin with `Test-`, `Test_`, `test-`, or\n",
+ "`test_`. Specifically, these satisfy the testMatch regular expression\n",
+ "`[Tt]est[-_]`. (You can also teach nose to find tests by declaring them\n",
+ "in the unittest.TestCase subclasses chat you create in your code. You\n",
+ "can also create test functions which are not unittest.TestCase\n",
+ "subclasses if they are named with the configured testMatch regular\n",
+ "expression.)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Nose Test Syntax\n",
+ "\n",
+ "To write a nose test, we make assertions.\n",
+ "\n",
+ " assert should_be_true()\n",
+ " assert not should_not_be_true()\n",
+ "\n",
+ "Additionally, nose itself defines number of assert functions which can\n",
+ "be used to test more specific aspects of the code base.\n",
+ "\n",
+ " from nose.tools import *\n",
+ "\n",
+ " assert_equal(a, b)\n",
+ " assert_almost_equal(a, b)\n",
+ " assert_true(a)\n",
+ " assert_false(a)\n",
+ " assert_raises(exception, func, *args, **kwargs)\n",
+ " assert_is_instance(a, b)\n",
+ " # and many more!\n",
+ "\n",
+ "Moreover, numpy offers similar testing functions for arrays:\n",
+ "\n",
+ " from numpy.testing import *\n",
+ "\n",
+ " assert_array_equal(a, b)\n",
+ " assert_array_almost_equal(a, b)\n",
+ " # etc."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Exercise: Writing tests for mean()\n",
+ "\n",
+ "There are a few tests for the mean() function that we listed in this\n",
+ "lesson. What are some tests that should fail? Add at least three test\n",
+ "cases to this set. Edit the `test_mean.py` file which tests the mean()\n",
+ "function in `mean.py`.\n",
+ "\n",
+ "*Hint:* Think about what form your input could take and what you should\n",
+ "do to handle it. Also, think about the type of the elements in the list.\n",
+ "What should be done if you pass a list of integers? What if you pass a\n",
+ "list of strings?\n",
+ "\n",
+ "**Example**:\n",
+ "\n",
+ " $ nosetests test_mean.py"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Test Driven Development\n",
+ "\n",
+ "Test driven development (TDD) is a philosophy whereby the developer\n",
+ "creates code by **writing the tests first**. That is to say you write the\n",
+ "tests *before* writing the associated code!\n",
+ "\n",
+ "This is an iterative process whereby you write a test then write the\n",
+ "minimum amount code to make the test pass. If a new feature is needed,\n",
+ "another test is written and the code is expanded to meet this new use\n",
+ "case. This continues until the code does what is needed.\n",
+ "\n",
+ "TDD operates on the YAGNI principle (You Ain't Gonna Need It). People\n",
+ "who diligently follow TDD swear by its effectiveness. This development\n",
+ "style was put forth most strongly by [Kent Beck in\n",
+ "2002](http://www.amazon.com/Test-Driven-Development-By-Example/dp/0321146530)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### A TDD Example\n",
+ "\n",
+ "Say you want to write a std() function which computes the [Standard \n",
+ "Deviation](http://en.wikipedia.org/wiki/Standard_deviation). You\n",
+ "would - of course - start by writing the test, possibly testing a single set of \n",
+ "numbers:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "from nose.tools import assert_equal, assert_almost_equal\n",
+ "\n",
+ "def test_std1():\n",
+ " obs = std([0.0, 2.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You would *then* go ahead and write the actual function:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def std(vals):\n",
+ " # you snarky so-and-so\n",
+ " return 1.0"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "And that is it, right?! Well, not quite. This implementation fails for\n",
+ "most other values. Adding tests we see that:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def test_std1():\n",
+ " obs = std([0.0, 2.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std2():\n",
+ " obs = std([])\n",
+ " exp = 0.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std3():\n",
+ " obs = std([0.0, 4.0])\n",
+ " exp = 2.0\n",
+ " assert_equal(obs, exp)"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "These extra tests now require that we bother to implement at least a slightly \n",
+ "more reasonable function:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def std(vals):\n",
+ " # a little better\n",
+ " if len(vals) == 0:\n",
+ " return 0.0\n",
+ " return vals[-1] / 2.0"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "However, this function still fails whenever vals has more than two elements or\n",
+ "the first element is not zero. Time for more tests!"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def test_std1():\n",
+ " obs = std([0.0, 2.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std2():\n",
+ " obs = std([])\n",
+ " exp = 0.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std3():\n",
+ " obs = std([0.0, 4.0])\n",
+ " exp = 2.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std4():\n",
+ " obs = std([1.0, 3.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std5():\n",
+ " obs = std([1.0, 1.0, 1.0])\n",
+ " exp = 0.0\n",
+ " assert_equal(obs, exp)"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "At this point, we had better go ahead and try do the right thing..."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def std(vals):\n",
+ " # finally, some math\n",
+ " n = len(vals)\n",
+ " if n == 0:\n",
+ " return 0.0\n",
+ " mu = sum(vals) / n\n",
+ " var = 0.0\n",
+ " for val in vals:\n",
+ " var = var + (val - mu)**2\n",
+ " return (var / n)**0.5"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Here it becomes very tempting to take an extended coffee break or\n",
+ "possibly a power lunch. But then you remember those pesky infinite values!\n",
+ "Perhaps the right thing to do here is to just be undefined. Infinity in \n",
+ "Python may be represented by any literal float greater than or equal to 1e309."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def test_std1():\n",
+ " obs = std([0.0, 2.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std2():\n",
+ " obs = std([])\n",
+ " exp = 0.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std3():\n",
+ " obs = std([0.0, 4.0])\n",
+ " exp = 2.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std4():\n",
+ " obs = std([1.0, 3.0])\n",
+ " exp = 1.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std5():\n",
+ " obs = std([1.0, 1.0, 1.0])\n",
+ " exp = 0.0\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std6():\n",
+ " obs = std([1e500])\n",
+ " exp = NotImplemented\n",
+ " assert_equal(obs, exp)\n",
+ "\n",
+ "def test_std7():\n",
+ " obs = std([0.0, 1e4242])\n",
+ " exp = NotImplemented\n",
+ " assert_equal(obs, exp)"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This means that it is time to add the appropriate case to the function\n",
+ "itself:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "def std(vals):\n",
+ " # sequence and you shall find\n",
+ " n = len(vals)\n",
+ " if n == 0:\n",
+ " return 0.0\n",
+ " mu = sum(vals) / n\n",
+ " if mu == 1e500:\n",
+ " return NotImplemented\n",
+ " var = 0.0\n",
+ " for val in vals:\n",
+ " var = var + (val - mu)**2\n",
+ " return (var / n)**0.5"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Quality Assurance Exercise\n",
+ "\n",
+ "Can you think of other tests to make for the std() function? I promise there\n",
+ "are at least two.\n",
+ "\n",
+ "1. How about std(string) or std(array)?\n",
+ "2. How about std(None)?\n",
+ "\n",
+ "Implement one new test in test_stats.py, run nosetests, and if it fails, implement\n",
+ "a more robust function for that case.\n",
+ "\n",
+ "And thus - finally - we have a robust function together with working\n",
+ "tests!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Further Statistics Tests\n",
+ "\n",
+ "The `stats.py` and `test_stats.py` files contain stubs for other simple statistics \n",
+ "functions: variance, median, and mode. Try your new test-driven development chops\n",
+ "by implementing one or more of these functions along with their corresponding tests."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Advanced Exercise\n",
+ "\n",
+ "**The Problem:** In 2D or 3D, we have two points (p1 and p2) which\n",
+ "define a line segment. Additionally there exists experimental data which\n",
+ "can be anywhere in the domain. Find the data point which is closest to\n",
+ "the line segment.\n",
+ "\n",
+ "In the `close_line.py` file there are four different implementations\n",
+ "which all solve this problem. [You can read more about them\n",
+ "here.](http://inscight.org/2012/03/31/evolution_of_a_solution/) However,\n",
+ "there are no tests! Please write from scratch a `test_close_line.py`\n",
+ "file which tests the closest\\_data\\_to\\_line() functions.\n",
+ "\n",
+ "*Hint:* you can use one implementation function to test another. Below\n",
+ "is some sample data to help you get started.\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "import numpy as np\n",
+ "\n",
+ "p1 = np.array([0.0, 0.0])\n",
+ "p2 = np.array([1.0, 1.0])\n",
+ "data = np.array([[0.3, 0.6], [0.25, 0.5], [1.0, 0.75]])"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": []
+ }
+ ],
+ "metadata": {}
+ }
+ ]
+}
\ No newline at end of file
projects / swc-testing-nose.git / commitdiff