"cell_type": "markdown",
"metadata": {},
"source": [
- "* FIXME"
+ "* Explain why it is not practical to prove a program correct by testing it.\n",
+ "* Distinguish between pre-conditions, post-conditions, and invariants.\n",
+ "* Correctly raise and handle exceptions.\n",
+ "* Explain why exceptions are a better way to handle errors than special return codes.\n",
+ "* Correctly write unit tests using an xUnit-style unit testing framework.\n",
+ "* Name and explain the three types of results a test can produce.\n",
+ "* Explain what test-driven development is, and use it to develop functions with well-specified behavior."
]
},
{
"source": [
"The short answer is that it's practically impossible to prove that a program will always do what it's supposed to.\n",
"To see why,\n",
- "consider a function that checks whether a character strings contains only 'A', 'C', 'G', and 'T'.\n",
+ "consider a function that checks whether a character strings contains only the letters 'A', 'C', 'G', and 'T'.\n",
"These four tests clearly aren't sufficient:"
]
},
"cell_type": "markdown",
"metadata": {},
"source": [
- "because this implementation of `is_all_bases` passes them:"
+ "because this version of `is_all_bases` passes them:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "because this implementation passes:"
+ "because this version still passes:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "And ensuring that we have the right answer is only one reason to to software.\n",
+ "Ensuring that we have the right answer is only one reason to to software.\n",
"The other is that it speeds up development\n",
"by reducing the amount of re-work we have to do.\n",
"Even small programs can be quite complex,\n",
"and changing one thing can all too easily break something else.\n",
"If we test changes as we make them,\n",
- "and re-test things we've already done,\n",
- "we can catch errors while the changes are still fresh in our minds,\n",
- "which makes fixing them much easier."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "It's important to realize,\n",
- "though,\n",
- "that testing itself doesn't make software better.\n",
- "As Steve McConnell once said,\n",
- "trying to improve the quality of software by doing more testing\n",
- "is like trying to lose weight by weighing yourself more often.\n",
- "Testing just tells us what the quality *is*;\n",
- "if we want to improve it,\n",
- "so that we don't have to throw away a week's worth of analysis because of a missing semi-colon,\n",
- "we have to change our programs,\n",
- "and change the way we go about writing programs."
+ "and automatically re-test things we've already done,\n",
+ "we can catch and fix errors while the changes are still fresh in our minds."
]
},
{
"The first step is to use [defensive programming](glossary.html#defensive_programming),\n",
"i.e.,\n",
"to put assertions in our programs so that they check their own execution as they run.\n",
- "Programs like the Firefox browser are littered with assertions—in fact,\n",
+ "Programs like the Firefox browser are littered with assertions:\n",
"10-20% of the code they contain\n",
"are there to check that the other 80-90% are working correctly.\n",
"Broadly speaking,\n",
"language": "python",
"metadata": {},
"outputs": [],
- "prompt_number": 1
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The first two assertions test that the inputs are valid,\n",
- "i.e.,\n",
- "that the upper X and Y coordinates are greater than their lower counterparts.\n",
- "Notice that the test is greater than,\n",
- "not greater than or equal to:\n",
- "this tells us (and the computer) that rectangles aren't allowed to have zero width or height.\n",
- "The last two assertions check that the upper coordinates of the scaled rectangle are valid:\n",
- "neither can be zero\n",
- "(because that would mean the rectangle had zero width or height)\n",
- "and neither can be greater than 1."
- ]
+ "prompt_number": 7
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Strictly speaking, these two assertions are redundant:\n",
- "if the inputs are correct,\n",
- "and our calculation is correct,\n",
- "then the last two conditions should always hold.\n",
- "But programmers aren't perfect, \n",
- "and if there *is* a bug in our calculations,\n",
- "we want the program to complain about it as early as possible."
+ "The first two assertions check that we've been given a legal rectangle,\n",
+ "while the last two check the output we're about to return to our caller.\n",
+ "Strictly speaking these post-conditions are redundant:\n",
+ "if the inputs and calculations are correct,\n",
+ "the last two assertions should always hold.\n",
+ "But those are pretty big ifs,\n",
+ "and having the program check itself can save us a lot of hunting around later."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Even when programs are careful,\n",
- "things sometimes go wrong.\n",
- "Some of these errors have external causes,\n",
+ "Even when programmers are careful,\n",
+ "things can still go wrong.\n",
+ "Some errors have external causes,\n",
"like missing or badly-formatted files.\n",
"Others are internal,\n",
"like bugs in code.\n",
"Either way,\n",
- "it's actually pretty easy to handle errors in sensible ways."
+ "most modern programming languages handle errors in more or less the same way."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Let's start with a look at how programmers used to do error handling.\n",
+ "Let's start with a look at how people used to do error handling.\n",
"Back in the Dark Ages,\n",
"programmers wrote functions to return a [status code](glossary.html#status_code)\n",
"to indicate whether they had run correctly or not.\n",
"and only then think about what might happen if something goes wrong.\n",
"When the two are interleaved,\n",
"both are harder to understand.\n",
- "The net result is that most programmers don't bother to check the status codes their functions return.\n",
- "Which means that when errors *do* occur,\n",
- "they're even harder to track down."
+ "The net result is that most programmers don't bother to check the status codes their functions return."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Luckily, there's a better way.\n",
- "Modern languages like Python allow us to use [exceptions](glossary.html#exception) to handle errors.\n",
- "More specifically,\n",
- "using exceptions allows us to separate the \"normal\" flow of control\n",
- "from the \"exceptional\" cases that arise when something goes wrong,\n",
- "which makes both easier to understand:"
+ "[Exceptions](glossary.html#exception) allow us to separate the \"normal\" flow of control\n",
+ "from the \"exceptional\" cases that arise when something goes wrong.\n",
+ "Using them produces code like this,\n",
+ "which is much easier to understand:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "To join the two parts together,\n",
- "we use the keywords `try` and `except`.\n",
+ "We join the normal case and the error-handling code using the keywords `try` and `except`.\n",
"These work together like `if` and `else`:\n",
"the statements under the `try` are what should happen if everything works,\n",
"while the statements under `except` are what the program should do if something goes wrong."
"output_type": "pyerr",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mIOError\u001b[0m Traceback (most recent call last)",
- "\u001b[0;32m<ipython-input-2-58cbde3dd63c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'nonexistent-file.txt'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'r'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[0;32m<ipython-input-8-58cbde3dd63c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'nonexistent-file.txt'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'r'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mIOError\u001b[0m: [Errno 2] No such file or directory: 'nonexistent-file.txt'"
]
}
],
- "prompt_number": 2
+ "prompt_number": 8
},
{
"cell_type": "code",
"output_type": "pyerr",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mIndexError\u001b[0m Traceback (most recent call last)",
- "\u001b[0;32m<ipython-input-3-7fed13afc650>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mvalues\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mprint\u001b[0m \u001b[0mvalues\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m999\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[0;32m<ipython-input-9-7fed13afc650>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mvalues\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mprint\u001b[0m \u001b[0mvalues\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m999\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mIndexError\u001b[0m: list index out of range"
]
}
],
- "prompt_number": 3
+ "prompt_number": 9
},
{
"cell_type": "markdown",
]
}
],
- "prompt_number": 6
+ "prompt_number": 10
},
{
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"source": [
- "Python tries to run the four statements inside the `try` as normal.\n",
+ "Python tries to run the four functions inside the `try` as normal.\n",
"If an error occurs in any of them,\n",
"Python immediately jumps down\n",
- "and tries to find an `except` whose type matches the type of the error that occurred.\n",
- "If it's an `IOError`,\n",
- "Python jumps into the first error handler.\n",
- "If it's an `ArithmeticError`,\n",
+ "and tries to find an `except` of the corresponding type:\n",
+ "if the exception is an `IOError`,\n",
+ "Python jumps into the first error handler,\n",
+ "while if it's an `ArithmeticError`,\n",
"Python jumps into the second handler instead.\n",
"It will only execute one of these,\n",
"just as it will only execute one branch\n",
"source": [
"Now that we understand how Python manages error,\n",
"we can return to the subject of testing.\n",
- "The biggest obstacle to doing it isn't actually whether or not it's useful,\n",
- "but whether or not it's easy to do.\n",
- "If it isn't,\n",
- "people will always find excuses to do something else.\n",
- "It's therefore important to make things as painless as possible.\n",
- "In particular, it has to be easy for people to:\n",
+ "Most people don't enjoy writing tests,\n",
+ "so if we want them to actually do it,\n",
+ "it must be easy to:\n",
"\n",
"- add or change tests,\n",
"- understand the tests that have already been written,\n",
"cell_type": "markdown",
"metadata": {},
"source": [
- "Test results must also be reliable to be useful.\n",
+ "Test results must also be reliable.\n",
"If a testing tool says that code is working when it's not,\n",
"or reports problems when there actually aren't any,\n",
"people will lose faith in it and stop using it."
"cell_type": "markdown",
"metadata": {},
"source": [
- "Let's start with the simplest kind of testing.\n",
- "A [unit test](glossary.html#unit_test) is\n",
- "a test that exercises one component, or unit, in a program.\n",
- "Every unit test has five parts.\n",
- "The first is the [test fixture](glossary.html#test_fixture),\n",
- "which is the thing the test is run on:\n",
- "the inputs to a function,\n",
- "or the data files to be processed."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The second part is the [test action](glossary.html#test_action),\n",
- "which is what we do to the fixture.\n",
- "Ideally,\n",
- "this just involves calling a function,\n",
- "but some tests may involve more.\n",
- "The third part of every unit test is its [expected result](glossary.html#expected_test_result),\n",
- "which is what we expect the piece of code we're testing to do or return.\n",
- "If we don't know the expected result,\n",
- "we can't tell whether the test passed or failed.\n",
- "As we'll see toward the end of this lesson,\n",
- "defining fixtures and expected results can be a good way to design software."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The first three parts of the unit test are used over and over again.\n",
- "The fourth part is the [actual result](glossary.html#actual_test_result),\n",
- "which is what happens when we run the test on a particular day,\n",
- "with a particular version of our software.\n",
- "The fifth and final part is a [test report](glossary.html#test_report)\n",
- "that tells us whether the test passed,\n",
- "or whether there's a failure of some kind that needs human attention.\n",
- "As with the actual result,\n",
- "this could be different each time we run the test."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "So much for terminology:\n",
- "what does this all look like in practice?\n",
+ "The simplest kind of test is a [unit test](glossary.html#unit_test)\n",
+ "that checks the behavior of one component of a program.\n",
"As an example,\n",
"suppose we're testing a function called `rectangle_area`\n",
"that returns the area of an `[x0, y0, x1, y1]` rectangle.\n",
"We'll start by testing our code directly using `assert`.\n",
"Here,\n",
"we call the function three times with different arguments,\n",
- "checking that the right value is returned each time.\n",
- "(We import `rectangle_area_1` rather than `rectangle_area`\n",
- "because we're going to use several different versions of this function in this lesson,\n",
- "and need to give each one a different name.)"
+ "checking that the right value is returned each time."
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
- "from rectangle import rectangle_area_1\n",
+ "from rectangle import rectangle_area\n",
"\n",
- "assert rectangle_area_1([0, 0, 1, 1]) == 1.0\n",
- "assert rectangle_area_1([1, 1, 4, 4]) == 9.0\n",
- "assert rectangle_area_1([0, 1, 4, 7]) == 24.0"
+ "assert rectangle_area([0, 0, 1, 1]) == 1.0\n",
+ "assert rectangle_area([1, 1, 4, 4]) == 9.0\n",
+ "assert rectangle_area([0, 1, 4, 7]) == 24.0"
],
"language": "python",
"metadata": {},
"output_type": "pyerr",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mAssertionError\u001b[0m Traceback (most recent call last)",
- "\u001b[0;32m<ipython-input-16-47f105cffcf1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m9.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m24.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[0;32m<ipython-input-11-ebf7f5f1c120>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m9.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m24.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mAssertionError\u001b[0m: "
]
}
],
- "prompt_number": 16
+ "prompt_number": 11
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "This is better than no tests at all,\n",
- "but look what happens if we change the order of the tests:"
+ "This result is used,\n",
+ "in the sense that we know something's wrong,\n",
+ "but look what happens if we run the tests in a different order:"
]
},
{
"cell_type": "code",
"collapsed": false,
"input": [
- "assert rectangle_area_1([0, 1, 4, 7]) == 24.0\n",
- "assert rectangle_area_1([1, 1, 4, 4]) == 9.0\n",
- "assert rectangle_area_1([0, 0, 1, 1]) == 1.0"
+ "assert rectangle_area([0, 1, 4, 7]) == 24.0\n",
+ "assert rectangle_area([1, 1, 4, 4]) == 9.0\n",
+ "assert rectangle_area([0, 0, 1, 1]) == 1.0"
],
"language": "python",
"metadata": {},
"output_type": "pyerr",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mAssertionError\u001b[0m Traceback (most recent call last)",
- "\u001b[0;32m<ipython-input-17-03f0be9f2eb4>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m24.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m9.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+ "\u001b[0;32m<ipython-input-12-548f3f32c981>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m24.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m4\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m9.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32massert\u001b[0m \u001b[0mrectangle_area\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mAssertionError\u001b[0m: "
]
}
],
- "prompt_number": 17
+ "prompt_number": 12
},
{
"cell_type": "markdown",
"collapsed": false,
"input": [
"def test_unit_square():\n",
- " assert rectangle_area_1([0, 0, 1, 1]) == 1.0\n",
+ " assert rectangle_area([0, 0, 1, 1]) == 1.0\n",
"\n",
"def test_large_square():\n",
- " assert rectangle_area_1([1, 1, 4, 4]) == 9.0\n",
+ " assert rectangle_area([1, 1, 4, 4]) == 9.0\n",
"\n",
"def test_actual_rectangle():\n",
- " assert rectangle_area_1([0, 1, 4, 7]) == 24.0"
+ " assert rectangle_area([0, 1, 4, 7]) == 24.0"
],
"language": "python",
"metadata": {},
"outputs": [],
- "prompt_number": 24
+ "prompt_number": 13
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next,\n",
- "we'll import a library called `ears`\n",
+ "import a library called `ears`\n",
"and ask it to run our tests for us:"
]
},
"Traceback (most recent call last):\n",
" File \"ears.py\", line 43, in run\n",
" test()\n",
- " File \"<ipython-input-24-180c0c8d0e69>\", line 8, in test_actual_rectangle\n",
- " assert rectangle_area_1([0, 1, 4, 7]) == 24.0\n",
+ " File \"<ipython-input-13-643689ad0a0f>\", line 8, in test_actual_rectangle\n",
+ " assert rectangle_area([0, 1, 4, 7]) == 24.0\n",
"AssertionError\n",
"\n"
]
}
],
- "prompt_number": 25
+ "prompt_number": 14
},
{
"cell_type": "markdown",
"source": [
"`ears.run` looks in the calling program\n",
"for functions whose names start with the letters `'test_'`\n",
- "and runs each one exactly once.\n",
+ "and runs each one.\n",
"If the function complete without an assertion being triggered,\n",
- "we count the test as a success.\n",
+ "we count the test as a [success](glossary.html#test_success).\n",
"If an assertion fails,\n",
- "we count the test as a failure,\n",
- "and if any other exception occurs,\n",
- "we count it as an error\n",
- "(i.e., we assume that the test itself is broken)."
+ "we count the test as a [failure](glossary.html#test_failure),\n",
+ "but if any other exception occurs,\n",
+ "we count it as an [error](glossary.html#test_error)\n",
+ "because the odds are that the test itself is broken."
]
},
{
"and reports how many passed, failed, or were broken."
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Most unit tests aren't as simple as a single function call,\n",
+ "and many include several assertions\n",
+ "to check several aspects of the values that functions return.\n",
+ "For example,\n",
+ "suppose we have a function called `border`\n",
+ "that's supposed to draw a black border around an image grid.\n",
+ "Here are a couple of unit tests for it:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "collapsed": false,
+ "input": [
+ "from ipythonblocks import ImageGrid\n",
+ "from border import border\n",
+ "\n",
+ "black = (0, 0, 0)\n",
+ "white = (255, 255, 255)\n",
+ "\n",
+ "def test_border_2x2():\n",
+ " fixture = ImageGrid(2, 2, fill=white)\n",
+ " border(fixture, black)\n",
+ " assert fixture[0, 0].rgb == black\n",
+ " assert fixture[0, 1].rgb == black\n",
+ " assert fixture[1, 0].rgb == black\n",
+ " assert fixture[1, 1].rgb == black\n",
+ "\n",
+ "def count_colors(grid):\n",
+ " num_black = num_white = num_other = 0\n",
+ " for x in range(grid.width):\n",
+ " for y in range(grid.height):\n",
+ " if grid[x, y].rgb == black:\n",
+ " num_black += 1\n",
+ " elif grid[x, y].rgb == white:\n",
+ " num_white += 1\n",
+ " else:\n",
+ " num_other = 0\n",
+ " return num_black, num_white, num_other\n",
+ " \n",
+ "def test_border_3x3():\n",
+ " fixture = ImageGrid(3, 3, fill=white)\n",
+ " border(fixture, black)\n",
+ " num_black, num_white, num_other = count_colors(fixture)\n",
+ " assert num_black == 8\n",
+ " assert num_white == 1\n",
+ " assert num_other == 0\n",
+ " assert fixture[1, 1].rgb == white # only white cell is in the center\n",
+ " \n",
+ "ears.run('test_border_')"
+ ],
+ "language": "python",
+ "metadata": {},
+ "outputs": [
+ {
+ "output_type": "stream",
+ "stream": "stdout",
+ "text": [
+ "...\n",
+ "3 pass, 0 fail, 0 error\n"
+ ]
+ }
+ ],
+ "prompt_number": 23
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The first test checks things directly;\n",
+ "the second uses a helper function to count cells of different colors,\n",
+ "then checks that those counts are correct\n",
+ "and that the only white cell is in the middle of the 3×3 grid.\n",
+ "If we go on to test grids of a few other sizes,\n",
+ "we can use this helper function to check them as well."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "This example also demonstrates that\n",
+ "writing tests can be as difficult as writing the program in the first place.\n",
+ "In fact,\n",
+ "if we don't build our program out of small functions that are more-or-less independent,\n",
+ "writing tests can actually be *more* complicated than writing the code itself.\n",
+ "Luckily,\n",
+ "there's a technique to help us build things right."
+ ]
+ },
{
"cell_type": "heading",
"level": 2,
"cell_type": "markdown",
"metadata": {},
"source": [
- "Of course,\n",
- "these libraries can't think of test cases for us.\n",
+ "Libraries like `ear` can't think of test cases for us.\n",
"We still have to decide what to test and how many tests to run.\n",
"Our best guide here is economics:\n",
"we want the tests that are most likely to give us useful information\n",
"Any actual implementation of `rectangle_area` will do *something* with one of these;\n",
"writing unit tests for boundary cases is a good way to specify exactly what that something is."
]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Unit tests are actually such a good way to define how functions ought to behave that\n",
+ "many programmers use a practice called [test-driven development](glossary.html#test_driven_development) (TDD).\n",
+ "Instead of writing code,\n",
+ "then figuring out how to test it,\n",
+ "these programmers:\n",
+ "\n",
+ "1. write some unit tests for a function that doesn't exist yet,\n",
+ "2. write that function,\n",
+ "3. modify it until it passes all of the tests, then\n",
+ "4. clean up the function, i.e., make it more readable or more efficient without breaking any of the tests."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The mantra often used during TDD is \"red, green, refactor\":\n",
+ "get a red light (i.e., some failing tests),\n",
+ "make it turn green (i.e., get something working),\n",
+ "and then clean it up by refactoring.\n",
+ "This cycle should take anywhere from a couple of minutes to an hour or so.\n",
+ "If it takes longer than that,\n",
+ "the change being made is probably too large,\n",
+ "and should be broken down into smaller (and more comprehensible) steps."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "TDD's proponents argue that it helps people produce better code for two reasons.\n",
+ "First,\n",
+ "it encourages them to write code in small, self-contained chunks,\n",
+ "and to actually write tests for those chunks.\n",
+ "Second,\n",
+ "it frees them from [confirmation bias](glossary.html#confirmation_bias):\n",
+ "since they haven't written their function yet,\n",
+ "their subconscious cannot steer their testing toward proving it correct\n",
+ "rather than finding errors."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Empirical studies of TDD have had mixed results:\n",
+ "some have found it beneficial,\n",
+ "while others have found no effect.\n",
+ "But even if you don't use it day to day,\n",
+ "trying it a few times helps you learn how to design functions and programs that are easier to test."
+ ]
+ },
+ {
+ "cell_type": "heading",
+ "level": 2,
+ "metadata": {},
+ "source": [
+ "Key Points"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "- Use `assert` to embed pre-conditions, post-conditions, and invariants in programs.\n",
+ "- Use `raise` to signal an error, and `try`/`except` to handle errors.\n",
+ "- Use a unit-testing framework to check and re-check code's correctness.\n",
+ "- Put each unit test in its own small function.\n",
+ "- Use test-driven development to define how functions should behave."
+ ]
}
],
"metadata": {}
projects / swc-testing-nose.git / commitdiff