| =========================== |
| Testing Django applications |
| =========================== |
| |
| Automated testing is an extremely useful weapon in the bug-killing arsenal |
| of the modern developer. When initially writing code, a test suite can be |
| used to validate that code behaves as expected. When refactoring or |
| modifying code, tests serve as a guide to ensure that behavior hasn't |
| changed unexpectedly as a result of the refactor. |
| |
| Testing a web application is a complex task, as there are many |
| components of a web application that must be validated and tested. To |
| help you test your application, Django provides a test execution |
| framework, and range of utilities that can be used to simulate and |
| inspect various facets of a web application. |
| |
| This testing framework is currently under development, and may change |
| slightly before the next official Django release. |
| |
| (That's *no* excuse not to write tests, though!) |
| |
| Writing tests |
| ============= |
| |
| Tests in Django come in two forms: doctests and unit tests. |
| |
| Writing doctests |
| ---------------- |
| |
| Doctests use Python's standard doctest_ module, which searches for tests in |
| your docstrings. Django's test runner looks for doctests in your ``models.py`` |
| file, and executes any that it finds. Django will also search for a file |
| called ``tests.py`` in the application directory (i.e., the directory that |
| holds ``models.py``). If a ``tests.py`` is found, it will also be searched |
| for doctests. |
| |
| .. admonition:: What's a **docstring**? |
| |
| A good explanation of docstrings (and some guidlines for using them |
| effectively) can be found in :PEP:`257`: |
| |
| A docstring is a string literal that occurs as the first statement in |
| a module, function, class, or method definition. Such a docstring |
| becomes the ``__doc__`` special attribute of that object. |
| |
| Since tests often make great documentation, doctest lets you put your |
| tests directly in your docstrings. |
| |
| You can put doctest strings on any object in your ``models.py``, but it's |
| common practice to put application-level doctests in the module docstring, and |
| model-level doctests in the docstring for each model. |
| |
| For example:: |
| |
| from django.db import model |
| |
| class Animal(models.Model): |
| """ |
| An animal that knows how to make noise |
| |
| # Create some animals |
| >>> lion = Animal.objects.create(name="lion", sound="roar") |
| >>> cat = Animal.objects.create(name="cat", sound="meow") |
| |
| # Make 'em speak |
| >>> lion.speak() |
| 'The lion says "roar"' |
| >>> cat.speak() |
| 'The cat says "meow"' |
| """ |
| |
| name = models.CharField(maxlength=20) |
| sound = models.CharField(maxlength=20) |
| |
| def speak(self): |
| return 'The %s says "%s"' % (self.name, self.sound) |
| |
| When you `run your tests`_, the test utility will find this docstring, notice |
| that portions of it look like an interactive Python session, and execute those |
| lines while checking that the results match. |
| |
| For more details about how doctest works, see the `standard library |
| documentation for doctest`_ |
| |
| .. _doctest: http://docs.python.org/lib/module-doctest.html |
| .. _standard library documentation for doctest: doctest_ |
| |
| Writing unittests |
| ----------------- |
| |
| Like doctests, Django's unit tests use a standard library module: unittest_. |
| As with doctests, Django's test runner looks for any unit test cases defined |
| in ``models.py``, or in a ``tests.py`` file stored in the application |
| directory. |
| |
| An equivalent unittest test case for the above example would look like:: |
| |
| import unittest |
| from myapp.models import Animal |
| |
| class AnimalTestCase(unittest.TestCase): |
| |
| def setUp(self): |
| self.lion = Animal.objects.create(name="lion", sound="roar") |
| self.cat = Animal.objects.create(name="cat", sound="meow") |
| |
| def testSpeaking(self): |
| self.assertEquals(self.lion.speak(), 'The lion says "roar"') |
| self.assertEquals(self.cat.speak(), 'The cat says "meow"') |
| |
| When you `run your tests`_, the test utility will find all the test cases |
| (that is, subclasses of ``unittest.TestCase``) in ``models.py`` and |
| ``tests.py``, automatically build a test suite out of those test cases, |
| and run that suite. |
| |
| For more details about ``unittest``, see the `standard library unittest |
| documentation`_. |
| |
| .. _unittest: http://docs.python.org/lib/module-unittest.html |
| .. _standard library unittest documentation: unittest_ |
| .. _run your tests: `Running tests`_ |
| |
| Which should I use? |
| ------------------- |
| |
| Choosing a test framework is often contentious, so Django simply supports |
| both of the standard Python test frameworks. Choosing one is up to each |
| developer's personal tastes; each is supported equally. Since each test |
| system has different benefits, the best approach is probably to use both |
| together, picking the test system to match the type of tests you need to |
| write. |
| |
| For developers new to testing, however, this choice can seem |
| confusing, so here are a few key differences to help you decide whether |
| doctests or unit tests are right for you. |
| |
| If you've been using Python for a while, ``doctest`` will probably feel more |
| "pythonic". It's designed to make writing tests as easy as possible, so |
| there's no overhead of writing classes or methods; you simply put tests in |
| docstrings. This gives the added advantage of given your modules automatic |
| documentation -- well-written doctests can kill both the documentation and the |
| testing bird with a single stone. |
| |
| For developers just getting started with testing, using doctests will probably |
| get you started faster. |
| |
| The ``unittest`` framework will probably feel very familiar to developers |
| coming from Java. Since ``unittest`` is inspired by Java's JUnit, if |
| you've used testing frameworks in other languages that similarly were |
| inspired by JUnit, ``unittest`` should also feel pretty familiar. |
| |
| Since ``unittest`` is organized around classes and methods, if you need |
| to write a bunch of tests that all share similar code, you can easily use |
| subclass to abstract common tasks; this makes test code shorter and cleaner. |
| There's also support for explicit setup and/or cleanup routines, which give |
| you a high level of control over the environment your test cases run in. |
| |
| Again, remember that you can use both systems side-by-side (even in the same |
| app). In the end, most projects will eventually end up using both; each shines |
| in different circumstances. |
| |
| Testing Tools |
| ============= |
| |
| To assist in testing various features of your application, Django provides |
| tools that can be used to establish tests and test conditions. |
| |
| * `Test Client`_ |
| * Fixtures_ |
| |
| Test Client |
| ----------- |
| |
| The Test Client is a simple dummy browser. It allows you to simulate |
| GET and POST requests on a URL, and observe the response that is received. |
| This allows you to test that the correct view is executed for a given URL, |
| and that the view constructs the correct response. |
| |
| As the response is generated, the Test Client gathers details on the |
| Template and Context objects that were used to generate the response. These |
| Templates and Contexts are then provided as part of the response, and can be |
| used as test conditions. |
| |
| .. admonition:: Test Client vs Browser Automation? |
| |
| The Test Client is not intended as a replacement for Twill_, Selenium_, |
| or other browser automation frameworks - it is intended to allow |
| testing of the contexts and templates produced by a view, |
| rather than the HTML rendered to the end-user. |
| |
| A comprehensive test suite should use a combination of both: Test Client |
| tests to establish that the correct view is being called and that |
| the view is collecting the correct context data, and Browser Automation |
| tests to check that user interface behaves as expected. |
| |
| .. _Twill: http://twill.idyll.org/ |
| .. _Selenium: http://www.openqa.org/selenium/ |
| |
| Making requests |
| ~~~~~~~~~~~~~~~ |
| |
| Creating an instance of ``Client`` (``django.test.client.Client``) requires |
| no arguments at time of construction. Once constructed, the following methods |
| can be invoked on the ``Client`` instance. |
| |
| ``get(path, data={})`` |
| Make a GET request on the provided ``path``. The key-value pairs in the |
| data dictionary will be used to create a GET data payload. For example:: |
| |
| c = Client() |
| c.get('/customers/details/', {'name':'fred', 'age':7}) |
| |
| will result in the evaluation of a GET request equivalent to:: |
| |
| http://yoursite.com/customers/details/?name=fred&age=7 |
| |
| ``post(path, data={}, content_type=MULTIPART_CONTENT)`` |
| Make a POST request on the provided ``path``. If you provide a content type |
| (e.g., ``text/xml`` for an XML payload), the contents of ``data`` will be |
| sent as-is in the POST request, using the content type in the HTTP |
| ``Content-Type`` header. |
| |
| If you do not provide a value for ``content_type``, the values in |
| ``data`` will be transmitted with a content type of ``multipart/form-data``. |
| The key-value pairs in the data dictionary will be encoded as a multipart |
| message and used to create the POST data payload. |
| |
| To submit multiple values for a given key (for example, to specify |
| the selections for a multiple selection list), provide the values as a |
| list or tuple for the required key. For example, a data dictionary of |
| ``{'choices': ('a','b','d')}`` would submit three selected rows for the |
| field named ``choices``. |
| |
| Submitting files is a special case. To POST a file, you need only |
| provide the file field name as a key, and a file handle to the file you wish to |
| upload as a value. The Test Client will populate the two POST fields (i.e., |
| ``field`` and ``field_file``) required by Django's FileField. For example:: |
| |
| c = Client() |
| f = open('wishlist.doc') |
| c.post('/customers/wishes/', {'name':'fred', 'attachment':f}) |
| f.close() |
| |
| will result in the evaluation of a POST request on ``/customers/wishes/``, |
| with a POST dictionary that contains `name`, `attachment` (containing the |
| file name), and `attachment_file` (containing the file data). Note that you |
| need to manually close the file after it has been provided to the POST. |
| |
| ``login(path, username, password)`` |
| In a production site, it is likely that some views will be protected with |
| the @login_required decorator provided by ``django.contrib.auth``. Interacting |
| with a URL that has been login protected is a slightly complex operation, |
| so the Test Client provides a simple method to automate the login process. A |
| call to ``login()`` stimulates the series of GET and POST calls required |
| to log a user into a @login_required protected view. |
| |
| If login is possible, the final return value of ``login()`` is the response |
| that is generated by issuing a GET request on the protected URL. If login |
| is not possible, ``login()`` returns False. |
| |
| Note that since the test suite will be executed using the test database, |
| which contains no users by default. As a result, logins for your production |
| site will not work. You will need to create users as part of the test suite |
| to be able to test logins to your application. |
| |
| Testing Responses |
| ~~~~~~~~~~~~~~~~~ |
| |
| The ``get()``, ``post()`` and ``login()`` methods all return a Response |
| object. This Response object has the following properties that can be used |
| for testing purposes: |
| |
| =============== ========================================================== |
| Property Description |
| =============== ========================================================== |
| ``status_code`` The HTTP status of the response. See RFC2616_ for a |
| full list of HTTP status codes. |
| |
| ``content`` The body of the response. The is the final page |
| content as rendered by the view, or any error message |
| (such as the URL for a 302 redirect). |
| |
| ``template`` The Template instance that was used to render the final |
| content. Testing ``template.name`` can be particularly |
| useful; if the template was loaded from a file, |
| ``template.name`` will be the file name that was loaded. |
| |
| If multiple templates were rendered, (e.g., if one |
| template includes another template),``template`` will |
| be a list of Template objects, in the order in which |
| they were rendered. |
| |
| ``context`` The Context that was used to render the template that |
| produced the response content. |
| |
| As with ``template``, if multiple templates were rendered |
| ``context`` will be a list of Context objects, stored in |
| the order in which they were rendered. |
| =============== ========================================================== |
| |
| .. _RFC2616: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html |
| |
| Exceptions |
| ~~~~~~~~~~ |
| |
| If you point the Test Client at a view that raises an exception, that exception |
| will be visible in the test case. You can then use a standard ``try...catch`` |
| block, or ``unittest.TestCase.assertRaises()`` to test for exceptions. |
| |
| The only exceptions that are not visible in a Test Case are ``Http404``, |
| ``PermissionDenied`` and ``SystemExit``. Django catches these exceptions |
| internally and converts them into the appropriate HTTP responses codes. |
| |
| Persistent state |
| ~~~~~~~~~~~~~~~~ |
| |
| The Test Client is stateful; if a cookie is returned as part of a response, |
| that cookie is provided as part of the next request issued by that Client |
| instance. Expiry policies for these cookies are not followed; if you want |
| a cookie to expire, either delete it manually or create a new Client |
| instance (which will effectively delete all cookies). |
| |
| There are two properties of the Test Client which are used to store persistent |
| state information. If necessary, these properties can be interrogated as |
| part of a test condition. |
| |
| =============== ========================================================== |
| Property Description |
| =============== ========================================================== |
| ``cookies`` A Python ``SimpleCookie`` object, containing the current |
| values of all the client cookies. |
| |
| ``session`` A dictionary-like object containing session information. |
| See the `session documentation`_ for full details. |
| =============== ========================================================== |
| |
| .. _`session documentation`: ../sessions/ |
| |
| Example |
| ~~~~~~~ |
| |
| The following is a simple unit test using the Test Client:: |
| |
| import unittest |
| from django.test.client import Client |
| |
| class SimpleTest(unittest.TestCase): |
| def setUp(self): |
| # Every test needs a client |
| self.client = Client() |
| def test_details(self): |
| # Issue a GET request |
| response = self.client.get('/customer/details/') |
| |
| # Check that the respose is 200 OK |
| self.failUnlessEqual(response.status_code, 200) |
| # Check that the rendered context contains 5 customers |
| self.failUnlessEqual(len(response.context['customers']), 5) |
| |
| Fixtures |
| -------- |
| |
| A test case for a database-backed website isn't much use if there isn't any |
| data in the database. To make it easy to put test data into the database, |
| Django provides a fixtures framework. |
| |
| A *Fixture* is a collection of files that contain the serialized contents of |
| the database. Each fixture has a unique name; however, the files that |
| comprise the fixture can be distributed over multiple directories, in |
| multiple applications. |
| |
| .. note:: |
| If you have synchronized a Django project, you have already experienced |
| the use of one fixture -- the ``initial_data`` fixture. Every time you |
| synchronize the database, Django installs the ``initial_data`` fixture. |
| This provides a mechanism to populate a new database with any initial |
| data (such as a default set of categories). Fixtures with other names |
| can be installed manually using ``django-admin.py loaddata``. |
| |
| |
| However, for the purposes of unit testing, each test must be able to |
| guarantee the contents of the database at the start of each and every |
| test. To do this, Django provides a TestCase baseclass that can integrate |
| with fixtures. |
| |
| Moving from a normal unittest TestCase to a Django TestCase is easy - just |
| change the base class of your test, and define a list of fixtures |
| to be used. For example, the test case from `Writing unittests`_ would |
| look like:: |
| |
| from django.test import TestCase |
| from myapp.models import Animal |
| |
| class AnimalTestCase(TestCase): |
| fixtures = ['mammals.json', 'birds'] |
| |
| def setUp(self): |
| # test definitions as before |
| |
| At the start of each test case, before ``setUp()`` is run, Django will |
| flush the database, returning the database the state it was in directly |
| after ``syncdb`` was called. Then, all the named fixtures are installed. |
| In this example, any JSON fixture called ``mammals``, and any fixture |
| named ``birds`` will be installed. See the documentation on |
| `loading fixtures`_ for more details on defining and installing fixtures. |
| |
| .. _`loading fixtures`: ../django_admin/#loaddata-fixture-fixture |
| |
| This flush/load procedure is repeated for each test in the test case, so you |
| can be certain that the outcome of a test will not be affected by |
| another test, or the order of test execution. |
| |
| Running tests |
| ============= |
| |
| Run your tests using your project's ``manage.py`` utility:: |
| |
| $ ./manage.py test |
| |
| If you only want to run tests for a particular application, add the |
| application name to the command line. For example, if your |
| ``INSTALLED_APPS`` contains ``myproject.polls`` and ``myproject.animals``, |
| but you only want to run the animals unit tests, run:: |
| |
| $ ./manage.py test animals |
| |
| When you run your tests, you'll see a bunch of text flow by as the test |
| database is created and models are initialized. This test database is |
| created from scratch every time you run your tests. |
| |
| By default, the test database gets its name by prepending ``test_`` to |
| the database name specified by the ``DATABASE_NAME`` setting; all other |
| database settings will the same as they would be for the project normally. |
| If you wish to use a name other than the default for the test database, |
| you can use the ``TEST_DATABASE_NAME`` setting to provide a name. |
| |
| Once the test database has been established, Django will run your tests. |
| If everything goes well, at the end you'll see:: |
| |
| ---------------------------------------------------------------------- |
| Ran 22 tests in 0.221s |
| |
| OK |
| |
| If there are test failures, however, you'll see full details about what tests |
| failed:: |
| |
| ====================================================================== |
| FAIL: Doctest: ellington.core.throttle.models |
| ---------------------------------------------------------------------- |
| Traceback (most recent call last): |
| File "/dev/django/test/doctest.py", line 2153, in runTest |
| raise self.failureException(self.format_failure(new.getvalue())) |
| AssertionError: Failed doctest test for myapp.models |
| File "/dev/myapp/models.py", line 0, in models |
| |
| ---------------------------------------------------------------------- |
| File "/dev/myapp/models.py", line 14, in myapp.models |
| Failed example: |
| throttle.check("actor A", "action one", limit=2, hours=1) |
| Expected: |
| True |
| Got: |
| False |
| |
| ---------------------------------------------------------------------- |
| Ran 2 tests in 0.048s |
| |
| FAILED (failures=1) |
| |
| The return code for the script will indicate the number of tests that failed. |
| |
| Regardless of whether the tests pass or fail, the test database is destroyed when |
| all the tests have been executed. |
| |
| Using a different testing framework |
| =================================== |
| |
| Doctest and Unittest are not the only Python testing frameworks. While |
| Django doesn't provide explicit support these alternative frameworks, |
| it does provide a mechanism to allow you to invoke tests constructed for |
| an alternative framework as if they were normal Django tests. |
| |
| When you run ``./manage.py test``, Django looks at the ``TEST_RUNNER`` |
| setting to determine what to do. By default, ``TEST_RUNNER`` points to |
| ``django.test.simple.run_tests``. This method defines the default Django |
| testing behavior. This behavior involves: |
| |
| #. Performing global pre-test setup |
| #. Creating the test database |
| #. Running ``syncdb`` to install models and initial data into the test database |
| #. Looking for Unit Tests and Doctests in ``models.py`` and ``tests.py`` file for each installed application |
| #. Running the Unit Tests and Doctests that are found |
| #. Destroying the test database |
| #. Performing global post-test teardown |
| |
| If you define your own test runner method and point ``TEST_RUNNER`` |
| at that method, Django will execute your test runner whenever you run |
| ``./manage.py test``. In this way, it is possible to use any test |
| framework that can be executed from Python code. |
| |
| Defining a test runner |
| ---------------------- |
| By convention, a test runner should be called ``run_tests``; however, you |
| can call it anything you want. The only requirement is that it accept two |
| arguments: |
| |
| ``run_tests(module_list, verbosity=1)`` |
| The module list is the list of Python modules that contain the models to be |
| tested. This is the same format returned by ``django.db.models.get_apps()`` |
| |
| Verbosity determines the amount of notification and debug information that |
| will be printed to the console; `0` is no output, `1` is normal output, |
| and `2` is verbose output. |
| |
| This method should return the number of tests that failed. |
| |
| Testing utilities |
| ----------------- |
| |
| To assist in the creation of your own test runner, Django provides |
| a number of utility methods in the ``django.test.utils`` module. |
| |
| ``setup_test_environment()`` |
| Performs any global pre-test setup, such as the installing the |
| instrumentation of the template rendering system. |
| |
| ``teardown_test_environment()`` |
| Performs any global post-test teardown, such as removing the instrumentation |
| of the template rendering system. |
| |
| ``create_test_db(verbosity=1, autoclobber=False)`` |
| Creates a new test database, and run ``syncdb`` against it. |
| |
| ``verbosity`` has the same behavior as in the test runner. |
| |
| ``Autoclobber`` describes the behavior that will occur if a database with |
| the same name as the test database is discovered. If ``autoclobber`` is False, |
| the user will be asked to approve destroying the existing database. ``sys.exit`` |
| is called if the user does not approve. If autoclobber is ``True``, the database |
| will be destroyed without consulting the user. |
| |
| ``create_test_db()`` has the side effect of modifying |
| ``settings.DATABASE_NAME`` to match the name of the test database. |
| |
| ``destroy_test_db(old_database_name, verbosity=1)`` |
| Destroys the database with the name ``settings.DATABASE_NAME`` matching, |
| and restores the value of ``settings.DATABASE_NAME`` to the provided name. |
| |
| ``verbosity`` has the same behavior as in the test runner. |