Unit Testing

1.Introduction

Frappe provides some basic tooling to quickly write automated tests. There are some basic rules:

Test can be anywhere in your repository but must begin with test_ and should be a .py file.
Tests must run on a site that starts with test_ . This is to prevent accidental loss of data.
Test stubs are automatically generated for new DocTypes.
Frappe test runner will automatically build test records for dependant DocTypes identified by the Link type field (Foreign Key)
Tests can be executed using bench run-tests
For non-DocType tests, you can write simple unittests and prefix your file names with test_ .

Writing Tests for DocTypes

2.1. Writing DocType Tests:

Test cases are in a file named test_[doctype].py
You must create all dependencies in the test file
Create a Python module structure to create fixtures / dependencies

Example (for `test_event.py` ):

# Copyright (c) 2015, Frappe Technologies Pvt. Ltd. and Contributors
# MIT License. See license.txt

import frappe
import frappe.defaults

from frappe.tests.utils import FrappeTestCase


def create_events():
    if frappe.flags.test_events_created:
        return

    frappe.set_user("Administrator")
    doc = frappe.get_doc({
        "doctype": "Event",
        "subject":"_Test Event 1",
        "starts_on": "2014-01-01",
        "event_type": "Public"
    }).insert()

    doc = frappe.get_doc({
        "doctype": "Event",
        "subject":"_Test Event 2",
        "starts_on": "2014-01-01",
        "event_type": "Private"
    }).insert()

    doc = frappe.get_doc({
        "doctype": "Event",
        "subject":"_Test Event 3",
        "starts_on": "2014-01-01",
        "event_type": "Public"
        "event_individuals": [{
            "person": "test1@example.com"
        }]
    }).insert()

    frappe.flags.test_events_created = True


class TestEvent(FrappeTestCase):
    def setUp(self):
        create_events()

    def tearDown(self):
        frappe.set_user("Administrator")

    def test_allowed_public(self):
        frappe.set_user("test1@example.com")
        doc = frappe.get_doc("Event", frappe.db.get_value("Event",
            {"subject":"_Test Event 1"}))
        self.assertTrue(frappe.has_permission("Event", doc=doc))

    def test_not_allowed_private(self):
        frappe.set_user("test1@example.com")
        doc = frappe.get_doc("Event", frappe.db.get_value("Event",
            {"subject":"_Test Event 2"}))
        self.assertFalse(frappe.has_permission("Event", doc=doc))

    def test_allowed_private_if_in_event_user(self):
        doc = frappe.get_doc("Event", frappe.db.get_value("Event",
            {"subject":"_Test Event 3"}))

        frappe.set_user("test1@example.com")
        self.assertTrue(frappe.has_permission("Event", doc=doc))

    def test_event_list(self):
        frappe.set_user("test1@example.com")
        res = frappe.get_list("Event", filters=[["Event", "subject", "like", "_Test Event%"]], fields=["name", "subject"])
        self.assertEqual(len(res), 2)
        subjects = [r.subject for r in res]
        self.assertTrue("_Test Event 1" in subjects)
        self.assertTrue("_Test Event 3" in subjects)
        self.assertFalse("_Test Event 2" in subjects)

2. Running Tests

This function will build all the test dependencies and run your tests. You should run tests from "frappe_bench" folder. Without options all tests will be run.

bench run-tests

If you need more information about test execution - you can use verbose log level for bench.

bench --verbose run-tests

Options:

--app &lt;appname&gt;
--doctype &lt;doctype&gt;
--test &lt;specifictest&gt;
--module &lt;module&gt; (Run a particular module that has tests)
--profile (Runs a Python profiler on the test)
--junit-xml-output&lt;pathtoxml&gt; (The command provides test results in the standard XUnit XML format)

2.1. Example for app:

All applications are located in folder: "~/frappe-bench/apps". We can run tests for each application.

- frappe-bench/apps/erpnext/
- frappe-bench/apps/erpnext_demo/
- frappe-bench/apps/frappe/

bench run-tests --app erpnext
bench run-tests --app erpnext_demo
bench run-tests --app frappe

2.2. Example for doctype:

frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost"
.
----------------------------------------------------------------------
Ran 1 test in 0.008s

OK

2.3. Example for test:

Run a specific case in User:

frappe@erpnext:~/frappe-bench$ bench run-tests --doctype User --test test_get_value
.
----------------------------------------------------------------------
Ran 1 test in 0.005s

OK

2.4. Example for module:

If we want to run tests in the module:

/home/frappe/frappe-bench/apps/erpnext/erpnext/support/doctype/issue/test_issue.py

We should use module name like this (related to application folder)

erpnext.support.doctype.issue.test_issue

Example:

frappe@erpnext:~/frappe-bench$ bench run-tests --module "erpnext.stock.doctype.stock_entry.test_stock_entry"
...........................
----------------------------------------------------------------------
Ran 27 tests in 30.549s

2.5. Example for profile:

frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost" --profile
.
----------------------------------------------------------------------
Ran 1 test in 0.010s

OK
         9133 function calls (8912 primitive calls) in 0.011 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.000    0.000    0.008    0.004 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:187(insert)
        1    0.000    0.000    0.003    0.003 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:386(_validate)
       13    0.000    0.000    0.002    0.000 /home/frappe/frappe-bench/apps/frappe/frappe/database.py:77(sql)
      255    0.000    0.000    0.002    0.000 /home/frappe/frappe-bench/apps/frappe/frappe/model/base_document.py:91(get)
       12    0.000    0.000    0.002    0.000

2.6 Running Tests without creating fixtures or before_tests hook

When you are building a feature it is useful to write tests without building test dependencies (i.e build fixtures for linked objects), with --skip-test-records
You can also skip the test initialisation script with --skip-before-tests

Example

bench --site school.erpnext.local run-tests --doctype "Student Group" --skip-test-records --skip-before-tests

3.0 `FrappeTestCase`

FrappeTestCase is Frappe Framework specific TestCase class extended from unittest.TestCase . Inherting this class in your tests ensures:

frappe.local.flags and other most used local proxies are reset after test case runs.
database - a new database transaction is started before testcase begins and rolled back after tests are finished.

Usage


# app/doctype/dt/test_dt.py

from frappe.tests.utils import FrappeTestCase

class TestDt(FrappeTestCase):
    @classmethod
    def setUpClass(cls):
        super().setUpClass()  # important to call super() methods when extending TestCase. 
        ...

Writing XUnit XML Tests

How to run:

bench run-tests --junit-xml-output=/reports/junit_test.xml

Example of test report:

&lt;testsuite tests="3"&gt;
    &lt;testcase classname="foo1" name="ASuccessfulTest"&gt;
    &lt;testcase classname="foo2" name="AnotherSuccessfulTest"&gt;
    &lt;testcase classname="foo3" name="AFailingTest"&gt;
        &lt;failure type="NotEnoughFoo"&gt; details about failure &lt;/failure&gt;
    &lt;/testcase&gt;
&lt;/testcase&gt;&lt;/testcase&gt;&lt;/testsuite&gt;

It’s designed for the CI Jenkins, but will work for anything else that understands an XUnit-formatted XML representation of test results.

Jenkins configuration support:

You should install xUnit plugin - https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
After installation open Jenkins job configuration, click the box named “Publish JUnit test result report” under the "Post-build Actions" and enter path to XML report: (Example: reports/*.xml )

</pathtoxml></module></specifictest></doctype></appname>