1. Preparation of a known testing "base"

2. Application of a known set of transactions against that base

3. Inspection of the results to ensure that the software has performed as expected.

Good testing practice requires that we know the results we should get from a test before that test is run. This means documenting the expected (correct) results as part of our test creation, then when the test is run checking that the results are as we expect. This approach ensures that any possible errors will be properly investigated.

Because designing our test data and working out what our test results should be can represent quite a lot of work, we want to be able to re-use that same test data every time we need to re run the test. This in turn means that the starting content of our test database must be the same each time the test is run. Hence we need to have some simple mechanism for "rolling back" or restoring our test database to a given starting point before each test run. This is normally a very simple procedure to set up, but you may need technical assistance from your database administrator or systems manager to help you to set up the required procedures.

Please recognize that this ability to "roll back" your test database and to re-use the same test data each time you run a series of tests is absolutely fundamental to effective and efficient testing, regardless of whether you plan to run the tests manually or to automate them. You can not test properly if you need to use different test data each time you run a test.

In addition to the "roll back" facility, it is important that during your testing that the contents of your test "base" can not be changed without your knowledge. (In other words, by someone else running other tests against the same database). If you try to share a test database with other testers, unless you are exceptionally careful you will find yourself chasing up errors which are the result of database differences, rather than a failure of your test cases.

The design of your test database is critically important to the success and efficiency of your testing efforts.

If a new test suite is being built for an existing application, many people are tempted to take a copy of the production database for that application and use that as their base test platform. Although this approach appears attractive superficially, and would seem to result in time saving through not having to create a special test database, there are a number of problems associated with it, one so serious as to completely exclude it from consideration. These problems are as follows:

1. The size of the production database would be many times that of a specially designed test database, resulting in an unnecessarily high usage of additional disk space.

2. Each time a new regression test is to be run, the test database will need to be restored to a pre-defined state. Where a copy of the production database has been used, this restoration process will take several times as long as it would take to restore a properly designed test database.

3. Even with the same number of test transactions, running the tests against a copy of the production database will invariably result in significantly longer test execution times, particularly where database searches are required.

4. Verification of a test is always a two part operation. Part one is verifying that the test has performed the functions which it was designed to do. Part two, which is often forgotten, is verifying that only the desired functions were performed, and that no other changes have been made to the database. It is this problem that makes using a copy of the production database unsuitable as a test base. If something unplanned has happened to other parts of the database, how do you find it when there are tens of thousands or perhaps millions of records in that database?

Individual test case design is also important. If you keep your individual test cases small (e.g. each test case tests just a single function), you will find that test design is easier because you can focus on just one important area at a time. Probably more important though is that small test cases are easily changed (or discarded and replaced if necessary) when your test application software changes.

Each time your application software is changed, either to correct a "bug" or to introduce some new function, in addition to running new tests that may be required, all original tests should also be re-run. This repetition of tests is intended to show that the software's behavior is unchanged, except for those changes deliberately introduced as a result of the bug fix or enhancement. This repetition of tests following software modification is known as regression testing. (This is where test automation can really save time!)

Although nothing we have discussed in this section is complex, it is so important to setting up an effective and efficient testing environment that it warrants a summary:

• You must have a test "base" which can be easily re-set back to a known starting point prior to the execution of any series of tests. For serious testing of large applications, you will probably need several different test "bases" so that you may design tests to most efficiently validate all parts of your application.

• Each tester (or testing group, if your test team is structured in such a way that several people work on one series of tests) must have access to their own test base which cannot be updated by others.

• The test base must contain sufficient realistic data to adequately test all required functions. However, this data must be carefully designed so that the test base contains the absolute minimum number of records to meet this requirement. This is important both from the point of view of the time needed to "roll back" the test base and the amount of disk space consumed, and to allow the test data to be exhaustively examined to ensure that no unplanned updates have taken place.

• Individual test cases should be designed to check a single function (e.g. add a record to the database). This will simplify test design, and perhaps more importantly allow easy modification of the tests when this becomes necessary as the result of changes to the application software.

• It is essential that the required outcome of each test be known before the execution of the test is attempted. This will ensure that all errors or suspected errors are properly investigated.