|
-
-
|
-
Introduction
-
Many books on programming and software system design contain
one or more chapters on software testing. Unfortunately, a great
deal of material presented in this way is of very little practical use
to the reader either because of the brevity of the treatment, or
because the author had little more than a superficial
understanding of the software testing process.
Some excellent books on software testing do exist, and two in
particular by Boris Beizer, published by Van Nostrand Reinhold,
are Software Testing Techniques and
Software System Testing and Quality Assurance.
The first of these addresses the more technical aspects of software
testing, whereas the second provides a broader view from a
system level of the work to be done. Both books are highly
readable and provide an excellent coverage of their topics.
However, the intent of this paper is not to provide a summary of
previously published material, but simply to present a broad
outline of factors which must be considered when planning and
managing a testing environment.
|
|
|
-
Test Planning
-
In many software development activities, testing is not seriously
considered until programming has been almost completed. This
approach to testing is clearly inadequate, but what in fact should
be done and when should the planning begin?
One of the greatest threats to successful completion of a Year
2000 project is that there will be so much to do, with no
opportunity to move the installation deadline.
Testing is expected to represent more than 50% of the work to be
done and early planning of test activities will be critical to
success. Test planning should begin as soon as the project has
commenced, so that by the time testing is ready to begin all
necessary planning has been completed, and actual testing can be
started immediately.
Test planning and design in parallel with impact analysis will
ensure a comprehensive set of tests are devised for each function
that the system must perform. If test planning is left to later in the
conversion cycle, much of the detailed knowledge gained during
the analysis stages will have been lost or forgotten, and the
resultant tests will either be incomplete, or will take much longer
to prepare because of the need to go back over (analysis) work
which has already been done.
A second major benefit resulting from parallel development of
tests is that the act of test planning and development will serve as
an independent check on the impact analysis at a time when errors
or omissions can be easily overcome.
The net result of parallel development of tests will be a conversion
plan which more completely covers all the functional
requirements, resulting in a much lower risk.
This type of test planning also helps to overcome the major
obstacle experienced by many test designers, that of what to test
and what data to use. These problems have natural and automatic
answers if test design is done at the right time and in the right
way.
|
|
|
-
Test Design
-
Test design can be approached in very much the same way as any
other software design problem, i.e. by breaking it down into
progressively smaller components until such time as it can be
easily and completely defined.
We believe that each individual test case should test a single
function only. In a large system, this will result in test cases
numbering in the thousands, with a consequent need to link these
together for subsequent re-run as a single regression test. Keeping
individual test cases small results in easier test case design.
When designing your tests, try to structure them in a way which
parallels the way the software works and is presented to the user.
This will help you to manage the testing process.
Individual test cases should be designed to check a single function
(e.g. add a record to the data base). This will simplify test design,
and aid in more accurately reporting errors.
Comprehensive validation tests must be designed, to ensure that it
is impossible to enter corrupt data into the data base. These tests
should be grouped together for execution, and all tests in the
group should be passed before tests which update the data base
are applied.
It is essential that the required outcome of each test be known
before the execution of the test is attempted. This will ensure that
all errors or suspected errors are properly investigated.
|
|
|
-
Test Database
-
Many people will be tempted to take a copy of the production data
base and use that as their base test platform. Although this
approach appears attractive superficially, and would seem to
result in time saving through not having to create a special test
data base, there are a number of problems associated with it, one
so serious as to completely exclude it from consideration. These
problems are as follows:
- Each time a new regression test is to be run, the test data base
will need to be restored to a pre-defined state. Where a copy of
the production data base has been used, this restoration process
will take several times as long as it would take to restore a
properly designed test data base. In Year 2000 testing, this time
could easily become a critical bottleneck.
- The size of the production data base would be many times that
of a specially designed test data base, resulting in an
unnecessarily high usage of additional disk space.
- Even with the same number of test transactions, running the
tests against a copy of the production data base will invariably
result in significantly longer test execution times, particularly
where data base searches are required. Again, time could be
critical.
- Verification of a test is always a two part operation. Part one is
verifying that the test has performed the functions which it was
designed to do. Part two, which is often forgotten, is verifying
that only the desired functions were performed, and that no
other changes have been made to the data base. It is this
problem that really precludes using a copy of the production
data base as a test base. If something unplanned has happened
to other parts of the data base, how do you find it when there
are tens of thousands or perhaps millions of records in that data
base?
The test base must contain sufficient realistic data to adequately
test all required functions. However, this data must be carefully
designed so that the test base contains the absolute minimum
number of records to meet this requirement. This is important
both from the point of view of the time required to reset the test
base, and to allow the test data to be exhaustively examined to
ensure that no unplanned updates have taken place.
You must have a test "base" which can be easily reset back to a
known starting point prior to the execution of any series of tests.
Each tester (or testing group, if your test team is structured in
such a way that several people work on one series of tests) must
have access to their own test base which cannot be updated by
others.
Designing and building a proper test data base will represent a considerable amount of
work. However, this time will be amply repaid as effective tests are run throughout
the conversion cycle. Your tests can subsequently be re-used throughout the
remaining life of the application software.
|
|
|
 Testing Papers
Test Automation
|
|
