|
|
|
|
On this page we list the FAQs we receive about testing and TALC2000. If
you have a question that is not answered here then please
email us your question
and we will answer you promptly.
Testing
-
What is automated testing ?
-
Can automated testing really help to save time and improve the reliability of the tested software?
-
Is test automation expensive ?
-
What benefits will I get from test automation?
-
We currently don't have any formal test plans or test environment. What are we going to need to set up to be able to use automated testing effectively?
TALC2000
-
What do I need to use TALC2000?
-
What platforms can TALC2000 test applications on?
-
What about Year 2000 testing ?
-
What about testing applications written in different languages, e.g. French, Italian, German, Spanish etc.?
-
What programming skills are required by people who will be involved in testing?
-
How much time will I spend recording tests?
-
How hard is TALC2000 to set up and use?
-
Can we use TALC2000 to test existing software?
-
What about maintenance of our test cases if the application software changes?
-
We don't have formal test plans. Can you help us to develop them?
|
 
|
-
Q... What is automated testing ?
-
Automated testing is the process of using a set of computer programs to provide test input to other programs, and to automatically verify the results of the test.
Automated testing can help you to create and input test data for the first major test of a system, or in re-testing programs after error correction or enhancement changes.
The complete re-testing of a system following program changes
(commonly referred to as regression testing) is essential to ensure that the changes have not introduced new errors.
Unfortunately, manual regression testing is an extremely labour intensive, costly and boring (and therefore error prone) exercise. For these reasons regression tests are often not performed fully, or not even attempted. This invariably results in errors being discovered when the changed software is moved into production.
The right testing tools can completely automate the execution and verification of all regression tests. The tests can be set up to run unattended at a predefined time or following a particular event (e.g. end of day backup), with the results presented to the tester next day, identifying all errors.
|
 
|
-
Q... Can automated testing really help to save time and improve the reliability of the tested software?
-
Yes, it can. However, it is important to understand what automated testing can and cannot do.
Test automation can help you to test much more quickly and economically than manual testing.
Unless your current testers are some kind of super humans, automated testing will also produce more thoroughly tested and reliable software. This is because errors which can easily be overlooked by a human tester are always identified when test validation is automated.
Because automated testing is fast and economical (tests can be run unattended, over night if required), you can afford to thoroughly re test software after bug fixes or maintenance changes. The result is quality software that will quickly become associated with your professional approach to the task.
Test automation will not magically fix all testing problems. If your current test database is a mess, your test plans are out of date or non existent and your tests depend more on the whim of the tester than on any defined test procedures, then automated testing is not going to help you much.
Setting up a proper test environment is not hard, and once the setup work has been done, managing your testing will become far easier. If you are not sure what needs to be done or how to go about it, please contact us, we can help you.
If your current test procedures do need attention, don't use that as an excuse for doing nothing. The benefits you will gain from properly managed automated testing will repay many times over the time you will spend in setting up proper test plans and procedures.
|
 
|
-
Q... Is test automation expensive ?
-
A survey of 200 Japanese software development groups (Nomura) rated automated test tools as having the highest productivity return: ahead of standard process, ahead of CASE, ahead of programming and design tools, and ahead of reviews. So test automation tools generally do provide a very good return on investment.
The expense of test automation depends on the tool and the task. Please take a look at our pricing page to gain an understanding of the total cost of ownership associated with test automation tools.
In the case of TALC2000 the testing software is priced on a per PC basis, and the price depends on the number of copies of the software that you require. Enterprise licences are also available, and these allow you to use an unlimited number of copies throughout your organisation.
Training costs vary depending on your requirements, and can be provided by your tool provider.
The important thing to recognize is that the "up front" costs will be small when compared to the savings that will be gained in the longer term. Our experience has been that if a person is engaged full time in testing, then TALC2000 will recover its full purchase cost in typically about 8 to 10 weeks.
The savings you make will not only be in testing time. Because testing will be more thorough and reliable you will experience far fewer production errors. This often proves to be the single largest cost saving in many organizations.
|
 
|
-
Q... What benefits will I get from test automation?
-
If you currently run a complete set of regression tests following changes to your application software, the benefits from automating these procedures will be largely financial.
An industry "rule of thumb" that is often used is that the life cycle maintenance cost of a software application is commonly in the vicinity of 1.5 times the initial development cost. An extremely large portion of this amount (usually well over 50%) is associated with regression testing.
If we assume an initial development cost of $1,000,000 (not a very large project these days) , then the life cycle maintenance cost will be in the vicinity of $1.5m, with testing accounting for more than half this amount.
Proper test automation could be expected to reduce this cost by up to 80%, resulting in massive savings over the life of the software.
If you do not have the resources to perform a complete regression test following changes to your application, you will still gain financial benefits, but you will also significantly improve the quality of your delivered software, simply because you can now thoroughly test all changes economically.
Financial benefits in this case are harder to quantify, because they result not from a measurable labour saving, but from reduced or eliminated costs in correcting errors found when the software is in production. The potential for savings here can be even greater than those mentioned earlier, and depending on the application can include software redistribution costs, direct trading losses, litigation costs and many other items.
In summary, the benefits from test automation will result in one or more of the following:
- Reduced testing resource (people) requirements, translating into direct cost savings
- Improved software quality
- Greater user satisfaction
- Enhanced professional image
|
 
|
-
Q... We currently don't have any formal test plans or test environment. What are we going to need to set up to be able to use automated testing effectively?
-
Software testing can be considered as a three stage process,
incorporating the following steps:
- Preparation of a known testing "base".
- Application of a set of known transactions against that base.
- Inspection of the results to ensure that the software has performed as expected.
Each time the application software is changed, either to correct a "bug" or to modify or enhance the functions provided, in addition to running new tests that check the results of the software modification all original tests should also be rerun. This repetition of tests is intended to show that the software's behaviour is unchanged except for those changes deliberately
introduced as a result of the modifications.
The repetition of tests after software change is known as regression testing. It is in this area that test automation can significantly speed up the testing process in addition to saving enormous amounts of tester's time.
Many excellent texts have been written on the subject of test design, and a detailed coverage of this topic is well beyond the scope of this answer.
However, if your testing is to be effective and efficient, you must be able to do the following things:
- You must have a test "base" which can be easily reset back to a known starting point prior to the execution of any series of tests. For serious testing of large applications, you will probably need several different test "bases" so that you may design tests to most efficiently validate all parts of your application.
- Each tester (or testing group, if your test team is structured in such a way that several people work on one series of tests) must have access to their own test base which cannot be updated by others.
- The test base must contain sufficient realistic data to adequately test all required functions. However, this data must be carefully designed so that the test base contains the absolute minimum number of records to meet this requirement. This is important both from the point of view of time needed to "refresh" the test base, and to allow the test data to be exhaustively examined to ensure that no unplanned updates have taken place.
- Individual test cases should be designed to check a single function (e.g. add a record to the data base). This will simplify test design, and perhaps more importantly allow easy modification of the tests when this becomes necessary as the result of changes to the application software.
- Comprehensive validation tests must be designed, to ensure that it is impossible to enter corrupt data into the data base. These tests should be grouped together for execution, and all tests in the group should be passed before tests which update the data base are applied.
- It is essential that the required (correct) outcome of each test be known before the execution of the test is attempted. This will ensure that all errors or suspected errors are properly investigated.
Test Automation Steps:
The simplest approach to test automation is to perform the test manually, while using testing software that captures everything you do with the mouse and keyboard, and is also able to capture responses from the application software under test. Application software responses are normally captured by taking "pictures" of the information returned to the screen by the application software as a result of input or control functions which you have performed.
You should plan to record a large number of small individual tests, and then join these tests together as a single playback stream using a simple "point and click" scripting procedure. Once this series of tests has been replayed automatically, the testing software can validate the results of the test run and produce comprehensive reports which detail the outcome of
the tests.
TALC2000 can perform these functions (and many more) for you, and save you countless hours throughout the development and maintenance lifetime of the system.
|
 
|
-
Q... What do I need to use TALC2000?
-
An IBM or 100% compatible Pentium 100 Mhz or higher with a minimum of 16MB memory and at least 20MB of available disk space, running Windows 3.1 or Windows for Workgroups, or Windows 95.
In addition, you will need some form of Windows based terminal emulator, to allow your PC to operate as a terminal to your host system.
The test tools (programs) are installed on this machine, and run in conjunction with your application/emulation software. As you perform tests manually, your actions and the application system responses are recorded. This recording process is completely invisible to the software being tested.
When you want to re run the tests, the testing software takes over the PC and provides the input that would normally be provided by a human operator.
All test input must be provided from the keyboard or pointing device, or any combination of input from these devices.
|
 
|
-
Q... What platforms can TALC2000 test applications on?
-
TALC2000 is able to test character based applications on any mainframe or midrange platform running proprietary or Unix operating systems provided the applications are accessible from a PC terminal emulator running under Windows.
TALC2000 is already used to test on Tandem, IBM mainfranes, AS400s, System 36s, a variety of Unix systems, HP3000, Digital VAX, Wang and many others.
The key is access to the applications to be tested through a Windows based terminal emulator. I you use the host applications from your PC under Windows using a terminal emulator then we can test it. TALC2000 doesn't care what host platform is running the application.
|
 
|
-
Q... What about Year 2000 testing ?
-
Year 2000 testing differs from other regression testing tasks in one significant way. Except for one special type of field, the date fields, all system input and output will be unchanged. No other bugs will (or should) be fixed, and no new functions will be added to the software. Because the changed applications should differ from the existing production software in only this area, special facilities can be provided to help create and use the tests that will be needed.
By allowing all date fields to be treated as variables in both test input and output, TALC2000 allows you to create the tests you will need using existing production software, before the Year 2000 changes have been made. These tests can then be used, without the need to perform any test maintenance, to test the applications as soon as the date changes have been made.
This means that you can begin to build (and test) your test suites now, using software that you know works correctly (i.e. production software).
When you begin to test the date changes, because your test suites are fully automated, and because the tests themselves have been tested, the time required to test your changes will be significantly reduced. You will in effect extend the "window" during which you can perform program changes, and still have them properly tested before they need to be placed into production.
|
 
|
-
Q... What about testing applications written in different languages, e.g. French, Italian, German, Spanish etc.?
-
In various countries people use their own native language keyboards with their Windows PCs and PC terminal emulators to access applications written in their own language.
While the TALC2000 tool and documentation is only available in English additional functionality has been added to allow many different native language keyboards to be used with TALC2000 for testing.
Most of TALC2000 functionality is accessed through graphical icons which are easy to use in any language once learnt. This means that with a little extra effort testers can be easily trained to use TALC2000 for testing applications with their own native language. For more information on which native language keyboards are currently supported please refer to our
multi-language testing support page.
|
 
|
-
Q... What programming skills are required by people who will be involved in testing?
-
The answer is "it depends". If you are talking about unit testing or integration testing, then programming skills and knowledge of the program structure are needed. If you are talking about system testing or acceptance testing, then programming knowledge is not required.
We believe that the most effective acceptance tests are designed by people with application knowledge and skills, rather than programming knowledge.
TALC2000 has been designed to be used by people with no programming knowledge whatsoever.
To be effective, a tester needs to have a good understanding of the application concerned, and the intellectual capability to devise and execute test cases. In addition, a good tester must also possess certain personality traits. The ideal tester is a mature individual who can quickly gain the respect of the programming team, is suspicious of everything, and wants and expects to find problems. Having done so, he/she can clearly explain where the program failed, rather than the programmer, and can help in defining what needs to be changed to correct the problem.
|
 
|
-
Q... How much time will I spend recording tests?
-
Test recording is an extremely simple process, which is performed by simply executing the test manually and clicking on a single button on a toolbar when it is desired to record information that has been returned by the application software being tested.
TALC2000 provides a "MENU" system (a TALC2000 "MENU" is a repository for test data), to help you to organise and structure your test environment. You can create as many test "MENUS" as you want, and each menu can contain up to 90 separate test cases.
When you have finished recording your test, that test is simply added to a menu, together with up to 40 characters of test description. You may also optionally add additional information to document the test purpose, pre-requisites, environment and any other items of importance.
TALC2000 allows you to create as many separate tests as you want, and because linking these tests together is exceptionally easy, you are encouraged to record a large number of very small and therefore easily created and maintained test cases.
The total time to record and save a test using TALC2000 is typically of the order of 15% longer than it would take to simply execute the test manually once.
Remember once the test has been recorded, it can be executed and validated automatically any number of times with no human intervention.
|
 
|
-
Q... How hard is TALC2000 to set up and use?
-
Learning to use the testing tools is easy. TALC2000 comes with a self running demonstration and an interactive tutorial. We also provide formal training courses which are tailored to the background and skill levels of the students, and involve several "hands on" exercises.
Complete training usually requires between 1 and 2 days.
If you already have a good test plan, it can usually be used as the basis for your automated tests with little or no change. If you do not currently have a test plan, take the time to create one. Starting from the beginning will allow you to create a test plan to take advantage of all the functions the tools provide.
All you then need to do is to perform each test once manually, and have the tools record everything that you do, and all the application program responses. As each test step is completed manually, the recorded information is saved as a single test case. When this task has been done, the test case can be re-run completely automatically at any time in the future, and the results of the test will be automatically verified.
Any number of test cases can be linked together to form a complete regression test, which can be run unattended. If your applications are running on a host, the test tools automatically adjust for differing host response times as the tests are run.
|
 
|
-
Q... Can we use TALC2000 to test existing software?
-
Yes. The testing tools view the programs being tested in the same way as a user views the software. The programming language used and the structure of your software are not important in the use of the tools.
Often, one of the problems with testing existing software is that the original developers of that software are no longer with the organisation, and current staff are not fully familiar with how the software works. This makes software changes both difficult and extremely error prone.
If when re-testing becomes necessary, you record and save the tests, then you are in effect also recording the knowledge of the system that was needed to run those tests. Future re testing can then be done with ease and confidence, even with only extremely limited knowledge of the system.
|
 
|
-
Q... What about maintenance of our test cases if the application software changes?
-
If you keep individual test cases small, maintenance is easy.
The testing software provides several different ways to maintain tests, based on the type of changes that have occurred in your application. If necessary, because the testing software maintains all test components separately internally, replacement of a complete test case, or just components of the test, can be done easily with no need to "re program" test scripts etc.
Legacy system test cases, (which essentially are character based even though your terminal emulator is running under Windows), can easily be edited. TALC2000 provides a special "pop up" editor to help you to edit input data, and a full screen editor to edit output.
|
 
|
-
Q... We don't have formal test plans. Can you help us to develop them?
-
Yes. We can help by providing checklists and step by step "How to" information. We can also provide formal training for you, or informal discussions and "hands on" assistance.
Sometimes people who do not use test plans are a little uncertain about what to do to create them. This is really a fairly straightforward task, and we would be pleased to assist you in creating your first test plans. We can just explain what to do, or we can actually help you to do the work. Often it is easier and quicker to have someone experienced in test planning and test case creation available to assist you the first time you attempt this work, and we would be pleased to discuss how we can help you in this area.
|
|
HOME
|
|