TMAP SURVEY 2006
Many people are interested in organizing test management processes. After two TPI-surveys (2002 & 2004), we conducted end 2005/early 2006 a TMap survey on the experiences in managing the test process and the use of TMap. In total 204 people have participated in the survey.
Some highlights are:
- Most respondents were in some kind of mixed development process, using bits of iterative and of waterfall development. TMap is used as much in waterfall as in iterative development. Agile development is not very popular (yet);
- Almost all respondents (91%) used a methodology for testing, two-thirds of the respondents used TMap or a method based on TMap;
- We found it good news that in more than 70% of the cases a master test plan was used. We consider a master test plan a sign of high test maturity;
- On the other hand, we were somewhat disappointed that a risk based test strategy is in only half of the cases part of the test plan. We feel a lot can be gained here;
- Business Driven Test Management, TMap's new way of defining a test strategy and managing the process, is gaining ground (21%) on the old way (42%), but still has a way to go. When the new approach is integrated in the TMap book, we think this will speed things up even more;
- The planned effort is overrun by a third in the development processes the respondents participated in;
- In less than half the cases, a testability review was performed. We feel this number should go up, as it's a very powerful technique for finding defects in the test basis early in the process;
- Reporting also leaves room for improvement, with a quarter of answers saying they have no written progress reporting and less than half saying the quality of the test object was part of their reporting;
- Scripted, exploratory and ad-hoc testing are used in a proportion of 60 to 20 to 20 by the respondents;
