Specification, quality and test management in agile projects

Erweiterte Scrum-ProzesseIn today's software development, the agile methods are becoming increasingly important. They are the answer to the rapidly changing or initially incomplete specifications of the software. The "Agile Manifesto" recommends a lightweight approach. One prominent and widespread agile method is Scrum. Whereby, with the help of self-organizing teams in short iterations (1 to 4 week sprints), the entire life cycle of software development processes is repeatedly run through and thus the software, over executable intermediate results (increments) is gradually developed to a delivery-ripe product.

The short development iterations bring challenges for the entire lifecycle. For agile development, customized concepts for Specification Management (AM), Test Management (TM) and an integrated quality management approach (QM) that covers the entire life cycle, are key for an effective and efficient project management for agile projects. Figure 1 shows an extended scrum process which illustrates the development lifecycle and associated management measures.

Specifications management

During an initial planning sprint, the product vision and specifications are added to the software product to be developed by means of an AM-concept and stored in a product backlog in the form of a macro-planning. Whereby a business analyst creates an actual and specification analysis with the customer (end user, Product Owner). The results of this analysis are documented in a product backlog, which roughly specifies the functional requirements and the desired temporal progress in the implementation of the requirements. The AM concept must ensure in the planning sprint, that the specifications are thematically fully documented, albeit roughly, prioritized and their development efforts are estimated.

After the planning sprint, the development sprints start , which may take 1-4 weeks depending on the project context and are run through n-number of times. The goal of a Sprint is to develop specifications from the Product Backlog within a Sprint to an executable increment, so that the customer can provide feedback after each sprint. Such a development sprint runs through the entire life cycle: In the Sprint Planning a manageable amount is selected from the prioritized and estimated expenditure requirements from the product backlog and stored in the sprint backlog. The AM concept must ensure that the selected specifications are refined together with the customer and the developers, so that the specifications in detail are understood and the technical feasibility and the necessary development activities are coordinated. The testers must also be involved in this phase, to help determine the testability and the acceptance criteria for the specifications.

Test management

Burndown-Chart für AnforderungenThe refined and with the developers coordinated requests from the sprint backlog are to be developed within a sprint and converted to an executable increment. One success factor of agile development is the Continuous Integration (CI), which ensures that development results are collected daily in a central repository, meet predefined quality expectations are compilable and have successfully completed basic developer tests. In addition to the developer activities, a TM concept must also focus on the end-user-oriented tests and offer the professional testers methodological support during testing.

The professional testers should both accompany the development sprints and the results of each individual test sprint (sprint tests), as well as carrying out at the end of the development phase a dedicated acceptance test (User Acceptance Test). For these test phases, test cases with a test sequence description and with the associated test data should be defined for individual specifications.

Ideally, the test cases in addition to the desired scenarios (positive test cases), should consider undesirable or unpredictable scenarios (negative test cases). If a test case fails, a meaningful error description is necessary so that the developers can reproduce the bug, identify the cause and correct the problem.

The TM-concept must ensure that the acceptance tests put a special focus on the interaction between software systems involved (end-to-end tests) and to the testing of non-functional requirements (NFR). For important NFRs usability requirements must be included with the safety and performance requirements to ensure that end-users can use the software product also according to their professional and personal needs.

Quality Management

While the AM and TM concepts describe constructive measures from the perspective of project management, the quality management concept defines analytical measures for project management. Thereby the quality of the activities carried out (process quality) and the quality of the software to be developed (product quality) should be measured and visualized by reporting. An often neglected aspect of quality is the test quality, which makes the measurement of product quality at all possible, because the product quality is only measurable as good as the test quality it allows.

Cumulative-Flow-Chart für Testfälle For the QM in agile development, lightweight metrics that represent a snapshot of the quality and vivid graphics that illustrate the progress of the development activities are available.

A Burnddown chart for specifications illustrates the relation between the ideal and the actual work progress. The dashed gray line (Ideal) in Figure 2 shows when the number of requests (Issues) would be ideally processed so that all tasks have been completed at the end of a sprint. The blue lines (Actual and Total) show the actual processed tasks. The difference between the ideal and actual lines shows how many requirements are still open and also a tendency, how realistic it is to expect a full settlement of the requirements at the end of the sprint in consideration of the recent progress. The closer the total line runs to the ideal line, the better is the process quality.

Another graph is the cumulative flow chart for test cases that show how the test cases develop. The newly defined test cases (Status: New) have to be completed with the time and the processing must be visible in their status change. Ultimately, it is the ratio of the successful test cases (Status: Closed) and the failed test cases (Status: Failed) that defines the product quality. The test quality is measured by the ratio between the specifications and test cases as follows: The test quality is only as good as the coverage of the specifications by test cases (both positive and negative test cases).


  • Baris Güldali, Masud Fazal-Baqie: resizing large agile projects with distributed backlogs. OBJEKTspektrum Online Theme Special Agility 2015
  • Silke Geisen, Baris Güldali: Agile Testing in Scrum - Test types and procedures. OBJEKTspektrum Online Theme Special Agility 2012

Dr. Baris Güldali

Dr. Baris GüldaliDr. Baris Güldali is Senior Researcher at the s-lab - Software Quality Lab at the University of Paderborn. His work focuses on specifications and test management. After studying for a Bachelor of Computer Engineering at the Middle East Technical University in Ankara, he worked as a software developer in the industry as well as IT employee of the Institute of Electrical and Information Engineering of the University of Paderborn. In 2005 he moved to the s-lab, where he received his doctorate in 2014 on model-based test techniques.

He is member of the Steering Committee of the Section for TAV (testing, analysis and verification of software) in the society for computer science (GI) and at the same time the Deputy Spokesman of the working group "Testing object-oriented programs / Model-based testing" (TOOP / MBT) in the Section TAV. Since 2014 he leads courses through the bib International College.

During his employment in s-lab he has accompanied multiple projects for S&N. Currently, Dr. Güldali is a quality consultant in an S&N project in the banking sector and is responsible for the preparation and implementation of concepts for specification and test management for distributed agile projects.