Continuous Integration Still Not Established

Interview with Armin Vogt, S&N AG

Armin Vogt

What is continuous integration (CI)?


CI is a tool-supported process which checks the source code to automatically compile and install the complete solution onto a system. Tests check whether the system is functioning properly. Errors will immediately reveal problems not detected in the development phase.

What sets projects apart that have adopted a CI process?


These projects benefit from an additional virtual team member—the build server. This builds and automatically integrates the entire source code and acts as a neutral party. If the integration test fails, this can then be easily traced back to the change in the source code. Thus, the developer will receive direct feedback on the components to be revised.

Does the test need to cover a significant amount of the solution?


Not necessarily, because there are direct benefits even without tests because releases can be built and installed with the same quality and without delays caused by holiday leave. Also, tools can be used for static code analysis. They remove careless errors and show incipient structural weaknesses in the code early on.

How much test coverage is required? Can you specify a percentage?


In many projects, no more than 10 per cent. The key question is deciding which 10 per cent of the code should be covered. This is where the ABC analysis helps because it classifies all pieces of code into three classes according to their criticality. The discussion of whether enough tests have been created is never conclusively decided in a project. More interesting is the question of how much time is scheduled for creating tests. My proposal: 50% of planned expenditures should be allocated for the implementation and maintenance of tests. If you are planning and iterating in the scrum method, the estimate should be doubled for each feature. This might be scary, but it's realistic.

CI has been around for over 15 years. Has it become old hat?


Yes. The tools are there and they are very good. But the effort and expense required to perform CI still has to be justified each time. Our customers love the capabilities of the tools, but don't always use them to purpose and over a longer period. In our consultation work, we always have to help our customers do the necessary rethinking and draw attention to hidden barriers. CI and test automation must be built and maintained by a dedicated project team and you have to continually work to keep up enthusiasm and belief in the team and project management. The architecture must be testable, the code design has to support testing. If you don't think about testing while programming, you will encounter insurmountable problems when writing the test.

Is there no alternative to CI?


Yes. Because even if I have an experienced team that has worked long term on a system and its continuous development, I have to support them as best as possible and protect them from the consequences of any potential errors. And it's becoming increasingly rare for teams to work together continuously over the long haul. 

Am I luring teams and management into a false sense of security through automated quality assurance?


I don't think teams become reckless just because tests will take care of finding blunders. It is certainly one of the advantages of still being able to correct architectural decisions late in the development process. This is what makes methods agile. I can achieve this only with the safety net of automated testing and continuous integration.

Thank you!