Code reviews in a small software company
Nikita Zhuk
Master’s thesis
University of Helsinki
Department of Computer Science
Introduction
One of the most important areas of software engineering is the quality assurance of software product which provides confidence in a product’s compliance with its requirements. Finding software errors in early phases of software process decreases the cost of software development because the earlier errors are found the cheaper it is to fix them.
Testing and reviews are the two most important techniques in software quality assurance. Testing is defined as executing software or a part of it by giving it test input and checking that the software produces the desired output. During reviews software documents or code are inspected manually, so reviews can be applied even to partially complete software artifacts. Reviews make it possible to find errors earlier than when testing is used. Reviews also enable developers to find non-functional errors, such as errors related to software maintainability or performance.
Reviews were first mentioned in the software literature during 1970s. Software professionals have successfully removed 50% – 90% of all known errors in software documents and code by using reviews. Grady and Van Slack report that a review technique called inspection was used at Hewlett-Packard and it generated 28.5% savings in requirements phase, 33.8% savings in design phase, 37.3% savings in implementation phase and 16.1% savings in testing phase. Russell estimated that each hour spent in a review avoided, in average, 33 hours of maintenance work in a large, over 15 million-lines-of-code project at Bell-Northern Research. Although benefits reported in various empirical papers do have large variation, reviews have caused positive effects in nearly every case reported.
Empirical research of reviews has been most active in middle-sized and large companies. However, reviews are also used in small software companies. Small companies don’t collect review metrics as widely as larger ones, so there isn’t much qualitative data available for small companies’ reviews.
Software quality assurance challenges also apply to small software companies with less than 10 employees. These companies may have two – three people working on a single software project. Finding errors as soon as possible in small software companies’ projects is crucial. Otherwise, more time will be used to find and fix errors. This, in turn, decreases the time available to develop new features. If this trend continues, all time resources will be used to find and fix errors, and no value will be produced to the client.
This master’s thesis will introduce various peer review techniques and present an overview of the latest software review research. The thesis focuses on code reviews. A case study will be performed in a small software company as a part of the thesis. The goal of the case study is to find out if code reviews improve the quality of a software product. Thesis presents planning, preparation, analysis and conclusions of the study.
The structure of the thesis is as follows. Section 2 introduces inspection, which is a review technique which is mentioned the most in the literature. In section 3 other review techniques are examined. Section 4 discusses the main directions of the latest review research and introduces the fundamental research results. Section 5 introduces the software project, the software company and the current software assurance process of the software company where the case study was performed. Section 6 discusses the research question and research methodology, and assesses research risks. Section 7 describes case study design, implementation and results. Section 8 discusses the results of the case study. Section 9 presents a summary of topics discussed in the thesis and evaluates the meaningfulness of the case study to the software company which was part of it.
Summary
Reviews have been studied in scientific literature since 1970s. Based on the results of the literature reviews are efficient quality assurance technique, because they allow developers to find errors as soon as possible. Empirical research has mostly been conducted in university projects and in middle-sized or large companies with established software engineering processes. Small software companies face quality assurance challenges which could be met with reviews. Application of code reviews in small software companies is currently not widely described in software literature. There’s little quantitative data available about efficiency of code reviews in small software companies.
The main goal of the case study was to determine if code reviews are efficient technique to improve the quality of a software product in a small software company. Literature overview presented history of reviews, key concepts, various review methods described in the literature and the main themes of recent review research. Inspections, first introduced in 1970s by Michael Fagan, are the most studied and the most formal review method. Inspections emphasize various roles of participants and they consider review meetings mandatory. There’re also other review methods which are less formal than inspections. Less formal review methods place less emphasis on participants’ role division and don’t necessarily require review meetings.
Recent review research has been divided into four distinct themes: reading techniques, review efficiency factors, review processes and content-specific issues. Especially the importance of review meetings and meetings’ effect on review efficiency has been questioned.
During the case study a review process was developed, which didn’t include mandatory review meetings and which allowed arranging follow-up meetings if necessary. The review process was applied in a Finnish software company with less than ten people. Research question was studied in a case study which investigated if code reviews were an efficient way to improve the quality of a software product of the company.
The study showed that code reviews were efficient and that they had a positive effect on quality. The company’s employees and the management found reviews useful. Code reviews proved to be an efficient tool for distributing information among software engineers about the software product and programming best practices. The company involved in the case study decided to expand the use of code reviews into other software projects.
Five code reviews were held during the case study. The small number of reviews limits the possible conclusions which can be drawn from the case study. The effect seen in one small software company cannot be generalized to all small software companies. A possible subject for further studies is to conduct more reviews and to repeat the study in other software companies. The review process used in the case study could also be developed further and the effect of process changes on software quality and on software engineers’ attitudes towards reviews could be studied.