Organisational Learning in Software Requirements Engineering and Management
Description
The current research project addresses the continuing low success rate of software development projects, which has been frequently reported in empirical studies. For example, the 2004 Chaos Report by the Standish Group found that only 29% of 9,236 application software development projects in a cross-section of companies worldwide were considered to have been successfully delivered. Of the remainder, 18% were classified as failed (i.e., canceled before completion or never released), and 53% were categorized as challenged, meaning that they had significant cost or time overruns or did not meet the users' needs. On average, cost overruns were 56% of the original project budget; time overruns were 84% of the planned time frame; and the software product features represented only 64% of the actually required functionality. The report found the total project wastage in the USA alone to be $55 billion, including $38 billion in lost dollar value and $17 billion in cost overruns.
The main causes of failure of software development projects have been described in empirical studies as (1) deficient requirements engineering, (2) lack of executive support, and (3) unforeseen changes in the project. Of these reasons, research has consistently identified requirements engineering as being the most critical factor. For example, according to a survey of 3,805 software units of various sizes from 17 countries, requirements engineering was regarded as causing the greatest number of major problems, whereas program coding was considered to cause the least (European Software Institute, 1996).
In recent years, the design and coding phases have been greatly simplified and made more reliable through advances in process, techniques and technology. However, this is not the case for the requirements engineering activities, primarily due to the fact that it is the most communication intensive and collaborative discipline in software engineering. Requirements engineering is a workflow in the software development lifecycle, which encompasses all tasks concerned with the scoping and definition of the goals for, detailed functions of, and constraints on a new or altered software system. The requirements analyst faces the challenge to elicit the (tacit) knowledge held by the various business domain experts and then to translate it into textual requirements specifications and abstract formal models that are understandable to the software developers who must implement the required system.
Initial studies, including our pilot study, have indicated that many organizations have difficulty in learning from their experience in order to decrease the failure rate of software projects. They appear to have particular learning challenges in the area of requirements engineering. The gap between research and practice in requirements engineering seems to be substantial and has been investigated by leading researchers in computer science. Some claim that organizations suffer from "learning failure" in that they fail to learn from their experience in software development, and have also "learned to fail" (Lyytinen & Robey, 1999). However, what has been missing in the literature is a systematic investigation about the organizational and psychological factors that affect the organization's willingness and ability to improve software development success rates by continuously reflecting on and learning from its experience.