Recently, I came across a post claiming that agile projects were nearly 1.5 times more successful. This headline seemed too good to be true, and indeed it was. The claim was based on a survey posted on X and some LinkedIn groups. While the survey is well-documented and certainly worth a read, there was no formal definition of how to measure success, and the definition was left to be determined by the individual (and self-selected) participants. Another issue with this study was the ways in which success was measured: “On Schedule,” “On Budget,” and “To Specification”. Considering Agile aims to generate value and Waterfall is usually measured against a business case, it is strange to measure success based on these criteria alone. Thus, I found little support for the premise posted in the post and asked if anyone could share more solid research on the subject. Suddenly, the dragons were gone, replaced by crickets…
Believing there should be something out there that proves Agile is a success, I started my search:
I found “Agile versus Waterfall Project Management:
Decision Model for Selecting the Appropriate Approach to a Project” which I expected to hit the bullseye, as making a selection should be based on solid studies showing which factors would make Agile successful over the Waterfall approach. Unfortunately, the authors probably faced the same issue I am encountering: The research showing the success of Agile is hard to find. So, they did the next best thing and interviewed 15 project management experts (yes, I know) and set up a survey to corroborate the experts’ opinions. The selection criteria are categorized in “project constraints” and “people and culture,” and phrased in a way that a ‘higher’ score points towards Waterfall. It felt a bit biased to me, and there were no hints (besides the experts’ opinions) of any proof that one approach might be better than the other. In addition, the criteria cannot be scored objectively. Also, one of their reasons for not choosing Agile is disproved in the following paper.
Does Agile work?-A quantitative analysis of agile project success is a nice read. The paragraph (2.2) on project succes is the best one from the studies I found so far and the moderator variables on project succes (2.3) (alignment of project with organization goals, project complexity and experience level of the team) seem strong summaries of the available literature on these subjects. This study concludes that there is a small, but significant, correlation between the application of agile principles in projects and project success. However the results might be somewhat depressed as the survey was conducted among members of PMI or LinkedIn project management groups. This might also explain why the measured up-front planning for more agile initiatives was found to be similar to their less agile counterparts (you still need some, but not as much up-front planning; feel free to contact me for a design and discovery sprint to get your agile initiative started on the right foot). Another interesting conclusion is on project complexity not being a moderator on Agile project succes (in contrast with what the paper on the decision model above would like you to believe).
“Does Agile work?-A quantitative analysis of agile project success ” is a nice read. Paragraph 2.2 on project success is the best one from the studies I found so far, and the moderator variables on project success (2.3) (alignment of project with organizational goals, project complexity, and experience level of the team) seem like strong summaries of the available literature on these subjects. This study concludes that there is a small, but significant, correlation between the application of Agile principles in projects and project success. However, the results might be somewhat depressed as the survey was conducted among members of PMI or LinkedIn project management groups. This might also explain why the measured up-front planning for more Agile initiatives was found to be similar to their less Agile counterparts. Another interesting conclusion is that project complexity is not a moderator on Agile project success (in contrast with what the paper on the decision model above would lead you to believe).
A Survey Study of Critical Success Factors in Agile Software Projects is a bit of dated work (e.g., Extreme Programming is over twice the size of Scrum), but it’s an interesting study as it focuses on what makes Agile successful. Paragraph 4.5 is an interesting read as it links the factors evaluated to the Agile Manifesto and identifies the overlaps and gaps alike.
All in all, I am still not impressed by the proof available in the scientific literature supporting the premise of Agile delivering more value. Although I found one study showing a significant improvement one swallow does not make a summer. So, if you know of any studies out there delving into this subject, please share them in the comments.