Waterfall or Agile?
by De Ceuster Academy
by De Ceuster Academy
All rights reserved. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the publisher, except in the case of brief quotations embodied in critical reviews and certain other noncommercial uses permitted by copyright law. For permission requests, write to the publisher, addressed “Attention: Permissions Coordinator,” at the address below. De Ceuster Academy @ APraCom, s.r.o. Drtinova 10/557 150 00 Praha 5 Czech Republic info@DeCeusterAcademy.com www.deceusteracademy.com
De Ceuster Academy @ APraCom, s.r.o.
Drtinova 10/557
150 00 Praha 5
Czech Republic
Luc De Ceuster, MSc, PMP
graduated from the Belgian Military Academy, Polytechnic Division, where he obtained a master's degree in Applied Sciences, Construction, and Mechanical Engineering.
After the completion of his studies, he joined the Corps of Engineers where he served in different Combat Engineer units and managed a team of engineers of the military construction agency. He left the armed forces after 13 years of active duty. During that period, he was promoted to Lieutenant and later to Captain.
After leaving the military, he joined the Belgian Aviation School where he taught candidate pilots. The subjects included instrumentation, inertial navigation, airplane construction, and airplane engines.
After 6 years, he returned to construction engineering in the field of environmental technology, joined IBM and AT&T where he was project executive and Account Director in the ICT field, and in 2006, he started his own training and coaching company APraCom in Prague, the Czech Republic.
At APraCom, he focused on academic and professional courses in the Czech Republic, Belgium, Slovakia, Switzerland, Ukraine, Russia, and others.
In 2018 the concept of De Ceuster Academy (deceusteracademy.com) started with the creation of a new website with its own Learning Management System (LMS) which provides access to all online courses.
In 2022, De Ceuster Academy created its YT channel (https://www.youtube.com/c/DeCeusterAcademy) and shares project-related
knowledge with the world. The next step is the creation of new books and
articles related to project management, finance, operations, quantitative
methods, operations management, and aviation.
The goal of this short publication is to introduce the concepts of Waterfall and Agile project management to people who are working in project management or business environments and want to understand the differences between these two different ways of dealing with projects. It is not intended for professionals who are experienced in Agile methodologies.
Project management has been around for many years and has undergone many transformations. The initial projects were very visible, and it was easy to detect issues and errors. Since the birth of modern Information Technology, the world changed. The products that are created cannot be seen as before and the combination of many lines of programming, subroutines, and interactions increases exponentially with the number of lines of code as well as tasks that are part of the program.
Finding errors and improvements becomes almost impossible and a line-by-line verification is rarely done. We have seen new programs or new releases that suddenly cause problems and some programs may stop from working while others may provide the wrong outputs which in some cases may even be more dangerous.
The classical “Waterfall” approach to project management is not always suitable and when we expect many changes during the project life cycle, it may not be the best approach. Waterfall project management expects a clear and complete definition of the scope before the project starts.
But what happens when this is not possible? Waterfall projects are difficult if not impossible to adjust to changes.
Projects in IT, R&D, product or service design, reorganization, mergers, design engineering to name some examples are typically suited for an Agile approach due to the frequent changes in the requirements and the scope and where delays can cause many problems.
The Agile method provides interesting concepts when we are dealing with projects that will be highly flexible and we expect a lot of changes. On the other hand, the Waterfall approach will be used when we have a good idea of what to expect and how to deal with adjustment down the road. And of course, we can always combine the two methods and apply each when suited for the best outcome.
In the following paragraphs, I explain the differences between the waterfall and agile approaches to project management and share some experiences and recommendations that will help you to evaluate what is the best approach for the project(s) you are or will be working on.
Project management has been around for many years and even centuries. The earliest examples of projects data back to the Pyramids in Egypt, the Great Wall of China and we can find many more examples. The modern ways to plan and execute projects started around the industrial revolution with the Gantt chart.
During the Second World War, methods were developed led to the development of techniques like the Arrow Diagramming (ADM), Program Evaluation and Review Technique (PERT), the Precedence Diagramming Method (PDM), and let's not forget the Critical Path Method (CPM). All these methods introduced major improvements which became the basis of modern project management and the creation of software that is still in use today.
Technology and business change constantly and project management methods and methodologies adjust to these constant changes. Company success and competitive advantage over competitors depends on the successful completion of projects. As a project manager, you should apply the PM method or methodology that is adapted to the specifics of the project characteristics.
The IT era started after WWII and introduced new needs and difficulties to manage the complexity. Developing software is very complex and the classical project management methods that were used to plan and execute IT projects did not provide the expected results.
From the late 1950s, project managers active in the IT sector and mainly writing programs were looking for better ways to develop software. Winston W. Royce described a model in an article in 1970 that was later referred to as the “Waterfall Model” and that divided the development of software into a sequence of steps: identify system or software requirements, analysis, design, coding, testing, and implementation.
The following figure shows the total overview of the model for IT project management as developed by Dr. Winston W. Royce in 1970 and is an extract from his article in IEEE WESCON, August 1970, pages 1-9.
Although the original model included iterations at each step, the system still has many flaws. One of the main concerns was that the software is only tested at the end of the process. Another important flaw is that the different elements or subroutines are developed separately and are just tested individually. Software projects following the waterfall methodology led to long delays, cost overruns, and even failures.
The part at the right shows the sequential overview of the implementation steps to develop a large computer program. It is this figure, or part of it, that inspired the term “Waterfall”.
The sequential approach originally included interactions between the successive steps. Nevertheless, it still provides a hierarchical process that starts from a scope definition that is completed before starting the work. Once the needs have been approved and the scope has been determined and approved, the process continues.
A deterministic environment relates to projects where the scope is defined at the start of the project and the project plan is completed in all details. For large projects, this is a complex planning process and results in a lot of documentation, planning information, detailed scope, schedule, and cost baselines that cover the entire project cycle. Once the project starts, the activities are executed as planned and progress is reported typically on a weekly basis.
A deterministic project environment means that we work in a stable environment and that the needs and the scope can be defined with a high probability. The plans can be developed in advance and provide stable baselines. This does not mean that no changes will happen. During project execution, issues may be identified, and changes may be needed. In some cases, the client or sponsor my request for changes but these must follow a strict change management process.
Many projects were executed before the introduction of the waterfall method with a variable success rate, and a negative element of the waterfall method was that it reduced flexibility. Projects always have a level of uncertainty and without a minimum level of flexibility, it is not possible to finish with success.
The waterfall method also leads to the segmentation of the work where people are completing their activities without coordination with other team members. All activities are completed independently from each other and are assembled later. Issues are detected late which may lead to redoing the initial work, loss of time and money, and in the worst-case project failure, huge financial and reputation losses.
Although a waterfall approach can be effective in some specific cases when:
In case the project does not comply with these conditions of stability, the waterfall method is not the best solution, and a different approach should be selected.
IT projects that were and are being developed using the waterfall method tend to be late and over budget and in the most pessimistic case, they fail, resulting in high losses and reputation. The same is true in the case of marketing, R&D, and other projects that are subject to frequent changes. Construction projects, on the other hand, are fairly stable and a Waterfall approach
During the 1970s when the first Japanese cars entered the European and US markets, people discovered their high quality that exceeded the major branches like Mercedes, BMW, and Audi. People discovered quickly that the success was related to the special way Toyota was managing its operations and it was later baptized as the “Toyota Production System”.
The concept is also known as Kaizen and one of its main components is lean management. Lean refers to cost reduction and one of the key elements is avoiding unnecessary work.
The Kaizen methodology has changed the way companies run their operations, improve quality, and increase profits. It is not the goal to explain the principles of Kaizen or the Toyota Production System or TPS but just to describe some elements that are at the basis of lean thinking in project management.
An important aspect that is in many cases identified with Kaizen, Lean Management, and the TPS is the concept of lean as in the expression “Lean and Mean”. Kaizen, Lean Management, and TPS are much more than lean thinking.
When we consider project management and the classic waterfall method, all plans are made for the entire project at its start. The projects may look like marvels when you consider all the details and the state-of-the-art plans but is this really necessary? The problem with the waterfall approach is that a lot of the work may just be unnecessary. Just imagine the possible changes and the impact they will have on the planning. Every change will result in a formal approval process, reviewing the different parts of the project that are affected and implementing new baselines.
Changes are complex in Waterfall projects; they increase risks and reduce the chances of success. In many cases, changes will be avoided or even not allowed. This has an important effect on the project because of even necessary changes or changes that should have been anticipated. A nice example is the project of the H.M.S. Victory in the UK which was nicknamed H.M.S. Windows XP. The story was that the press that visited the ship saw computers running on Windows XP. The story was later contradicted by the ministry of defense.
At the time the original article was written in 2017, the reaction of the project management team was a simple “All was completed according to the original specification”. The point here is that even in this “Waterfall” based project, this evolution had to be anticipated knowing the long duration of the project (started in 2008 and completed in 2017). Applying the waterfall concept rigorously can lead to similar situations. In this example, the necessary upgrade to a recent operating system was not foreseen.
Applying lean principles on projects means that we will avoid unnecessary work or work that may have to be redone later. The application of lean principles is the basis for the development of different methodologies like Scrum, Extreme Programming, Kanban, and others. All these methodologies are grouped under the denominator of Agile. In the following paragraphs, I will focus on Scrum.
Scrum and basically all Agile project management methods use an incremental approach where each sprint results in a usable product. It does not have to be a completed product and each sprint adds more and more functionalities to the product until the product meets customer expectations.
But before starting with sprints, we must obtain information about the desired outcome and create stories that describe the work to be completed during the different sprints including priorities and duration estimates. We always start from the initial product description.
At the beginning of each sprint, the product manager selects the stories to complete so that it results in a workable product at the end of the sprint. Sprints have a fixed duration, typically from one week up to four weeks. Once the duration of the sprint has been chosen, it will not change anymore.
The stories that are added to a sprint have a weight proportional to the effort to complete them. At the end of the sprint, we have a working product and the total number of points that have been completed during the sprint.
The elements in a sprint are the product backlog, the sprint backlog, the sprint iterations with daily standups, the product increment, and a sprint retrospective.
The product backlog contains all the stories that are necessary to provide the desired product. This list can be adjusted without any complications. In some projects, the sprints started with a limited product backlog that was just sufficient to cover the first iterations and the product owner added more stories whenever more information was necessary to add more components to the solution.
In the product backlog, the stories are prioritized. The product owner can change the priorities to ensure that the most desired functions are implemented asap. Before the start of each sprint, the product owner selects the stories to be completed during the next sprint.
The number of stories depends on the number of points we expect to complete during a sprint, and we expect that due to improved performance this will increase during successive sprints. When the sprint is completed, we have a new product increment, and we will conduct a sprint retrospective to review how we can improve performance.
It is also important to understand that we plan the stories to complete during a sprint but in some cases, there may be issues and some stories take longer to complete than the duration of the sprint. In that case, these stories are added back to the backlog and the product owner must decide what to do: add the story to the next sprint, review it, or delete it.
For each sprint, a simple work breakdown structure and plan can be developed including dependencies between the different stories. The stories will be assigned to the team members who will work on one story only and whenever a story is completed, a new one will be assigned to the free resources. All are represented on a Scrum board.
The scrum master is an important person and is the facilitator who ensures that the procedures set and agreed upon by the team members are followed. The scrum master is also a facilitator who removes obstacles and distractions that may hinder people to reach their meeting goals. It is the link between the team and the outside world.
With scrum, except for a Work Breakdown Structure (WBS) and a limited plan, no extended project plans are developed. The advantage is that there is no resistance to change because nothing has been planned. The product owner can change the priority of the stories, add, or remove them. The only item to consider is the effort of the stories and when another story is added, another story or in some cases more stories must be moved to a lower priority.
Contrary to a classic waterfall project, an agile or scrum project is completed when the customer is “happy”, or when it complies with the expectations. It is not even necessary to complete all the stories that were included when the project started. It is also possible that the project looks completely different than originally conceived or imagined. The changes can be implemented rapidly without a lot of additional costs or risks.
When the initiators of Scrum and Agile developed their concept, they evaluated the issues with the classical or waterfall methods and defined the core values of Agile:
In February 2001, representatives from the main Agile software development methods came together and created the Agile Software Development Manifesto which contains the 12 principles of Agile Project Management:
#1 Satisfy Customers Through Early & Continuous Development
#2 Welcome Changing Requirements Even Late in the Project
#3 Deliver Value Frequently
#4 Break the Silos of Your Project
#5 Build Projects Around Motivated Individuals
#6 The Most Effective Way of Communication is face-to-face
#7 Working Software is the Primary Measure of Progress
#8 Maintain a Sustainable Working Pace
#9 Continuous Excellence Enhances Agility
#10 Simplicity is Essential
#11 Self-organizing Teams Generate Most Value
#12 Regularly Reflect and Adjust Your Way of Work to Boost Effectiveness
The Agile manifesto and the Agile methodology have been developed as an improvement of Waterfall for IT projects, but it has also found its way to other industries, mostly these characterized by high flexibility and many changes.
When I listen to some adepts of Agile, I get the impression that all projects must be Agile! It seems that the classical waterfall approach has lost its power. And we can ask the question if that is really true? The world of project management is not black or white. Projects are not 100% predictive nor are they 100% adaptive. All projects contain elements that are different.
The first element is to understand the nature of the project environment you are working in. You must ask the right questions to understand this before defining, planning, and implementing the project. Using the wrong approach will lead to catastrophe and we should avoid that at all costs.
A determinist approach provides an easy-to-manage plan and clear time management where the project execution is thoroughly planned. On the other hand, this approach is characterized by a lack of flexibility, strict change management, doing unnecessary work, and delivery in long cycles. Due to its specific characteristics, we should only apply the waterfall method when the requirements are well known and stable at the beginning of the project, the technology is well understood, there are no ambiguous requirements, sufficient resources with required expertise and skills are available and the duration is relatively short although this varies with the type of project, e.g., some projects suitable for the waterfall approach can be rather long (such as infrastructure projects).
When we expect changes, work that takes place in a fast-changing environment, the customer has no clear idea what to implement, the waterfall approach is probably not the best way to go, and an adaptive approach could be more efficient. Do we expect the requirements to change?
If so, in the Waterfall approach, you would need to have all the requirements and the scope before starting with the plan and it may take a long time before you see any results. A better approach would be Agile where you can start with a minimal functioning solution that would include some basic functionalities of the end product.
Once this is finished, you can review and adjust where necessary. In addition, this minimal solution can already be implemented, and you can test the functionality in a real environment. During the next sprints, more items can be added, and you can adjust the content of the next steps without re-planning. Corrections can be made easily and quickly, and the users are involved with the development process via the product owner.
The adaptive or Agile approach is far from perfect. At the beginning of the Agile steps, it is difficult to predict the duration and cost of the project and management does not like that. On the other hand, Agile projects need collaboration between the team members, and when this does not happen, the project is doomed. Some agile projects are large and different teams are working on components that will be linked together and this must be coordinated by someone.
The last disadvantage relates to documentation. As defined before in an Agile project we prefer a working software or product over documentation.
There are also some myths circulating about Agile:
Projects are becoming more and more complex and without the right tools, it is impossible to complete them successfully. Today, the buzzword is Agile, and it seems to be the one in all solution for any project. Agile is a valuable and very powerful methodology when applied in the right conditions on the right projects. However, in many cases, it may be better to use a Waterfall approach.
Agile and Waterfall can be combined in larger projects. The parts that are well known, stable, and well defined can be completed using waterfall while the parts that require flexibility can be completed using Agile methods.
The selection of the methodology to use lies with the person who understands the best and that is the project manager. Together with the team, they will use the best solution, Waterfall, Agile or Hybrid to plan and execute the project. It is important that the project manager and the team are Agile in the application of the appropriate methodology!
Join our state-of-the-art courses on our website and get a certificate after completion. Earn PDUs for your PMI or other certification programs. We offer courses in Project Management, Earned Value, Finance, Quality Management, Inventory Management, CAPM and PMP Preparation, and many others.
Get 60% off of our courses with the coupon below:
60%OffDCA
In this short overview, I wanted to compare Waterfall with Agile approaches to the project. It was certainly not the intention to give a deep dive into the principles and methods applied in Agile.
The introduction of Agile in IT projects has shown that it is a powerful tool in the hands of the project manager and that it has the capacity to implement complex IT projects with success early and within budget. The success of Agile also spread to other disciplines and industries.
De Ceuster Academy is located in Prague, Czech Republic, and is a part of APraCom. Since 2006, we have been providing project management training, coaching, and consultancy in the Czech Republic, Belgium, Slovakia, Switzerland, Ukraine, and Russia. Since 2018 we started online courses under the concept of De Ceuster Academy and integrated a Learning Management System for online courses on our website. In 2020, we created our YouTube Channel providing content worldwide.