Waterfall vs. agile

By Kevin Miller and Tommy van Schaik

The topic of waterfall vs. agile can feel like a battle between good and evil, if you’re a firm believer in only one of the methodologies. Each methodology, however, has advantages over the other and should be applied to projects where those advantages will make meaningful differences. 

Too often, we allow our logical thinking to lead us to create a hybrid of waterfall and agile; our reasoning has us believe that combining the two methodologies will leave us with the best of both. That seductive illusion, however, leads to the worst of both worlds—and it can’t be avoided.

Don’t fall for this trap. This article will help you determine which methodology is right for your project and explain how to escape the dreaded wagile hybrid.

Waterfall is our base measurement. Waterfall has been around for more than half a century and is a sequential methodology, where projects follow a specific order of events. The software development lifecycle, consisting of gathering requirements, designing a solution, building that solution, testing it, releasing it and maintaining it, is based on the waterfall methodology.

Waterfall is good for...

• Small projects, since they are less likely to change

• Repeatable projects, since there is a known end goal

• Physical projects, since the architecture is already defined

And because waterfall is an established methodology, there is little confusion around how it works.

A great time to apply waterfall is when you’re porting an existing product to a new platform, such as a board game to a mobile app.

Waterfall has challenges with…

• Medium-to-large sized projects, where the scope is bound to change during implementation

• Innovative work, where the end goal is unknown at the start of the project

• Digital projects, where the underlying architecture is always changing

Unfortunately, waterfall has received a terrible reputation over the years. It’s been proven to fail more often than succeed, likely because it is frequently used for the types of projects listed above. As a result, there’s a tendency to push all requested changes into a “phase two.” The organization, having been burned before by never actually receiving their promised “phase two,” tends to push for everything and the kitchen sink to be included in phase one, thereby making the project scope grow out of control.

You’ll want to make sure you avoid applying waterfall when updating a website or mobile phone app, since infrequent releases cause user frustration and provide opportunities to the competition.

About two decades ago, agile was born. The intent was for it to be a better methodology. In most cases, it is. Agile is good for all of the projects that waterfall falls short to support.

Agile is good for…

• Medium-to-large sized projects, where the scope is bound to change

• Innovative work, where the end goal is unknown at the beginning of the project

• Digital projects, where the underlying architecture is in flux during development

A great time to apply agile is when you’re developing anything with a social media feature, since user feedback is critical to the direction of the product and ignoring it can backfire very quickly.

Like waterfall, agile isn’t always the answer. It struggles to accommodate... 

• Immediate changes, since—unlike waterfall where the business partner can request changes at any time they want—changes should only really be added in-between, and not during, sprints

• Limited employee involvement, since it requires more interaction with the business by way of a representative (like a product owner) who joins the development team

• Higher overhead, since it requires daily meetings and recurring half- to full-day planning sessions

And just like waterfall, agile also struggles with a poor reputation. Because agile is great for innovative work, that often means there is no predetermined ending, which can make the project linger. That lingering status tarnishes its reputation.

Of course, it’s important to know that many people view these “cons” as positive attributes of agile. The rigidity keeps people focused, and while business representatives may view more communication as intrusive on their time, more communication leads to less assumptions and misunderstandings between them and the development teams, yielding a better product. Problems with agile’s reputation and overhead don’t actually come from agile; they come from the hybrid state of wagile. (More on this in a bit.)

Agile won’t always meet your needs, especially when government regulations dictate your scope and timeline, such as is often the case in the healthcare industry.

As previously mentioned, people think they can combine waterfall and agile to create an even better hybrid methodology commonly known as agile-fall, scrum-fall, FRagile, and our favorite, wagile.

Many leaders and teams create their wagile process with the intent to capitalize on...

• The best of both worlds, since waterfall is good for small, repeatable and physical projects, and agile is good for medium-to-large sized, innovative and digital projects

• Less disruption, since a hybrid approach would not force extreme change on the team 

• The “special” needs of their organization, since every organization thinks they are unique and a hybrid approach will help them meet these needs

Of course, all of the above is false. People inexperienced in agile think wagile offers them the entire list above. Logically, it makes sense; in practice, it doesn’t work.

Unfortunately, wagile can’t be avoided. You’ll may find yourself in this state if you are beginning a transition from waterfall to agile.

Wagile can be detrimental to your goals. It offers…

• The worst of both worlds, since it’s intrusive on the business partners and allows for a lot of scope creep, causing projects to never end

• Confusion, since it demotivates early agile adopters and true believers when agile rules are ignored

• High overhead, since management often wants to keep waterfall weekly meetings and add agile planning sessions, daily stand-ups and retrospectives on top so they can stay informed the way they always have

During this transition from waterfall to agile, you may hear people say that agile is not working. But truthfully, when you’re in a wagile state, you’re not being agile. Agile will work just fine once you can get there. 

We’ll cut straight to the chase: there are no pros to wagile. This graph opening this section is a seductive illusion; it does not exist in the real world. Wagile is the worst.

Always… or when you want to succeed.

If you find yourself in a wagile state—either because you’re transitioning from waterfall to agile or because you were enticed by the illusion of a hybrid methodology—you’ll want to move your team through it. Here’s how to make the full transition to agile.

Let’s take a look at a visual timeline of events, starting with phase one, which shows a team using the waterfall methodology. They realize waterfall has issues and want to transition to agile. Inexperienced IT leaders and business contributors assume the change to agile is as simple as turning on a light switch, that everyone will be working in this new methodology and that things will be better from here on out.

As soon as the change is implemented, people enter phase two. They quickly realize change is a process, and it will take some time to transition from waterfall to agile. However, inexperienced individuals will assume everything will slowly improve over time, and they will eventually be agile. We like to call this incline “the path of hope.”

People enter phase three just as fast as they entered phase two. When an agile transition starts, things almost immediately get worse. If you are experiencing this right now, we have some good news and some bad news.

The good news is that this is not unusual. You are not alone. Every person and every organization experiences this phase. Some people will resist the change. Others will be on-board but simply do not have enough experience to consistently do things the right way. As a result, performance drops. As people try to figure it out, deadlines and stress creep up on them. And what does anyone do in times of stress? They revert to what they know to get the job done, which in this case is waterfall.

As you already know by now, wagile is not a methodology; it’s an awkward transition state between waterfall and agile. Being in a wagile state is nothing to be ashamed of. It honestly can’t be avoided if you’re transitioning from one methodology to the other.

Teams can be in phase three for a very long time. Every team that’s gone through an agile transformation has experienced the wagile state, and every team that will make the agile transformation in the future will also experience the wagile state.

How do we know this? Because transformations involve people, from both IT and the business. It’s extremely rare to find an individual that can navigate a challenging transition without any setbacks. Now imagine finding a team of those individuals working together at the same organization. Do you see our point? Wagile exists because it takes just one person on the team to struggle for the entire team to struggle.

Phase three is the most critical. One of three things will happen:

1. Leadership abandons the agile transformation and returns to the waterfall methodology. This is acceptable in some situations and better than remaining in a wagile state.

2. Leadership realizes they are in a wagile state and makes the effort to completely let go of the waterfall and agile hybrid, thus allowing them to move to an agile methodology. (Represented in the graphic above.)

3. Leadership holds onto the flawed idea that a hybrid of waterfall and agile methodologies is the best path forward, thereby remaining in a low performance environment indefinitely... until one of the other two outcomes occurs or the company goes out of business. 
   
   If leadership opts for choice number two, letting go of wagile so they can move to agile, they enter phase four.

Phase four is when the team finally realizes that adopting agile will be a bumpy road, with many successes and failures. They may experience a few good sprints and then all of a sudden, the business representative introduces scope creep and doesn’t want to wait for the next sprint. Or perhaps someone from another area of IT transfers onto their team, and that individual struggles to let go of waterfall and wagile and fully embrace agile. Or a thousand other scenarios. Eventually, though, the team will stop using waterfall and be agile. This journey is a long one but well worth it if your projects meet the criteria we reviewed earlier.

While some people reject agile and believe waterfall is the only methodology worth using, there are also many people who have adopted agile and now reject waterfall completely. We won’t exclusively recommend one methodology for you because, frankly, both points of view are flawed. Evaluate your organizational needs and the projects you’re working on before you make a decision on which methodology is right for you. But, as you make that decision, we will give you this definitive guidance: don’t choose wagile—and if you find yourself there, actively transition out of it.

For tips on how to escape the dreaded wagile state, read our previous guide, Move away from wagile: How to break anti-patterns in agile transitions.

Kevin Miller

Kevin Miller's mission is to close the gap between IT departments and the businesses they support by opening IT to change. His goal is not to change what IT does; it's to change how IT does it by changing how IT thinks. Kevin's expertise is helping organizations of all sizes adopt and embrace agile methodologies for faster deliveries, reduced costs and increased customer satisfaction. He possesses numerous degrees and certifications and more than 20 years of progressive experience in a wide range of technical areas, including software development, operations, project management and leadership. His hobbies include learning, teaching, scuba diving, playing chess, and traveling. Kevin can be reached at http://www.DeltaTechnology.net.

Tommy van Schaik

Born and raised in the Netherlands, Tommy van Schaik found out early on that his natural habitat was between business and IT. After graduating with his masters in the area of Business Process Management & IT, he started his career as a Dutch Airforce Officer. After a beautiful ex-pat adventure in the United States, he switched to civilian life and moved back to Europe to continue his career in the Aerospace and Defense domain. Currently, Tommy is working as an IT project manager for the NATO Communication and Information Agency. If he has any spare time left between family, work and being a Pluralsight author, Tommy tries to stay fit either in the ring or on the basketball court. Tommy can be reached http://www.linkedin.com/in/tommyvanschaik.