Do Waterfall Projects Still Have a Place in Agile Organizations?

In the age of Agile, the traditional waterfall project approach may seem like a relic of the past. Many Agile coaches are quick to dismiss it, and I’ll admit, I get a slight chill whenever someone mentions “waterfall projects.” However, dismissing it entirely oversimplifies a more nuanced reality. Even the most committed agilists need an open mindset to recognize that the question isn’t whether waterfall belongs in an Agile organization, but when and where it might still serve as the best approach. In this article, I’ll explore this balance and search for clarity. So, let's do it!

Understanding the Cynefin Framework: Complicated vs. Complex

To understand where different approaches like waterfall and Agile fit, it’s helpful to look at the Cynefin framework, developed by Dave Snowden. Cynefin identifies different domains of problems, each requiring a distinct approach. Two domains are particularly relevant here:

• Complicated:
  This domain deals with situations where there is a clear relationship between cause and effect, though it may require expertise or analysis to uncover. Solutions are knowable and repeatable, often following best practices or established processes.

• Complex:
  In this domain, cause and effect are not immediately clear and can only be understood in hindsight. Uncertainty dominates, and outcomes emerge through experimentation, learning, and adaptation.

With these definitions in mind, we can begin to explore where Agile belongs. Where does Agile excel?

Agile: Deliver Early and Often

Agile is about, among other things, delivering early and often. This approach reduces risk, facilitates learning, and enables adaptation. It is precisely the mindset required in the complex domain, where cause-and-effect relationships can only be understood in hindsight. In this space:

• Outcomes are unpredictable. We cannot fully know what will happen or what is needed at the start.
• Learning is required before action. Decisions need to evolve as we gain knowledge.
• Customer needs shift. As customers experience the system, their expectations and requirements often adapt.

Complex environments demand iterative and experimental approaches—hallmarks of Agile. Agile methodologies help teams work incrementally, test hypotheses, and adapt to new learnings in real time.

Digital and cyber-physical systems, which dominate today’s development landscape, often fall into the complex category. Their intricacy and interconnectivity make it impossible to predict outcomes entirely upfront. The iterative nature of Agile is a natural fit for navigating these challenges, enabling late-stage decision-making when more is known.

Waterfall in the Complicated Domain?

For sure, waterfall projects have a place—in the complicated domain. This is the space where cause-and-effect relationships are predictable, though they may require expert analysis. Here, outcomes are known or can be reliably predicted: “If we do this, the following will happen.” In such contexts, planning everything upfront and following a sequential approach makes sense.

Examples might include:

• Tasks where repetition is common and variation is minimal.
• Scenarios involving well-understood processes, like deploying standard software or migrating data systems.
• Engineering efforts where designs are fixed and prototyping isn’t necessary.

Waterfall excels when predictability is high, risks are low, and requirements are unlikely to change significantly.

So, it is that simple?

Agile for complex and waterfall for complicated. Easy, right? Not so fast. The reality is far more nuanced. Most projects don’t fit neatly into a single domain; they often straddle multiple domains within the Cynefin framework.

For example, a project might include:

• Complicated parts: These could involve predictable tasks like system configuration or database migrations. In these areas, it is possible to plan ahead and follow established processes.
• Complex parts: These might involve creating innovative solutions, designing user experiences, or developing features for which customer needs aren’t fully understood yet. Here, experimentation and iterative feedback loops are essential.
• Clear parts: Routine or straightforward tasks, like running diagnostic tests or performing basic quality assurance, fall into the clear domain and can be managed with standard operating procedures.
• Chaotic parts: Occasionally, projects encounter crises or unexpected disruptions (e.g., a major system outage or an unanticipated shift in stakeholder priorities). These chaotic situations demand immediate action before transitioning to another domain.

Where Waterfall Fits in an Agile World

While system development rarely repeats itself—software is almost never built twice in the same way—certain implementation tasks might. For example:

• System installation or deployment: Installing the same system for multiple customers might follow a well-defined, repeatable sequence.
• Infrastructure rollouts: Building predictable, uniform environments across multiple locations can be straightforward and suited to waterfall.

These activities are complicated, not complex. They might benefit from waterfall’s structured, step-by-step approach, which ensures efficiency and minimizes variance.

Planning in Agile: A Balanced Perspective

A common misconception about Agile is that it dismisses planning altogether. This couldn’t be further from the truth. The Agile Manifesto emphasizes:

"Responding to change over following a plan."

The keyword here is over. Agile still values planning—it just advocates for planning enough and at the right time. By planning incrementally and as late as possible, teams stay responsive to change while avoiding wasted effort on over-detailed plans that might become obsolete.

In Agile, we also use forecasts and roadmaps. The key distinction between a forecast and a plan lies in their intent. A forecast is exactly that—a prediction. Based on what we know now, it represents what we expect to deliver within a given time frame, such as a quarter. Unlike rigid plans, forecasts acknowledge uncertainty and allow room for adaptation as new information arises.

The fact is, many Agile organizations are more predictable than those using traditional waterfall approaches. Agile’s iterative nature and emphasis on continuous feedback and improvement allow for greater transparency and the ability to adjust based on real-time data.

So, if Agile provides reliable forecasts and increased predictability, why should we still consider using waterfall?

Waterfall in the Complicated World

Yes, we might use waterfall in the complicated world, if, and only if:

• All requirements are well-known and complicated, not complex: The scope is clear, and cause-and-effect relationships are predictable.
• The risk level is very low: There is minimal uncertainty, and the likelihood of surprises is negligible.
• There will be no changes: Stakeholders are confident that requirements and priorities will remain stable throughout the project.
• The outcomes are highly predictable: The path to success is well-established and doesn’t rely on iterative feedback.

Even in these situations, however, we can borrow from Agile’s playbook. For instance, a waterfall setup doesn’t necessarily mean delivering everything all at once. We can still use iterative delivery. Instead of pushing the entire system to all users simultaneously, we might deploy it incrementally, allowing for phased rollouts and early feedback. This approach combines the predictability of waterfall with the adaptability of Agile where it makes sense.

Examples of Waterfall in the Complicated World

While system development itself never fits the waterfall model, certain implementation tasks or projects with high predictability may. Here are some examples where waterfall might still be possible:

1. Infrastructure Rollouts
   Setting up identical server environments across multiple locations is often predictable and benefits from a planned, step-by-step execution. Each site follows the same checklist, with minimal deviation.

2. Routine IT Deployments
   Tasks like rolling out standard operating systems or upgrading network infrastructure for multiple offices often follow a template. The lack of variability makes waterfall planning possible.

3. Simple Construction Projects
   While large-scale construction increasingly incorporates iterative feedback cycles, simpler projects, like building a standard house, are typically straightforward and could benefit from a linear approach.

Combining Iterative Delivery

Even in these waterfall-leaning scenarios, incorporating incremental delivery can add value. For example:

• Staged Rollouts: Instead of deploying an enterprise system to all users at once, begin with a pilot group to test the setup, gather feedback, and refine the process for subsequent deployments.
• Phased Infrastructure Deployment: Roll out new environments location by location, incorporating lessons learned as you go.

By blending iterative delivery with traditional waterfall, we achieve greater flexibility while retaining the predictability that waterfall projects often demand.

The Hybrid Approach: Mixing Agile and Waterfall

Organizations often blend methodologies in an attempt to manage diverse project portfolios and appease various stakeholders. This is usually rooted in familiarity with running large-scale waterfall projects—a shared understanding of how to initiate and finalize them. The mistake often lies in thinking of Agile as just another method to fit into a traditional project framework. This hybrid approach typically involves:

• Applying waterfall at the portfolio level: Decisions, budgeting, and governance follow established, linear processes.
• Attempting Agile for complex development tasks: Agile is used for iterative and learning-focused activities within these projects.

At first glance, this might seem like a sensible balance. However, the reality often plays out differently. The so-called hybrid approach frequently leads to:

• Waterfall planning upfront: Gathering all requirements, setting fixed budgets, and establishing fixed timelines without room for flexibility.
• “Agile” execution in the middle: Expecting teams—often IT departments—to adopt Agile practices within rigid, pre-defined constraints.
• A rigid output: Delivering exactly what was specified upfront, regardless of new insights or changing needs uncovered during the process.

A Recipe for Disaster

This approach combines the weaknesses of both methodologies without fully leveraging the strengths of either. The result? A rigid project setup that is doomed to fail in complex environments where adaptability is essential. Teams are left attempting to navigate a fixed-scope, fixed-time, fixed-budget scenario while being asked to “be Agile” in the middle. It’s the worst of both worlds.

And let’s be honest: putting lipstick on a pig doesn’t change the fact that it’s still a pig. Trying to blend Agile and waterfall in this way simply perpetuates inefficiency and frustration. It creates the illusion of adaptability while reinforcing the rigidity that Agile methodologies were designed to overcome.

What’s the Alternative?

Instead of forcing Agile into a waterfall structure—or vice versa—organizations should:

• Choose the right approach upfront: Assess whether the project fits into the complicated or complex domain and adopt the appropriate methodology.
• Commit to genuine agility where needed: If the project requires iterative development and learning, embrace Agile principles fully, including incremental planning and evolving scope.
• Retain clarity for straightforward tasks: For tasks that truly fall into the complicated domain, consider using waterfall where it makes sense—but don’t shoehorn Agile into the process unnecessarily.
• Adapt the project portfolio process to Agile: Organizations need to restructure their processes to fully embrace Agile rather than forcing Agile to conform to outdated waterfall frameworks.

A true hybrid approach isn’t about mixing methodologies arbitrarily. It’s about understanding the context and applying the right tools at the right time.

Build the new office with waterfall. But deliver the new cyber-physical system with agile methodologies.

Conclusion

Waterfall is far from obsolete—it’s a valuable tool when applied in the right circumstances. In the complicated domain, where predictability and repeatability reign, waterfall offers structure and control. But as complexity increases, Agile becomes indispensable, allowing teams to adapt, learn, and deliver value in uncertain environments.

The challenge for modern organizations isn’t choosing between Agile and waterfall—it’s knowing when and where to use each. By understanding the nature of the work and leveraging frameworks like Cynefin, we can create systems that thrive in both complicated and complex worlds.

However, organizations must avoid the trap of so-called hybrid agile. Combining rigid upfront waterfall planning with an expectation of agility during execution is not a balanced approach—it’s a recipe for disaster. Hybrid agile too often delivers the worst of both worlds, perpetuating inefficiency and frustration. In the end, it’s better to commit fully to the methodology that best fits the project’s domain.

Waterfall has its place, and so does Agile. But hybrid agile? That’s a pig in lipstick.

References and further reading

• "Cynefin: Weaving Sense-Making into the Fabric of Our World", Dave Snowden, 2020.

Picture of Cynefin framework by Dave Snowden, Cynefin.io, licensed under CC BY 4.0.

Last updated Dec 17, 2024