How to apply Scrum in large projects?

Introduction and Context

There is a general perception, at least in some circles, that Scrum can be used only for small projects. This perception arises partly from the Scrum framework that stipulates small team sizes of 8-10 resources and a daily stand up meeting of no more than 15 minutes. Many organizations and teams new to Scrum raise doubts and fears about the applicability of Scrum framework for large projects. As an example, how will Scrum work for large projects requiring 50 people or more?

This article discusses the adoption of Scrum framework and the application of Agile principles for large projects. The distinction needs to be noted that applying “Scrum for large projects” is different from scaling “Scrum across the enterprise”. Scaling Scrum across an organization executing multiple projects, each consisting of 8-10 team members is different to using Scrum on a single large project requiring 50 or more people. While concepts such as “Structuring of Scrum teams” and “Scrum of Scrums” will be discussed, it is not within the boundaries of this topic to discuss concepts such as SAFe (Scaled Agile Framework).

The Case Study

Let us consider the example of a large ERP (Enterprise Resource Planning) Project in a medium-sized organization and discuss how Scrum framework and principles can be applied for such a project. Many people may be of the view that Agile approaches cannot be adopted for large-scale ERP or similar enterprise application projects. Hopefully, discussions in this article will persuade readers to think differently.

Case Characteristics

Project type: ERP Project (Business critical operational system) for one business unit of a large enterprise

Project budget: $6m (For implementation)

Project schedule: 18 months

Estimated team size: 40 FTE (Full-time equivalent resources)

7 Key Questions

Should the project be outsourced to an external supplier or be executed internally?
Can such projects be done using Agile approaches such as Scrum?
How are requirements managed if Scrum is adopted?
How will the team be structured?
Can the team be distributed across multiple locations?
How will the project be scheduled?
How should the project be governed?

Discussions

Outsourcing versus In-House Execution

The question of whether the ERP implementation project should be outsourced to a Supplier or be executed in-house within the organization is a decision relating to the IT Strategy of the organization. Usually this decision depends on factors such as Cost, Capacity - scale and availability of resources and skills, Risk, Physical space and location, etc. However, there is an implication of this decision on the question of adopting an Agile approach.

So, if the decision is made to outsource the ERP project to a Supplier, then it implies that the Supplier should be willing to accept responsibility for the success or failure of the project. Accountability should always be with the Customer organization. It is the view of this author that any project delivery approach or framework including “Agile” should not be dictated in an outsourced engagement. It should be left to the discretion of the supplier organization to deliver the project using an approach (Waterfall or Agile) they deem fit. Of course, many organizations have standards and frameworks that need to be adhered to by Suppliers and that may influence Supplier selection. It is better for Customer organizations to focus on communicating what they need and refrain from “directing” their supplier organizations on how to do the work.

If, however, the decision is made to plan and execute the ERP project using a mix of in-house and supplier resources then the customer organization must take full accountability and responsibility for the success and failure of the project. Agile approaches such as Scrum will work well even when resources from multiple organizations are engaged on a single project. However, the commercial terms of engagement of Supplier resources must be based on T&M (Time and Material) and preferably on a full-time basis for the whole duration of a Sprint or even multiple sprints. Responsibility for the project should not be assigned to individuals from Supplier organizations or freelance contractors. Obviously, individuals are responsible for their tasks and inputs to the project but not to the project itself.

Scrum for large projects

Scrum can certainly be applied for large projects by thoughtful structuring of the product or system (project) and forming multiple Scrum teams that work on individual or groups of components / features of the overall product. Features and components are two key abstractions used for architecting and building large software applications and systems.

Features are behaviours of the system that fulfil some user need and differentiate solutions. Customer Order Management, Product pricing, Returns, etc. are features of an ERP system.
Components are distinguishable parts of a system that provide common functions needed to implement features. Databases, Front-end or User-Interface, Workflow, Etc. are Components or building blocks of systems.

One of the key principles of Agile that focuses on delivery of value, emphasises features (usually represented in the form of User Stories) that solve User needs. However, large-scale enterprise systems are built out of components that deliver specific functions of the overall system. Components provide a foundation for fast system evolution. Thus, it is important to combine the two aspects of Components and Features to evolve robust and resilient systems that deliver value to users.

Fig 1 illustrates how a large ERP system can be abstracted as Components (Building Blocks) and Features (Business Processes).

Fig 1: Component and Feature abstraction of a large ERP system

The various components such as Infrastructure, Kernel, Database, Application such as Materials Management, Sales and Distribution, Pricing, Reporting, etc. and the Presentation layers are all discrete individual components of the system that connect to form the whole. It is possible to group various related functions together to create business processes or features of the system, such as Order to Cash or Procure to Pay process. Users typically interact with the system through these features.

In the context of managing requirements for Implementing the ERP System, it is highly recommended that a single set of Backlog Items be created for the entire system. That single Backlog list should be groomed (defined, arranged and detailed just in time) as per priority and dependence between features and components. It is not recommended that separate sets of Backlog Items be created according to features and components. It is easier and more practical to deliver value to the customer or end users from a single list. A good analogy as to why a single backlog list of prioritized components and features is more useful is related to the way a cake should be sliced – A cake should always be vertically sliced through the layers and eaten rather than be sliced horizontally along the layers of base, core and topping.

Scrum teams can be structured along the lines of Components and Features as shown in Fig 1. In this example, there are three Scrum teams consisting of ~10 members in each team. Two teams deliver large processes (features) such as Order to Cash and Procure and Pay and one team delivers all the components that make up the system.

Individual Scrum teams can also be distributed across geographic locations as shown in Fig 2.

Fig 2: Distributed Scrum team (Component or Feature Team)

Every Scrum Team for the project will follow the Scrum framework guidelines and have no more than 10 members, including a Product Owner for that specific Component or Feature and a Scrum Master. It is also highly recommended that there be a “Chief Product Owner” to own the complete product (ERP System) and the complete list of Product Backlog Items. The Product Owners for the Component or Feature teams such as “Order to Cash” Feature will own the backlog for only that feature. While it is not essential to have a “Chief Scrum Master” for the overall project, it may be help if the Product / System is developed using many Scrum teams.

Fig 3 illustrates the options for structuring multiple Scrum teams along Components with a common set of Product backlog.

Fig 3: One Product and multiple components

Fig 4 illustrates the option for structuring a Feature-based Scrum teams that draws team members from separate Component teams.

Fig 4: Combined Feature and Components Team

Regardless of how the Product / System is deconstructed into its components and features resulting in the structuring of multiple Scrum teams, a typical issue faced by large projects is the coordination of inter-team development according to Agile principle of inspection and adaptation. A good way to overcome this challenge is the concept of “Scrum of Scrums” which is illustrated in Fig 5.

Fig 5: Scrum of Scrums

In the Scrum of Scrums, one team member represents each of the Component and Feature teams in the overall Project Daily Scrum ritual for inspection and adaptation. In this daily stand up, the representatives from the Component and Feature teams provide an update on the progress of daily tasks and backlog items and discuss higher-level obstacles that hinder progress. Also, just as representatives from all the component and feature teams participate in the Daily Scrum, they will also participate in the overall project Sprint planning, Sprint Review and Sprint Retrospective rituals and act as the integration unit for the work delivered by component and feature teams.

Another challenge faced by large projects involving several Scrum teams is the issue of coordinating releases by various teams. The concept of “Release trains” illustrated in Fig 6 helps overcome this challenge.

Fig 6: Release Train Concept

The Release train is an approach to aligning the vision, planning and interdependencies of many Scrum teams by providing cross-team synchronization based on a common cadence (cycle of sprints and rituals). The objective of a Release Train is to focus on a fast, flexible flow of the overall product.

Guidelines for Adopting Scrum on Large Projects

Scrum can and has been successfully adopted for large projects. Structure several smaller Scrum teams according to a hybrid model of Components and Features. This way it is possible to scale several Scrum teams of 8-10 members to a large project of 50 or more team members.
Designate Product Owners and Scrum Masters for every Scrum team led by a Chief Product Owner and Chief Scrum Master for the overall product / project.
Use the concept of “Scrum of Scrums” or program level scrum for multiple teams to coordinate their inter-team work.
Use the concept of “Release Trains” to align the Product Vision, Product Planning, Sprint execution, Review and Retrospective and most importantly component and feature releases to facilitate a fast and flexible evolution of the product / system.

References and further reading