Agile Portfolio Project Management
- By Jochen Krebs
During the first two chapters, agile software development was introduced from a manager’s perspective and we discussed why organizations are getting more and more interested in agile development. Following agile principles requires a rethinking of the traditional project management style as well. In this chapter, I’ll showcase why certain project management practices need to be removed or adjusted while new agile management practices need to be acquired. These new skills are necessary for a true agile organizational transformation. Therefore, this chapter focuses on two major topics: the challenges with traditional project management in an agile environment, followed by a new agile-based definition of the role of project manager. Only the correct interpretation of agile development and agile project management will enable agile portfolio management. This chapter will be our bridge to the next part of the book.
Traditional Project Management
Project management is a well-established discipline across all industries. I once met a senior project manager who managed large enterprise information technology (IT) projects. Within his organization, he was extremely successful and had a great reputation. As it turned out, he had actually very little knowledge of information technology. His background was in manufacturing.
First I thought that, because of the size of the projects, his position was so elevated that knowledge in IT might not be necessary. But then he told me that he had experiences with all sizes of IT projects. The keys of his success were that he trusts the team and focuses on leadership and stakeholder management. He also gave me a clear understanding of why project management is a discipline on its own, which is not necessarily dependent on knowledge inside a particular industry. Too many insights can even block the real view of the project, he explained.
I thought about my meeting with him for a long time. I was skeptical and tried to imagine myself in a situation where I was, for example, managing a project in the construction industry. I then realized that even though agile project management was far from being an established concept, he had created his own recipe for project management. Years later, it did not surprise me that agile project management was founded on very similar principles: trust, shared ownership, and stakeholder collaboration. Ironically, the Project Management Institute (PMI), which takes a lot of heat from the agile community, promotes project management as a cross-industry discipline. The issue is that several concepts from the PMBOK are difficult to endorse when applied in an agile context. The PMBOK is short for the Project Management Body of Knowledge, which is the framework released by the PMI. According to this process, a project steps through clearly defined and separated phases. This proves to be extremely challenging for IT projects, where requirements and expectations frequently change.
The following are four typical deliverables that are commonly created in traditional projects and promoted through the PMBOK:
Critical path analyses
Let’s take a look at them in more detail and see what challenges they might present if applied in an agile context.
In traditional development processes, breakdown structures are hierarchical decompositions, often used in the context of work. With a work-breakdown structure (WBS), work is broken down into activities and tasks, and then these work items are broken down into more detail. (See Figure 3-1.) Sometimes cost-breakdown structures and product-work-breakdown structures are created in parallel to the WBS, always using the same approach, which is to drill down into more detail from the top down.
Figure 3-1. Work-breakdown structure (WBS)
The lowest level in the WBS hierarchy is also called a work-package, which is the level at which estimation is performed and assignments are done. The levels higher in the hierarchy are always combinations of the lower elements. So the bigger the project scope, the more branches there are in the WBS. The challenge with work-breakdown structures traces back to the problem of producing at an early stage a so-called master plan of work. Any work that is not listed in the WBS won’t be estimated. As a result, any work performed that is not in the WBS will produce cost overruns. We could argue that the WBS could be constantly updated to reflect the changes caused by scope changes, but that is hardly done in practice. The structure is difficult to adjust.
Another issue with estimating work up front is that the development team has not had the chance to work together and verify the estimates. If one estimate is significantly off, there is a chance that the other estimates are also out of line. Corrections of estimates while the project is under way are difficult to incorporate.
From a psychological perspective, the WBS is owned and managed by the project manager. Work is assigned to team members, and completion is controlled. This top-down, command-and-control management style does not work with agile teams, and it violates the principle of using a team-managed and team-organized approach.
I’ve met project managers who have told me that they will need the WBS for their own benefit, even though they won’t tell the agile team about it. These managers were so accustomed to this work product that they continued using it. It is hard to be against something that gives the project manager confidence, but if the team is not using it, why bother? Are the costs of creating and maintaining a WBS for a project manager justified? Shouldn’t the agile project manager work on something else and make better use of her time? Shouldn’t she work on something more important and relevant for the project team—for example, removing organizational impediments or facilitating a planning session?
Instead of seeing the project from the perspective of “work to be performed,” the agile team takes a different view. They look at the features and functionality of the system and document them as stories on index cards. Because the tasks in every iteration are the same, the focus of work is the story card. The activities for the iterations run parallel to one another; there is no requirements iteration or testing iteration in agile development. The activities are merged together and are continuously performed in parallel. The cards are estimated and prioritized, and the ones found relevant for the business are assigned to iterations. Remember, the feedback loop established through iterative-incremental development easily allows for this.
A Gantt chart illustrates the orchestration of the tasks according to timing and dependencies. In traditional projects, this format is frequently used to depict the project schedule. (See Figure 3-2.)
Figure 3-2. A Gantt chart
Very much like the WBS, the Gantt chart lays out the project schedule up front. The problem is the lack of predictability and accuracy of the schedule, especially for tasks late in the project. The planning window with the highest accuracy is the short time frame ahead of the day of planning. That is why iterations are so intuitive for many managers. Imagine a project manager in the beginning of a new long-term project. Ask him what the scope of work will be in iteration 20? It seems almost ridiculous to expect the manager to know, right? It is human nature that we think about the first iteration, the second iteration, and maybe a rough sketch of iteration 3. Iteration 3 will be refined once we start with iteration 2, and so forth.
Agile teams, however, know the project goals and have a rough estimate of how to get there. They know where the features and requirements stand at any given moment and the amount of progress expected to be achieved in an iteration.
Try the following experiment the next time you meet with a traditional project team. Ask for the project schedule, which is most likely shown as a Gantt chart. Then compare the actual day (the present day) with the plan. Ask a team member, “Based on this plan, you are currently working on task XY, correct?”
Based on my experience, the answer is most commonly an excuse for why the team is not working on that task, such as the schedule being outdated. The team’s good spirit and best instincts are directed most of the time to working dynamically with the project manager instead of following a plan. Developers follow their instinct and trust their common sense. Without those, the project schedule would collapse anyway, because there are so many tasks unaccounted for in the original schedule that just seem to pop up. I have also never seen a project team start work in the morning by looking at the latest edition of the schedule. “Based on the schedule, I should be working on XY.” Call it unprofessional or undisciplined; it just hasn’t happened in my world.
In all fairness, maintaining the schedule is a full-time job on its own because projects are naturally very dynamic. Occasionally, especially when executives like to get an overview of the progress of the project, it is a team effort to get the Gantt chart up to date. I call this pastcasting (instead of forecasting) because the schedule’s past is put back in order. For various reasons, the plan’s history might be put back in order, but it provides no value to the project itself.
In agile projects, which embrace change and a dynamic way of building systems, planning with Gantt charts encounters limitations and is too expensive to maintain. Most important, such planning does not reflect what is really going on in the agile team. An agile team can tell you, however, what its plan for the next iteration is and, possibly, the plan for the following iteration also.
Critical Path Analyses
Performing a critical path analysis is a technique used to identify the shortest way through the project schedule. Any delay of any task on the critical path will delay the completion date. Figure 3-3 demonstrates that activity E cannot start before C and D are completed.
Figure 3-3. Task dependencies
Knowing and monitoring the critical path allows project managers to control the progress of the project. One form of illustrating the project schedule and its critical path is the activity-on-node diagram. Each node carries the following information:
Name of the activity
By subtracting the earliest start from the latest start, the float is determined. If, for example, the earliest start is day 12 in a schedule and the latest start is day 18, we have a float of 6 days. That means this task could be delayed up to 6 days without having to sacrifice the overall schedule. Therefore, this task is not on the critical path. Every task without any float is, however, on the critical path.
Although this technique seems extremely powerful for controlling projects and puts emphasis on the critical tasks in the project, it has its shortcomings when applied in an agile project. A typical sequence for traditional project management is to have the up-front planning followed by the execution. The controlling mechanism of the critical path analysis does not include the adaptive development that takes place in an agile project. As a matter of fact, in agile projects tasks are performed in parallel—for example, writing small units of tests followed by the appropriate messages and methods in objects. A clear separation of tasks is not possible. The detailed level of the tasks would be measured in minutes rather than in hours and days, which are typically the unit of measure captured on an activity-on-node diagram.
In addition, agile projects are not considered to be late as long as the iteration is completed on time. Instead, agile teams measure how much progress has been accomplished within the last iteration. This approach creates a totally different perspective about a project’s schedule.
On agile projects, the critical path is nested inside the iteration, and every iteration is on the critical path. Daily stand-up meetings expose the most critical impediments. Suppose a team planned 50 story points and completed 55. They made more progress than planned. If they again plan 50 points for the following iteration and then complete 58, the team seems to be able to deliver more than planned. Having a rough estimate for the entire project, the average progress can be used to draw conclusions on the final delivery. Having each iteration serve as feedback about progress based on tangible output seems to be more effective than projecting the completion of tasks on paper.
Project reports present the state of the project to stakeholders. They provide information about how the project is doing compared with its original plan. In a perfect world, a project manager can ask her team members about the progress of a task. We know by now that IT projects are often far from a perfect world. So there are basically two problems with this approach. First, the plan is often outdated and cannot be compared with reality, and second, the tasks are too broad to measure as “done” or “not done.”
For example, a broad schedule might include a task of “Write User Guide,” which is estimated to take 100 person-days. It is extremely difficult for a technical writer to give a status report on this task not knowing what is left to complete. A typical answer is 50 percent, 80 percent, or 90 percent done. Also, the last 10 percent might take three times as long as the first 90 percent. So how predictable is this form of status reporting?
Executives who believe in status reports might be disappointed when they realize that a project that was reporting positive figures all of a sudden has to report the breaking news, “We are running late...” Whoever believes that the project status reports are exclusively created by the project manager is misinformed. Usually, status reports are a team effort that takes a lot of energy out of the entire team. But worse, while the team is providing status updates and explaining the work it has performed, team members are not making progress on the project itself.
Again, agile project teams use the increment and the iterative rhythm to report their status. For example, using two-week iterations, a project team can demonstrate its accomplishments in producing executable software. The features and requirements are either done or not done, and every claim or measurement is tangible. Using the earlier example, a team can report that a feature was not only converted to running software during an iteration, but it was also documented in the user-guide documentation. Executable software, a build, automated test cases, and a piece of documentation have been completed. All aspects of a manageable piece of functionality have been completed.
In addition, executives and stakeholders can always sneak into the daily stand-up meetings and observe the mood, progress, and issues of the project firsthand.
Summary About Challenges
By examining four typical work products found in traditional project management, you saw the challenges that arise when they are applied in the context of agile projects. The way that scope (via WBS), schedule (via Gantt charts), and progress (via progress reports) are managed and controlled (through critical-path analyses) must be adjusted to techniques that are easier to apply in the context of agile project management. The fundamental idea of project management is, however, still the same. We are still planning work, but agile projects plan around requirements, provide schedules through iterations, analyze the progress (iteratively and daily), and report on the progress to stakeholders through demonstrations and tangible progress.
Keep in mind that the challenges presented here concerning the traditional work products are based on their use in agile IT projects. In other, more predictable and reliable, industries, or in very short projects, their use and application might be appropriate. This is, however, outside my judgment.