Scenario-Focused Engineering: Take an Experimental Approach

  • 12/16/2014

Looking deeper

An interesting pattern that’s embedded in the Fast Feedback Cycle is worth pointing out. As Figure 3-2 shows, the top half of the cycle is all about clearly understanding the problem you are trying to solve. The bottom half of the cycle is all about creating an actual solution. As the cycle plays out over time, this split ends up being a sinusoidal sort of rhythm—identify a problem, try a solution, see whether it works, adjust based on what you learn, try a different solution, and keep on cycling. It’s very powerful to go back and forth between understanding and creating based on that understanding, which then leads to deeper understanding, from which you can create something even better, and on and on. Expert practitioners sometimes call the creating phase learning by making, as compared with learning by thinking, and believe that it is a crucial skill to master to get the most out of an iterative approach.

FIGURE 3-2

FIGURE 3-2 The top half of the Fast Feedback Cycle focuses on understanding the problem; the bottom half focuses on creating a solution.

Achieving a balanced rhythm across all stages of the cycle is ideal, though different teams will have natural strengths in different parts of the cycle. Predictably, most engineering teams are very strong at building. However, can they build rapid prototypes as well as they write code? And are they equally strong at generating and exploring multiple alternatives? Is the team practiced at identifying insights from customer research? Is the team good at framing a problem to identify the heart of the most important end-to-end experience to solve? What are the strengths of your team?

Haven’t I met you somewhere before?

The fundamental concepts of the Fast Feedback Cycle have been around a long, long time. In fact, you can see a very similar cyclical rhythm in many other creative domains:

  • Architects who draw and build foam core models to review before they finalize their plans.
  • Artists who do pencil sketches before pulling out the paints, and who paint in layers, adding detail and refining as they go.

Or, bringing it to a more technology-oriented realm:

  • Agilists who cap each weekly sprint with a demo or touch point to get feedback from their customer or product owner before starting the next sprint.
  • Entrepreneurs who use the Lean Startup build-measure-learn cycle to quickly find and validate new business opportunities.

People from diverse backgrounds and disciplines have independently concluded that a fundamentally iterative, cyclical approach is the most effective, efficient way of developing new ideas.

The scientific method

We don’t think this is a coincidence—at their core, all of these approaches stem from the scientific method.

While the scientific method is applied with some variation in different domains, it is generally accepted that it entails several key stages. First, you observe some phenomenon and notice something unusual or unexplainable about it. You then identify a specific question you want to answer. Next, you consider various hypotheses about how to answer that question to explain the phenomenon you observed and select a hypothesis to test. Then, you design an experiment and predict what you expect will happen, which outcome would give you the ability to prove or disprove your hypothesis. You then conduct the experiment and collect data. If the data disproves your hypothesis, you repeat the process, first confirming that you are asking the right question, then trying a different hypothesis and experiment, and on again until you land on a hypothesis that actually works to explain the phenomenon you observed. Scientific rigor is achieved by continually challenging the dominant hypothesis through other scientists repeating an experiment in a different lab and by designing new experiments to continue testing the validity of the hypothesis in other situations. The parallels to the Fast Feedback Cycle are plainly obvious, as you can see in Figure 3-3.

FIGURE 3-3

FIGURE 3-3 The Fast Feedback Cycle has much in common with the scientific method.

Historically, many groups of people have approached problem solving in a way that also has roots in the scientific method. Following is a brief survey of the main inspirations for the ideas behind Scenario-Focused Engineering and the Fast Feedback Cycle, as well as related methodologies that have substantial overlap in methods and mindsets.

User-centered design (UCD)

The most substantial inspiration for Scenario-Focused Engineering comes from the field of user-centered design. Both academics and practitioners have been developing, studying, using, and applying user-centered design techniques for decades. Many universities offer degrees in interaction design, user research, and numerous related specialties.

The core ideas behind user-centered design are central to this book: develop empathy with your customers, use observation to discover unarticulated needs, use storytelling to frame the problems to be solved, brainstorm and visualize alternative solutions, test prototypes with customers to get feedback, iterate repeatedly at increasing levels of fidelity in response to customer need, and mindfully narrow the funnel of solutions at each iteration to ensure that the best amalgamated solution is delivered. The fundamentally experimental, cyclical nature of a user-centered design approach is a close analog of the scientific method and is embodied in the Fast Feedback Cycle.

A mainstay of user-centered design is a form of logic called abductive reasoning.3 Unlike deductive or inductive logic, abduction is a form of logical inference that takes a set of data and creates a hypothesis that could possibly explain that data. When designing solutions, abduction is a logical leap of mind that suggests a reasonable design that could solve an attendant problem. Logically, you can’t know for sure that this solution will work, but it might. Abduction creates a hypothesis that you can test, measure, validate, or disprove—just as a scientific hypothesis is tested through the scientific method.

In the past few years, the field of user-centered design has been broadened and is often referred to as design thinking, which is a recasting of the same core ideology and methods, recognizing its applicability to a much wider class of problems than just user-interface design. We concur that as a variant of the scientific method, these approaches are indeed very broadly applicable. They can be applied not just to designing products and services, but also to developing new business strategies, designing optimal organizations and systems, and even solving seemingly impossible world-scale problems, such as creating viable ways for people to get out of poverty and improving community access to clean water.

Agile

Agile was born from the software developer community in response to the need to build higher-quality software with less wasted effort in an environment in which precise customer requirements change over time and are hard to articulate upfront. Agile was invented independently by software engineers, but fascinatingly, it aimed to solve a lot of the same root problems that user-centered design aimed to tackle.

If you’re working on an Agile team, a lot of this chapter probably feels familiar. You already loop quickly in sprints, likely somewhere from one to four weeks in length. It’s easy to squint and see how one loop around the Fast Feedback Cycle could map to an Agile sprint, complete with a customer touch point at the end to get direct customer feedback on the increment you just delivered. It’s likely that some of the same activities we discussed already occur during the course of your sprints.

Scenario-Focused Engineering differs from Agile in two main areas, which we believe help fill in some missing gaps. First, we believe that it’s not necessary for every sprint to deliver shippable code. In early sprints, it’s often more efficient to get customer feedback on a sketch, mockup, or prototype before investing in production code. However, we completely agree that getting customer feedback at the end of every sprint is absolutely essential, whether that sprint built prototypes or production code.

Second, we believe that the product owner is actually a mythical creature. We challenge the idea that any one person on the team, no matter how senior, no matter how often they talk with custo-mers, can accurately channel exactly what real customers need and desire and provide accurate “customer” feedback at the end of a sprint. Agile oversimplified the whole business of understanding customers by giving that job to one person—the product owner. Anything beyond the most basic customer needs are too complex for that approach to work reliably in today’s market. We hope you’ll find that the ideas in this book give you practical ways to break out of that mold and actually talk to real customers.

Interestingly, most Agilists have concluded that shorter sprints work better than longer ones: sprints of one to two weeks are better than sprints of three to four weeks. We have come to a similar conclusion; you should probably aim to finish a single cycle within two weeks. Left to their own devices, most teams will spend too long on an iteration. Time boxing is key to keep people moving, and Agile sprints are a really natural way to time box on a software project.

Lean Startup

The ideas of Lean Startup were made popular by Eric Ries’s book of the same name and are rooted in a combination of Steve Blank’s “Four Steps to the Epiphany” approaches to entrepreneurship and the continuing evolution of lean approaches that started with “lean manufacturing” at Toyota in the 1990s.4 The lean approach believes in finding ways to achieve equivalent value but with less work. Lean Startup takes this core belief and applies it from a business-minded entrepreneur’s frame of mind to identifying and incubating successful new business ventures. However, the ideas in it are just as relevant to developing new services, products, and offerings in established companies as they are for an entrepreneurial startup.

There are particularly strong parallels between the Fast Feedback Cycle, the scientific method, and Lean Startup techniques. The build-measure-learn cycle is really just a simpler statement of the essence of the scientific method: experiment, collect data, create hypothesis. Lean Startup’s focus on “innovation-accounting” measurement techniques emphasizes the importance of collecting data rigorously to prove or disprove your hypothesis, and to not allow “vanity metrics” or other psychological biases to get in the way of objectively assessing progress against your actual business results. The idea of a faked-up home page for your new offering is another form of a rapid, low-cost prototype intended to get customer feedback as quickly as possible to verify that you are on the right track with a minimum of sunk cost. As Ries would say, your goal is to reduce your “mean time to learning.” Similarly, a “minimum viable product” is a powerful concept to help keep a team focused, usually on solving a single scenario and seeing how it does in the market before branching out.

Mix and match your toolbox

At the core, these approaches are inherently compatible because they are fundamentally based on the rhythm of the scientific method. Therefore, it’s ridiculous to say, “We have to decide whether to use Scenario-Focused Engineering or Agile for this project.” Or “Is it better to go with user-centered design or Lean Startup?” It absolutely does not need to be an either/or proposition.

Rather, the questions to ask are “Which Agile techniques would be helpful for this project?” and “Which Scenario-Focused Engineering techniques would apply?” and “Which Lean Startup techniques would work?” Each methodology provides a set of techniques that enrich the toolbox; these techniques are born from different perspectives and emphasize different sets of expertise, but they are all fundamentally compatible. Use the tools and techniques that work for your situation and your team. When you consider them in the context of the Fast Feedback Cycle, you will find that they fit together nicely.

As we detail each stage in the Fast Feedback Cycle in the following chapters, you will see that you can choose from dozens of different specific techniques within each stage. In most cases, we encourage you to pick one or perhaps only a small handful of techniques to use at any given time—certainly don’t feel as though you have to do it all, or even think that doing it all would be ideal. Actually, the idea is to select the fewest number of techniques and activities at each stage that give you just enough information to move forward to the next stage as quickly as possible. Remember that fast iterations are the key!

We have coached many different teams as they adopt an approach based on Scenario-Focused Engineering, often with a mix of Agile or Lean Startup (and sometimes both) mixed in. Our experience is that this is not a cookie-cutter process, and there is no single best recipe that works for everyone. Each team is responsible for building its toolbox to complement the scale of its project, its timeline, the skills already on the team, and all the other myriad factors that make each team unique. Each team’s toolbox is different. However, we’ve found that a few tools are used more often than others, and these are the ones we elaborate on throughout the next chapters in sections that we call “Deep Dive.”

If you aren’t sure where to start, rest assured that experts are available for nearly every topic discussed in this book. We’ll highlight the places where getting expert help is a particularly good idea.