A conjoined Agile outside-in design and development process

At this point we've got a model to help us understand Agile's iterative development incremental release lifecycle, and a Garrett's model to help us understand a software design process. Both models think and work outside-in.

Common Concepts

Looking at both these models, a couple common concepts emerge.

Moving from Abstract to Detail

Both the Agile and Elements models move from a high level of abstraction to a lower or more detailed level of abstraction.

Using the Agile Development model I'm ultimately concerned with developing features, but I get there by planning iterations, and I get there by planning releases, and I get there by chartering a project.

Using Garrett's User Experience model, I'm ultimately concerned with describing an effective user interface, but I get there by understanding page structure, and I get there by understanding navigation structure, and I get there by understanding the scope of tasks and features supported, and I ultimately get there by understanding the goals of the organization building the product and the people who will use it.

Both models give useful names to these layers of abstraction: feature, iteration, release, project; and surface, skeleton, structure, scope, strategy. The names and the concerns they represent help us constrain our thinking to what matters at the particular level of detail we're concerned with at the moment.

Decision Dependencies

It may be immediately obvious that there are dependencies between these named levels of abstraction. The decisions made at a high level of abstraction directly affect the options I have and the decisions I make at the next level of abstraction.

In the Agile Development model, the features I choose for a product are a function of the goals for the product. The features in a particular iteration are chosen from the features chosen for a release.

In the User Experience Model, the navigation of the user interface of the software is a function of the features chosen for the software. The general layout, or skeleton, of a particular page of the software is a function of the features that page must support and the navigation structure of the software. The specific visual design of the page is a function of the skeleton of that page.

Evaluation & Decision Dependency

When working with software, there's a tendency to progress directly to the lowest, most concrete level of abstraction - the specific user interface for the specific feature we'd like to start building now. That's not by itself a bad thing. But, it's critical that when we make a decision on a more detailed plane, we must validate that decision with those made on the higher level of abstraction planes. That process might go a bit like this:

• Here's the way I think this screen should look for the software.
• What tasks or activities does this user interface support?
• Does this user interface support this task or user activity well?
• Is this task or activity important to our target users of the application?
• Are both these target users and these tasks or activities the ones we want to support to earn the best return on our software development investment?

In this case it's easy to see that if we don't have information about our financial goals, the users we intend to support, and their activities that even a simple level of validation is difficult. Without that information, it's impossible to say if a particular design decision is indeed good or bad. Yet, on most software projects individuals are often expected to make such evaluations on the fly without a good understanding of the decisions made at higher levels of abstraction. How do they manage that? They guess. OK, I guess that they guess. I know I do.

Guessing isn't entirely bad. In fact I'm a pretty good guesser. The difficult part comes when I forget later why I made a particular guess. Or when someone else working alongside me makes different guesses. This is often how design arguments occur. The next time you observe a couple team members arguing a design decision, instead of discussing the merits of one decision over another, try moving up an abstraction level and discussing their reasons for making that decision. You'll observe very quickly the dependency between these abstract levels of understanding and decisions we make at each.

Differences

In spite of their similarities, these two models emphasize different things. The Agile Development model emphasizes the cyclic nature of development and the dependencies higher level cycles have on the lower level cycles they're composed of.

The User Experience Model emphasizes the dependency of information acquired and decisions made at higher levels of abstraction on the information acquired and decisions made at lower levels of abstraction.

The Agile model is about planning and building software.

The User Experience model is about design decisions and their dependency.

Since we can't engage in a process of building software without making some decisions about specifically what to build, we'll need to contend with both models.

Avoid waterfall thinking

There's a misconception that we might use a user centered design process or other requirements elicitation techniques on Agile projects by first doing requirements stuff, then engaging the Agile software development to do what it does. That is; we'll use the user experience model first to create our design, then slice it into features to build using an Agile incremental development process.

There's a misconception that we that we might decide initially what features to include in software, build the software roughly, then have a user interface designer "make it pretty" as the product nears completion. That is; do the scope setting part of the user experience design work first, slice it into features to build using Agile incremental development processes, then have developers guess at best structure, skeleton, and surface design with the intent of correcting those decisions later.

Both of these strategies, while they may work in many contexts, are symptoms of "waterfall" thinking. By waterfall I'm referring to the general approach referred to as waterfall development where all work of a particular type is done in its own phase: requirements, then design, then development, then test.

Royce [ref], the person first describing the waterfall approach, never intended for the phases to be isolated - for the decisions made in a design phase to not be informed by information learned in a development phase for instance.

In an Agile context where we have the explicit goal to defer design decisions and then learn by evaluating our work in progress, both the "design the outside then iteratively build the inside" and the "choose the scope, iteratively build it, then clean up the outside" strategies are especially risky.

Fixing the scope early in a project strips out much of the purpose and power of evaluation at the end of feature development, iterative development, and incremental release.

Deferring user interface decisions till late strips away the ability for actual users to evaluate the suitability and the quality of the software for their day-to-day use. Not being able to place software in front of actually users for their evaluation strips away an important layer of feedback, and an important opportunity for change and adaptation.

To really leverage these two models and their ideas well, lopping them into pieces and sequencing them waterfall style isn't going to work. We'll need to weave them together into something better. Something that allows us to constantly make effective design and requirements decisions based on the continuous evaluation found in the Agile lifecycle.

Weaving the Requirements and Design Process into Agile Software Development If we lay these two models side by side, we can start to get a little guidance on how we might begin to weave these two lines of thought together. Recalling the cyclic model for Agile Software Development, planning and design happens inside of every cycle. Design decision making happens continuously, not all at once at the beginning. It's threaded throughout the lifecycle. Removing the design and plan thread from the Agile lifecycle would unravel it. The challenge now becomes what sorts of design decisions to make, and when. If we lay the Agile Development model side by side with Garrett's Elements Model, we'll begin to get some direction on specifically what and when.

What to Design and Plan, When

For the Product and Project

We'll need some sort of project charter to start. The Element's models Strategy and Scope planes help inform us a bit here. Understanding strategy will help us work through how the software can earn some return for its builders, what user constituencies it should support to do so, and what sorts of features it should have to appeal to and be useful for those user constituencies.

Project chartering is about Strategy and Scope.

For the Incremental Release

We'll need some sort of release plan that lets us understand what software we'll be building over time and some rough idea of the effort it will take to build it. A rough idea of scope is important here, especially the most important scope to meet the strategy goals of the project. To give rough estimates on development time, it may be important to understand the shape of the software a bit - how many screens might it have, roughly what sorts of activities will each screen support. Understand that Structure may help us guess at complexity and estimate effort to build.

Incremental release planning is about Scope and Structure.

For Iterative Feature Development

To plan iterations and deliver more precise estimates, well need a better understanding of the specific user interface of the software, more specifically how it looks and behaves in use. We'll need details about the Structure, Skeleton, and Surface of our software.

Iterative feature development is about Structure, Skeleton, and Surface.

Evaluating Up

As we evaluate finished features we need to be concerned that the finished software behaves as we'd predicted it would when we initially designed it. We'll look closely at the Structure, Skeleton, and Surface to make changes to those based on our evaluation of these features.

As we evaluate finished iterations of features, we'll begin to see a better representation of how the features work together. We'll look more closely at the Structure and Features and make changes to those based on the evaluation of the group of features.

As we evaluate finished releases, we'll better see how large groups of features coordinate to support specific users. We can determine if we're supporting users well enough to earn the return we expected on our software. We'll look closely at the Scope and Strategy and make changes to those based on the evaluation of the release.

Merging the Models into an Agile Requirements Model

If we flip and merge these two models we might end up with something a bit like this.

The simple view of the outside in agile model threads different types of design activities into the Agile lifecycle.

The Planes

Planes in this Agile Requirements Model correspond with both an Agile Development Lifecycle, and an element of design and requirements concern. The levels are arranged from abstract to detailed.

Arrows Point the Way

Think of this chart as a simple board game. Start your token in the upper left circle to design and plan for a project. Notice that you can move down the planning and design stack until you reach the feature level, then you can start to move up the evaluation stack. From any circle in the evaluation stack, you can circle back to design and plan in that same plane. The circles and arrows give us an idea of the dependencies involved when designing and documenting requirements and planning our Agile project. While there are always exceptions, focusing on the design, planning, and evaluation concerns at the right abstraction level works pretty well.

Now What? - next steps to implementing an outside-in process

At this point you may have some understanding of the Agile value system and the Agile lifecycle. The Elements of User Experience model gives some important guidance for understanding the dependencies of concerns when making software design decisions. The merged outside-in Agile model gives us some guidance on what design concerns to think about at what time in the Agile Development Lifecycle.

Thinking about a conjoined software design and development process won't get your software written. We'll need to do something specific.

The remainder of this book focuses primarily on what to do in an Agile project to design leveraging knowledge of business and user goals and feedback from constant evaluation. The techniques described make up a toolbox of techniques for anyone responsible for determining requirements, planning, and actively evaluating and guiding the development of software in an Agile context. The techniques are arranged from most abstract to most detailed, just like our design and development approach.

The foundation for collaboration describes the fundamental techniques used for practicing the design techniques collaboratively - critical in a performing Agile environment.

Don't follow this process - use the process as a framework and stay adaptive

A friend and colleague, Christine Perfetti drew this simple, but striking picture during one of her presentations.

She explained the picture roughly this way.

A methodology is a documented and regulated process. A process is composed of techniques practiced at different times as orchestrated by the process. Tricks are the situation specific improvisations we do by combining techniques, inventing new techniques, or using a technique for an alternative purpose.

Christine explained that she and her colleagues at User Interface Engineering had been studying successful design teams for years. Their goal had been to identify the best performing design process. What they'd actually found was surprising.

Until Christine and UIE publish results formally, this is only anecdotal evidence that following a strict process or enforced methodology is a bad idea.

This finding make sense. When we practice a process we do so in the context of the team we have, the product we're building, and the project constraints at the time. The best process adapts itself to all those things. Hardening that process into a methodology has the result of disregarding that context - or not continuously considering it in order to make process adaptations. In an Agile context especially, one where we continuously reflect on and alter our process in response to changing context, hardening your process is an "illegal move."

Treat the process described here as a framework that gives a general idea of the timing in which to perform specific techniques, and create specific models. Treat the techniques and models described here as your tool-kit full of techniques. I'll strive to give you enough detail to give you the confidence to begin practice of the techniques strait from this text.

Given practice of the techniques and an attitude of adaptation, you'll invent your own situation specific tricks. Practice those tricks long enough and consistently enough, and those tricks will evolve into regular techniques. The improvisations we make today, become our day-to-day practice tomorrow. That's the level of process adaptation and invention that's really necessary for a team to be successful in your environment.

The techniques described here are "edited for agility"

Often when a movie starts on television or on an airplane you'll get that annoying disclaimer saying the movie has been "edited for television" which I always take to mean they've chopped off the right and left sides of the picture so it fits on the screen and they've removed bad language or explicit scenes that might offend prime time viewing audiences or some of the other folks on the airplane. The television or video screen along with the context of an airplane or home has some constraints a movie screen in a theater doesn't.

Similarly, practicing these design an development techniques in an Agile context comes with the constraints of that context. Typical Agile projects might come with time and resource constraints not true of other environments. Typical Agile projects might be staffed with cross functional collaborating teems where the responsibilities of specific roles are often blurred. Techniques that work well in an Agile context can't rely too heavily on specialist knowledge and should leverage collaboration.

The techniques described in this book have been tailored to work best in the context of a typical Agile environment (if there is such a thing as a typical Agile environment.) They're tailored to both balance constraints and leverage advantages found there.

Experienced UCD and business analysis practitioners will see techniques like they've practiced before, but with some adjustments that make the techniques quicker to practice, more collaborative, and the results of the activity more consumable by inexperienced practitioners.

Those familiar with Agile development environments but unfamiliar with UCD or other analysis approaches will see techniques that are easy to fit into their current Agile approach. These techniques may seem similar to techniques you're already practicing - in some cases these techniques replace an existing practice.

All these techniques are described simply from a practitioner's perspective. They've all been "field tested" and benefit from lots of use by lots of practitioners. You'll see sidebars written by people like you describing their experiences with these techniques. For each technique you'll see "gotchas;" common mistakes and things to watch out for when practicing these techniques along with some ideas on how to fix or avoid them.

Design and analysis techniques build on a foundation of collaboration techniques

You'll encounter two strata of techniques: design and analysis techniques arranged into specific subject areas, and foundational techniques that help with basic modeling and collaboration skills.

You'll need the foundational techniques to support the practice of the design and analysis techniques. It's actually the foundation based on collaboration and low fidelity modeling that makes these techniques most different, and most applicable to an Agile development environment.

Concepts support design and analysis techniques

Design and analysis techniques are usually directed at building a model to distill information, progress design, or evaluate an existing design. Basic concepts support each technique. Understanding those concepts will help you practice and lead others to practice these techniques.

Practice practice practice

One characteristic of a technique, and these techniques especially, is that benefit from practice.

You'll find you feel awkward practicing the first time. Choosing a simple problem to tackle, and a collaborative group of friends to work with will give you the early win that's helpful for building confidence. You'll find that the more you use these techniques the more confident you become. You'll focus less brain power on practicing these techniques, and more on the actual problem you're using them to solve.

Starting with a story

The next section tells a story describing a simple Agile project while giving you some guidance on where and when techniques were practiced. The remaining sections of the book are organized by topic in the order you're likely to encounter each topic in an Agile Outside-In process. Within each section you'll find stories that help to contextualize the techniques described there.