Empirical Team Thinking

Agile development is now commonly referred to as those set of methods that come under the umbrella term agile or support agile thinking. To avoid a circular set of definitions we will define the word agile.

Agile is being quick enough to avoid or take advantage of those things that can hurt or help in your pursuit.

A  pursuit in this context would often be called an effort, work or project. However, notice the phrase “quick enough” why is that wording used and relevant. The biggest difference here and said in a very summarized way (which does not reveal the depth of thought behind it) is the difference between predictive thinking vs. adaptive thinking.

No matter who you are and what you do we will all fall prey to predictive thinking in varying degrees. To reduce the likely hood of being tripped up by predictive thinking agile frames a state of mind that leverages those around us to help us detect when we are being predictive and should be adjusting our plan based on new information rather than ignore it. Said another way is that we fail to detect/ignore when our assumption are no longer valid. So, agile is a social agreement to be empirical as a team. Agile development is a description of how we can put that framework to use as a well formed team. A framework for agile development is especially important as the complexity of our effort increases. For simple problems or things that are complicated but knowable a standardized process is suitable. For complex problems that can change just by looking at them we need an empirical framework.

Spectrum of Agile Methods

There are several Agile methods which support an Agile Development process. However, each one is best called a framework because it is applied in a unique way for each context of use.

The above diagram represents the best way we have seen to describe the agile development methods available. The ones on the right are more Bodies of Knowledge and are vast. It is their very nature of vast that can cripple a team of people trying to focus their effort. Typically large systematized thought models can become more than I want to think about and crowd out the purpose for me focus which is my effort/work/project or product that I am trying to build.

There are more such as TSP / PSP but, this model paints out a quick spectrum that summarizes the models of thought and those that sit on the boarder such as RUP. The bodies of knowledge on the right have some very good practices, processes and methods in them and should not be ignored just because they are vast. Often when you scale past 70 or so people on one project effort you will need many of the techniques that can be found in the bodies of knowledge on the right. Or if you are simple running a call center where things are complicated but, knowable and creativity for transcending the current established patterns is not desired then they are more appropriate. We would call this things that fit SOP (standard operating procedures). However, were creativity is desired and the problem is complex then the models that support agile thinking are the only ones that seem to work.

Scrum is by far the most popular of the agile models and has shown some of the best success and transaction because it is not vast. Scrum is a simple rational framework that can be memorized in 20 minutes. Applied Scrum is exceedingly difficult to do well because it requires a tremendous amount of discipline and challenges standing assumptions. The ability to reveal and challenge standing assumptions is what has made Scrum so successful. Scrum has proven itself beyond the realm of software development (form where it orginated) however, its language and purely rational approach are its weaknesses.

A quick list of agile methods

• Scrum
• FDD
• Lean
• XP
• Crystal
• Kanban
• Other often proprietary Special forms…

When doing software development work a popular pattern is to use the Scrum method wrapper the XP coding practices. This is recomended and has been found to be one of the best patterns for success.

Transitioning to an agile thinking process is best done by building Agile Pathways that supports and nurtures a pattern of adoption. It is important when applying agile to not toss out established processes that work but, instead reveal how they can be improved and reinvigorated. However, a solid implementation requires an almost prescriptive start. There are too many hidden assumptions that need to be re-explored and often changed.

• Some short phrases that help focus on agile thinking
  • Let the product lead
  • One bite at a time
  • No head works alone
  • Be empirical
  • Reflect early and often
  • Make it visible
  • Pay attention and adapt

The agile movement got it start in the public sector in 2001 with the signing of the Agile Manifesto. The first project done under the term agility and done with 1 week development cycles was in 1983 (more on this later).

For those new to agile, Scrum is a recommended place to start.

In all cases of applied agility or scrum the analysis sets the tone. Without agility in the analysis the remaining part of your effort will “waterfall” or fall prey to predictive thinking. The following set of techniques assumes a complex problem or situation. Analysis is looking for an opportunity to move. There are a few analytic techniques that show up

• No head works alone: peer review your thought process with someone and avoid heavy process driven peer review
• Don’t make a belief out of a model: use a data rich subject matter orientation so that you don’t try to overly tidy your thinking at the wrong time
• Highlight areas of risk and uncertainty: the number one risk is building the wrong thing! adjust your analysis to mitigate that risk first
• Conceptualize appropriately through time: watch which techniques you are thinking with when you move from past certainty to future uncertainty (i.e. accounting vs. finance)

All good agile processes follow Agile’s Big Rule and are targeted at creating and sustaining well formed teams. A good agile process contains both the seed of it’s own evaluation and the seed for the product’s evaluation.  This forms a double feedback loop. Both loops are owned and managed by the team. The team is the fundamental unit in an agile organization.  Therefore the process cannot be too big otherwise the team cannot adequately focus on the needs of the product or it’s development effort. Agile process are characterized as light weight and not large mind consuming systematized thought models

The following list of process would qualify..

• Scrum
• FDD
• Kanban
• Lean
• Agile (some methods called Agile that others have built, often proprietary)
• Crystal

Choose

1. No way, they are opposites: Kanban is for flow / Scrum batch
2. Yes, Scrum is old school big planning steps
3. Yes, Kanban minimal planning / Scrum is heavy planning
4. No, Scrum can reduce to KanBan

As much as we love scrum, even we would have to admit that it’s not perfect. Nothing is.  In  fact, a  large part of our book describes workarounds  for various deficiencies that scrum presents to us in certain circumstances.

One of the more commonly noted deficiencies in scrum is that it plans its work a whole Sprint at a  time.   This “batch” planning process  is often not agile enough  to cope with the actual rate of change of requirements.    In fact, Chapter 4.4 on PlaceHolder Stories, the  discussion  of  the  mid-Sprint  Re-planning  in  Chapter  4.8,  and  the  discussion  of renegotiating the scope of a Sprint in Chapter 4.3 are all about resolving this deficiency.

There  is  another  agile  process,  called  KanBan,  which  solves  this  problem  and  is becoming popular  for  software development projects.  In our upcoming book we will describe the main strength of KanBan and how to integrate it into scrum.

Message from Ken at the Scrum Alliance

• Scrum is a framework for iterative, incremental development using cross-functional, self-organizing teams. It is built on industry best practices, lean thinking, and empirical process control.
  
• Scrum is optimized for high yield product management and product development. Scrum is particularly appropriate for high risk, complex, large projects and can be used when other parts of the endeavor are hardware or even waterfall development.
  
• If waterfall suits current needs, continue using it.
• An enterprise can use Scrum as a tool to become the best product development and management organization in its market. Scrum will highlight every deficiency and impediment that the enterprise has so the enterprise can fix them and change into such an organization.
• Whenever an enterprise modifies or only partially implements Scrum, it is hiding or obscuring one or more dysfunctions that restrict its competence in product development and management.
• The iterative, incremental nature of Scrum puts stress on the product development organization to improve its engineering skills and on the product management organization to optimize the return on investment of every release and project.  The phrase, “That can’t be done here” really means that it will be very difficult to do so. The gap between current practices and target practices is a measure of incompetence and competitive risk.
• The use of Scrum to become an optimized product development and management organization is a change process that must be led from the top and requires change by everyone within the enterprise. Change is extremely difficult and fraught with conflict, and may take many years of sustained effort. Turnover of staff and management can be expected.
• The most serious impediments to using Scrum are habits of waterfall, predictive thinking over the last twenty to thirty years; these have spawned command and control management, belief that demanding something will make it happen, and the willingness of development to cut quality to meet dates. These are inbred habits that we aren’t even aware of anymore.
• The focus of using Scrum is the change from old habits to new ways of doing business. Scrum is not implemented or rolled-out as a process; it is used to foment change.
• Scrum is not a methodology that needs enhancing. That is how we got in trouble in the first place, thinking that the problem was not having a perfect methodology. Effort centers on the changes in the enterprise that is needed.
• Iterative, incremental development is much harder than waterfall development; everything that was hard in waterfall engineering practices now has to be done every iteration, and this is incredibly hard. It is not impossible, but has to be worked toward over time.
• Managing a release or project to deliver only the highest value functionality and not deliver the rest optimizes value is the job of product management and customers.
• Self-optimizing teams are extremely productive. When they work closely with the customer to derive the best solution to a need, they and the customer are even more productive.
• A team consists of people under pressure to do their best. Conflict is natural and the team needs to know how to deal with the conflict and have resources to draw on when needed.
• The role of an enterprises management changes from telling people what to do to leading and helping everyone do their best to achieve goals. People aren’t resources and managers aren’t bosses.

After peering closely at what got them to their current position of success we see a “well formed team”. It is amazing how quickly people in corporate america can forget the power that was present from these teams and what they are able to produce. One thing is clear. A WFT was at the heart of their success and everytime they try to create that magic without a WFT ideas just don’t seem to flourish.

Saying it this way puts and emphasis on starting with self and looking inward before blaming outward.

For an agile process to be successful, and probably any process at all, it must allow some focus on people, passions and teams working together.

• Without an aim we have no purpose and not much of a reason to be a team. When we have that product aim we can test our ability to work together by looking at the results of our product development effort.
• Without a focus on people we often on mechanics of the process to the exclusion that people matter.
• And without working towards our passions we fail to ignite the energy within ourselves and we fail to contagiously spread our enthusiasm to others around us.

When we properly include the above three in our line of vision and don’t fall prey to using a model of software development that is so complex there is no room in our heads for the above three, we can win BIG time. What we focus on matters and if the process is so complex there is little room to think about anything else, then we loose.

A recurring pattern, that we have observed in large scale adoption of Agile “The Pursuit of Enterprise Agile” . Is a successful pilot that really focuses on the people working together, can ignite a ground swell approach to adopting agile.

Typically, organizations get started by contracting with outside an outside expert who advises them how to start agile. The advice is typically some setup a pilot that will be small enough to influence yet important enough to keep attention focused. The idea behind a pilot is simple, “Let’s setup a little experiment with this thing called agile and see how it works out for you?” After a business case review or senior leadership buy-in, the pilot is set up with an appropriate amount of autonomy.

Usually, an internal person is to be part of the pilot and run it. That person is charged with setting up the pilot and documenting their observations so that the organization can learn. A team is formed, trained and started down the agile path. The team works shoulder to shoulder with an on-site agile consultant who demonstrates the habits and values of agile while in motion. Often the team quickly emulates the behavior and then falls into a rhythm of product development and delivery.

Then someone “process experts” typically take the observations of the people working together and abstract them. Now things really get whacked. The process expert believes they can take observations of the pilot and fold lessons learned into the enterprise methodology view. This is a too “hard problem” we have been trying to do this for years and are still in our infancy for understanding basic communication. However, based on what is written and documented in classic process literature you would be lead to believe we have evolved far more than we have.

After organization have absorbed the lasted round of process learning into their enterprise methodology. The very real skills and valuable team behaviors that were gained are lost. The next agile effort looks very much like the old process the organization ran before just with new words and language. The part that made the pilot effective is lost.

That is why “You Don’t Do the Agile Process You Are the Agile Process” works so well as a phrase. It constantly reminds me that agile starts with self.

People, Passions and Teams must be elevated above Principles, Philosophies and Practices. Without the first three the last three will not matter.

This is great and seeing agile gain acceptance at this level as well as publications from mainstream places like Business Week and CNN is a huge step forward.

Why they don’t seem to “get it”
Many of the remaining trends are focused on individuals and not teams. This continues to be counter to the agile movement where brilliance is really a team commodity and that corporations are still looking for the individual to lead them to greatness. Our need for education remains strong in the area of teaching people to join teams and leverage the emergent qualities of social brilliance. In other words have a conversation that establisher a trusted report so that we get smart together.