Agile Misconceptions

Note

This article was originally published on jdkaplan.com and is republished here with permission.

With the adoption of new techniques comes the inevitable and inadvertent introduction of bad habits. Agile transformation is no exception. The countless frameworks, certifications, and literature (including articles like this one) describe a wide landscape with varying practices and differing opinions. This lack of clarity makes it hard for teams without practical experience to adopt Agile methods effectively and easy to misinterpret best practices. In this article, I describe some of the most common Agile misconceptions, why they are harmful to teams, and what you can do to recognize and fix them.

"Agile is about moving faster"

By far the most common misconception I've encountered is that "Agile is about moving faster". This is commonly touted by teams to focus on development at the expense of strategic thinking. At best this ignores the main point of Agile which is to build better software and at worst creates haphazard systems that no one really understands, riddled with bugs and untamable technical debt.

Agile is not about making development faster, it is about getting feedback faster. Principles like "harness[ing] change for the customer's competitive advantage" and "deliver working software frequently" (Agile Manifesto, 2001) declare this outright. The goal is to benefit the customer or end-user and this is done by delivery smaller pieces of functionality at high frequency to get feedback early in the development process. Agile is about building the right product faster.

A Note on Terminology

Terms like agile and velocity quickly become conflated with their everyday English meaning but their definition is more technical and specific. Misunderstanding these terms and lack of sufficient training leads engineers and managers alike to misunderstand the goals of frameworks like Scrum.

Process misinterpretations

When adopting Scrum in particular, misinterpretation of the Scrum framework and the processes it defines can manifest in many ways. The simplest, and one of the most common, is what I call the just keep sprinting approach. It goes something like "We do Sprints! We're agile now!". There is simply so much more to Scrum than this. It often comes from a place of ignorance and usually a lack of training and education.

The second big misinterpretation we see is ignoring processes. This often manifests as team members skipping Scrum ceremonies (e.g. sprint planning, daily stand up, etc.) in order to "focus on development". In some cases this is even supported by management when coupled with the "Agile is about moving faster" misconception.

Following process too rigidly

Not every process works for every team. Equally as problematic as ignoring process is the blind following of rigid processes without accounting for the needs of the team. Context is key. For example, continuous deployment doesn't work without sufficient investment in automated testing.

In his talk on The Future of Programming, Robert C. Martin describes Agile as a set of disciplines not processes. The practices outlined in Scrum and the principles behind the Agile Manifesto describe disciplines, ways of working or doing engineering, that guide the execution of product development. These practices require team discipline and an understanding of their value, but should be adapted to the needs of the organization based on context. In short, no amount of process replaces the need for critical thinking.

Insufficient planning

A common problem among agile teams is the lack of planning. I break planning down into three types:

• Short-term planning: This is the planning outlined in Scrum that occurs every ~2 weeks or so and impacts the day-to-day work of the engineering team.
• Long-term planning: I might also call this the vision. This refers to planning high-level goals and strategic objectives at least 1 to 3 years (or more) in the future.
• Medium-term planning: This is the critical connector between the short-term and long-term plans. Typically a 2-6 month plan (often dubbed release planning in Scrum or program increment planning), this sets organizational and product objectives that flow down as inputs to short-term planning.

Frequently when teams adopt agile practices for the first time, they focus almost entirely on short-term planning and ignore the other two. "We're agile now, we don't do planning" has been uttered by more than a few teams.

Long-term planning

A long term vision for an organization is critical to success because it keeps teams aligned to a common goal and helps organizations identify irrelevant work. While the long-term change can change, it shouldn't change too frequently, changes should be justified for the business, and changes to the vision need to be strategically rolled out and clearly communicated.

While traditional project management techniques don't play exactly the same role they used to, tools like Gantt charts are still useful for forecasting the long-term plan and understanding the impact of changes.

Medium-term planning

The medium-term plan is the most frequently overlooked, but it by far the most critical to effective agile project execution. It bridges the long-term vision with the actionable tasks executed by the product team.

A medium-term plan sets objectives for the next 2 to 6 months. It might define feature-level goals (or epics) that user stories and tasks can clearly map to. It helps ensure that short-term (sprint) planning aligns to organizational goals.

Effective Planning

When done well, the long-term vision sets goals that drive a forecasted schedule which in turn helps set objectives for the medium-term plan. Medium-term objectives are tangible, achievable, and measurable. Hit or missed objectives then provide feedback to the long-term plan so the forecast can be adjusted.

This same type of two-way feedback loop is also done between the short-term and medium-term planning where short-term items are user stories and actionable tasks completed within a sprint.

Scope control

In Scrum, the sprint scope is sacred. Once the sprint is started, the scope shouldn't change. That means never adding new stories to an in progress sprint. This is important for two reasons. First, it allows developers to stay focused on the sprint goals and drive them to completion.

Second, it is necessary for effective team-level planning. Context switching can be particularly derailing. Productivity decreases exponentially as developers' focus is split or redirected. This throws off story point estimates and skews velocity metrics, hindering the teams ability to effectively plan future work.

Observations

I've seen scope changes become particularly debilitating to some teams. It tends to be driven by reactive management that feels an urgency to respond to some sort of external stimulus (e.g. a new bug, a goal or concern from senior management, or a new technology). In actuality, this tends to be an impulsive behavior driven without strategic consideration. It is very rare for external circumstances to warrant changing the sprint scope.

Effective Retrospectives

Psychological Safety

Google's Project Aristotle identified psychological safety as the most important factor in team effectiveness. Yet, psychological safety is still ignored by many teams. Psychological safety is the "shared belief held by members of a team that the team is safe for interpersonal risk taking" (Edmondson, 1999). In other words, the team environment, and retrospectives in particular, need to be a safe space for team members to take risks. In the context of retrospectives, this means being able to voice concerns, sometimes bluntly, about things the team isn't doing well. After all, we need to identify what we're doing poorly before we can fix it.

A Case Study

One of the worst cases of poor psychological safety I've experienced was one in which the team's senior manager used retrospective notes and survey outcomes to berate people for not agreeing with leadership. Surveys were not kept anonymous and as a result, people stopped responding to surveys or responded with only good things to say.

This same manager began asking scrum masters to provide retrospective notes so that they could talk to individuals about their concerns (which was typically in the form of a verbal berating). This resulted in non-existent psychological safety on the team, continually decreasing team effectiveness, and a ~40% annual turn-over rate of engineering staff.

Actionable outcomes

The other critical outcome of retrospectives is action items. A retrospective that only identifies problems without identifying actionable steps to start addressing those problems does not move the team forward. It's easy for a retro to devolve into nothing more than a vent session – and, while venting can be important – it is not the purpose of a retrospective. Identifying actions to take is key to solving problems.

One technique I've found helpful is to start every retro by reviewing the notes and action items from the previous retro. This helps address whether or not past issues were really solved and identify if they remain persistent so actions can be adjusted.

Correcting Misconceptions

The best thing to do about Agile misconceptions and bad habits is to focus on education, training, coaching, and mentorship. For teams new to Agile, avoiding misconceptions entirely is unlikely. But bad habits can be corrected with effective coaching and mentorship.

The typical Agile education (i.e. the typical 2-5 day Scrum course) is not enough. While it is necessary to provide foundational knowledge, building good engineering discipline requires experienced Agile professionals and senior engineers capable of guiding the team towards continuous improvement in day-to-day project execution.