Throughout my journey of helping teams deliver great products, one common theme tends to stand out (especially for larger organizations). That theme is that an agile process can't provide long term planning.
I contend that we do more planning in agile than most other processes. The difference is that it isn't a phase of planning; its on-going. I believe most organizations have at-least 2 questions of our development team when undertaking any type of work:
1) When will this work be done?
2) How much will it cost?
If your organization has those questions, we should absolutely be able to answer those questions with Scrum.
Metrics
Scrum is founded on the empirical process control model. Empirical literally means
"based on, concerned with, or verifiable by observation or experience rather than theory or pure logic".
Metrics are a fundamental part of an empirical process as we try to make decisions with "known" data. We typically separate the size of our work from the duration with estimation techniques and metrics gathering of our team. We should be able to size the work our organization needs and derive how long it will take with our empirical data. For instance, our organization wants us to undertake a new feature for the product. Our team should be able to size the work of that feature and then measure their pace with velocity. If the feature is 100 units of work and the team is able to produce at an average rate of 10 units per sprint, then we can deduce that the 100 units of work will take roughly 10 sprints. That helps shape the answer to the questions the organization has:
1) When will this work be done? about 20 weeks (10 sprints * 2 week sprints)
2) How much will it cost? about $300,000 (cost of a sprint is $30,000 for all team members)
Naturally the first question is a bit tougher to answer, we usually won't spend 100% of our time on the single feature, but we can determine that about X% of our velocity is typically allocated to that feature and then start using that as a calculation. The point is that metrics should help us answer these questions and most teams simply don't do a good job tracking their metrics or worse, the metrics are misused.
Can metrics be misused? Absolutely, if metrics are used as performance indicators they will become dysfunctional and then not serve the team well. So we have to educate the organization how to use the metrics and that the metrics are first and foremost the team's planning tool. The team decides how/what metrics to track to help them answer the questions the organization/stakeholders have throughout the project. If the metrics are distracting, then perhaps the organization doesn't need to know how we answer the question. It is much like an API from the Development Team to the Organization.
Long Term Estimating
Most team's are good at low level estimates (the bigger the size of the work, the less we know about it). The key is to establish a planning cycle (I like quarterly) where the teams can be exposed to those high level ideas coming from the stakeholders, customers, etc...This allows them to be more focused on long term goals rather than just short term, unrelated activities. Given that we can size the work for low level estimates, we should also learn to size the high-level items coming to us so the Product Owner can properly prioritize based on cost and value. This helps in sequencing the roadmap items and ensuring the stakeholders are aligned to the order of delivery based on the capabilities of the team.
I like to use relative estimating techniques and if that works for us at the lower level, why not apply it to high-level, more ambiguous items. Nine months is about the longest release horizon I like to use for roadmap planning. You can certainly have a 3-5 year plan for your product, but as far as estimated / agreed upon delivery items; 9 months is usually sufficient for the organization. In addition, it is a little more accurate than a 1-2 year committed roadmap.
Ideation
Long-term estimating should lead into our ideation process. Naturally there are tons of ideas coming into the team on what to build. We have an infinite amount of work paired with a finite capacity. The organization needs a regular cadence in which we intake ideas, filter them, and decide which ones we should spend the effort on elaborating. I like to use the concept of an Innovation Council made up of stakeholders. OurProduct Owner facilitates the stakeholder's alignment on the priorities of the team. This helps prevent organizational-ADD.
Let's say Organization A has representatives from Sales, Marketing, Support, Product Management, etc... New ideas come in, they assess the value based on a certain set of criteria. They may ask the person bringing the idea to do a bit more research and provide more data. In the end, each of the stakeholders are involved in the prioritization proces so the Product Owner can facilitate alignment within the group. It is very difficult to assess the value of an item without understanding the cost. This is where our development teams can help and provide high-level estimates to help the team determine value. It helps ensure there is a cadence to ideation that can feed a healthy Product Backlog for a Scrum Team.
Given that Scrum does not prescribe a way to do this, it is up to our teams / organization to determine the best way. Experiment and determine what works best. There are even some organizations that don't need this type of process. They are happy to react to the market in the short-term. Some markets are more established and have less thrash and need longer term forecasting. How do you do long-term planning in your agile environment?
