Are you tired of hearing that Agile means no documentation? That's nonsense. Agile means documentation that actually serves a purpose, and visibility—well, that's a non-negotiable in an agile process. It's how we build trust. And yes, it's often through documentation that we provide visibility.
Hi, my name is Lance Dacy, otherwise known as Big Agile. And I would like to talk today about documentation, the right amount of documentation, and what does Agile really have to say about it.
Understanding the Agile Manifesto's Stance on Documentation
I want to clarify the idea that when we read the Agile Manifesto—I teach a lot of entry-level Scrum classes and people often think that in the Agile Manifesto where it says "we value working software over comprehensive documentation," we're saying documentation doesn't matter. All we're trying to say is we value the things on the left more than the right in Agile, but it doesn't mean we get rid of them at all.
We've all been a part of an organization where documentation is waste. I have no doubt about that. But documentation in itself is not waste. It's how it's used, who decides to maintain it, and is it always up to date?
The Three Pillars: Transparency, Inspection, and Adaptation
But more importantly, I want to talk about visibility. So let's kind of discard documentation right now. Let's talk about visibility, transparency, inspecting, and adapting. That is the empirical nature of any agile process and is what we consider the pillars of success for that process.
Transparency
Transparency is the idea that we're showcasing everything that's going on. We're making things visible, not necessarily for us to coordinate better—that's part of it—but so the business can make better decisions. If I can't see it, I can't decide about it. So we want to include the business in that and document anything that they think might be required to help them do that.
Inspection
But transparency alone is not enough. We need to inspect what's happening. So transparency is showing what's going on. Inspection is kind of clarifying why did that happen or what is happening?
Adaptation
And then adaptation is responding to that in a way that helps us get better as we go. And I often say Agile doesn't necessarily mean faster, it just means sooner. And what's the difference in that? Well, the idea is just adopting Agile doesn't mean we magically go faster. That's like a magic diet pill. Really what it means is we're going to focus on continuous improvement every single day. Every single thing that we do, we're going to try to get better. The byproduct of that is hopefully we become more efficient and effective at what we do and we get higher quality. Well, that's all going to kind of manifest itself in a way that allows us to deliver sooner.
Agile vs. Waterfall: It's About Smaller Slices
And Waterfall—you look at the Waterfall method, all of those steps in Waterfall, they have to be done. Agile just says we want to do smaller slices of that because everything in Waterfall is TBD until we actually deliver. And so I did a video on computational irreducibility, which is the notion that we really don't know if we've accomplished our goal until we've accomplished the goal. And a lot of times we don't know that until we show it to the stakeholders. So Agile in itself is going to breed that intensity of inspection, adaptation, and transparency along the way.
Debunking the Myth: Why Organizations Need Documentation
So I'd like to talk about the myth that documentation is frowned upon. Most organizations love documentation. They love processes, they love tools. The main reason is when they have attrition and they hire people to come back onto the organization, do we want them onboarding and contributing as quickly as possible or do we want them languishing?
And I know you all know the answer to that, but a lot of ways the best way to get people integrated into how we work is that they can refer to some kind of visualization of what we're doing or some kind of document to help them get up to speed quicker and provide as a reference. So documentation's important just as it is for the user using the product. But more importantly, I want to talk about it from a stakeholder perspective.
A Real-World Agile Transformation Story
So I worked at an organization one time where we were experimenting with Agile. We weren't doing the agile processes early on in our growth, and we finally decided to pivot to that. And it was one of the best experiences I had. It was really the first experience I had with Agile transformation. And so I have a lot of scar tissue from that one.
But I also want to say it was probably one of the best teams that I worked with because it was small enough. We only had about 200 people in the organization and the development team grew from 10 to maybe 20 or 30 people. Pretty doable if you're trying to focus change in the organization.
The Product Backlog Iceberg: Beyond Sprint Work
But let's talk about documentation a little bit. When we have teams that are working on product development activities, it doesn't mean that the requirements just come into the team and they work on them. There requires a great deal of activity ahead of the team to build those products.
And so if we're practicing Scrum, I typically point out that Scrum will say we start with this thing called a product backlog, and we kind of want it to look like an iceberg, if you will. Small, detailed work items at the top and larger ambiguous items at the bottom.
And so then we have this idea that we go through a sprint to build an increment of the product so that it can be inspected and then feedback goes back onto the backlog. Usually our Scrum-level classes focus on this activity. I call it the right-hand side of the product backlog.
The Left Side: Roadmap Planning and Software Envisioning
But there's a whole host of work that has to happen on the left side of the product backlog that often is overlooked in a Scrum implementation. I kind of just brand it as roadmap planning. Software envisioning is not really part of the build-out in Scrum. In fact, we have to dedicate some time in the sprint to look ahead at the backlog and get it prepared.
And in the old days of practicing Scrum, they would do quarterly activities. So the first two months would be the analysis, design, and what we call product backlog refinement now. And then the third month of that quarter would be the sprint to actually go build it. Well, they did away with that, and backlog refinement just kind of has to happen ongoing.
A Practical Example: Visualizing the Entire Product Development Flow
So I've got an example of a visualization that I worked with the team, and this was what we would call software envisioning over here on the design side. The product team would be working with the stakeholders to ideate—what are some good ideas that we want to work on? And we visualized that process using documentation as well.
So we would print out what we're working on in the current state. And it did take quite a bit to keep this board up to date. And this was of course back in the day where remote working was really not one of those activities that happened a lot. If you worked at home, you didn't have a good way to connect with people.
Creating Physical and Digital Visibility Boards
So we were often in the office and you can of course recreate these digitally as well. But anybody in the organization could come to this room. We put it in kind of like a playroom because a lot of people would go there to decompress, and it would be there and available for people to read or show our customers.
And so this design area was like all the ideation that was happening, and we would start attaching designs and ideas to those product backlog items and eventually they would be ready to go ahead and start working on. And so we would have this realization board that was everything that was happening in the sprints at a high level based on the stakeholders' documentation and their needs.
And so you can see the different colors meant something for this team. You could attach designs to those things. And I often say it's the same way digitally when you're creating an epic, we can attach documentation to that. So when the team starts breaking it down into the work of the sprints, then it'll be easier to see.
The Three-Stage Visualization System
And so we had this realization section, which is where Scrum lived, where the product backlog item would go get scheduled for an upcoming quarter and then go through the review process and finally be made available.
And then the last part was the go-to-market board because us as developers on a Scrum team, we often focus on just getting our product into production. But there's a whole host of activities that have to happen after that. The product person is usually involved in all three of these areas and maybe some people from the team. But it was just a really good way to visualize. So help documentation, the support team, marketing and sales could all see on that final board what was actually coming out that quarter.
Release Cadence vs. Sprint Cadence
Our customers preferred a quarterly release cadence, but we were able to release to production or in a staging environment every sprint. Remember, releasing is independent of your sprints. So just because our customers wanted to release every quarter didn't mean we couldn't be agile more frequently inside of our data center.
The Scrum Board: Making Team Work Visible
And so this is what the board looked like for what I would say more of the stakeholder groups. And then as they would flow into the teams, you would have a Scrum board. So then you could walk around to the development organization and see this Scrum board where we had Q1, Q2, and Q3.
Notice how Q1 is really small or it seems like a lot more work items. That's because they're small, detailed, and ready to execute. Where things in Q2 and Q3, there could be too many changes that happen. So we didn't spend a lot of time elaborating those things. So they were more like placeholders or epics that we would be working on, but at least everybody could see the current sequence of work and that could change.
So eventually we had our teams—we had four Scrum teams here—we had our metrics that we would showcase, and then every sprint, what were the dates and what were the goals of that sprint? And then you can see the actual product backlog items move across their board in their development process, which they could deploy to production dark, which had feature toggling. Back in those days that was really difficult, but the team had that.
The Documentation Requirement: A Year in the Making
So you could go from looking at our product roadmap wall and then go around the corner. The development team had this big room and this big whiteboard and you could follow along the work items. That would be, does that require a lot of documentation?
Absolutely. It took me a year or two to get this process figured out where we could now have a way for people to pull information at the rate that they wanted. And just remember that is what Scrum is all about, is making things transparent where if anybody needs to know anything, they can go get it. So you have to build that into the process.
The Transaction Cost of Transparency
And that requires a fair degree of coordination, collaboration, and more importantly, agreement with the team overhead in their process. That's more than just about coding and testing so that they can surface this visibility. But I guarantee you it made the whole organization much better.
And so there is a transaction cost, there is cognitive load on the team to do this. And as the Scrum lead or as the Scrum Master or the Agile coach, it was my responsibility to find a way to shepherd this across the way.
Creating Process Documentation for Consistency
So this required a fair degree of documentation. And to be honest with you, I had to produce a document on how to track the backlog items as we go through the work. So I actually created a document that I managed all the time. It was pretty lightweight, but anybody who joined our development team or part of our product development group could now start realizing how do we track the information so that the system at large could continue to showcase what was happening.
And so we talked about the Scrum product, the Scrum framework. So new people went through a training class as well. But then as far as product support's concerned, we told them how to categorize some of the work items. We talked about how to handle production issues or getting onboarding of new clients and things like that. So that requires documentation.
The Bottom Line: Documentation Enables Agility
Is that bad? Should we not do that? And that's absolutely not the case. So the next time your Agile team says, "Hey, we should not have any documentation," I believe that agreeing on visibility and transparency often requires process and documentation to create. And don't shy away from that because I guarantee you it was the secret sauce of being agile in that organization.
Take Your Agile Journey to the Next Level
So like our videos here, we'd like to show you one just about every week on some of these topics. We really hope that it helps you out. Join our newsletter. Or we would love to see you in one of our public or private training opportunities. And I'd love to share more on how to accomplish this at large.
So much to do, so much work to be done. But too often we just focus on the Scrum and there's so much more to it. So hope to see y'all again in the future.
Ready to master Agile practices that actually work? Explore the comprehensive training classes that Big Agile offers and learn how to implement documentation and visibility practices that will transform your organization. Y'all have a great day.