Project documentation

I got the following questions from a friend who wants to understand agile practices prior to implementing them on his project. I will try to answer them based on my experience.

” What needs to be documented in agile ?”

The Agile principle is “Working software over comprehensive documentation”. Documentation can be an intermediate product towards delivering this working software. It can be created for reference or for articulating business rules and in case of distributed development, to communicate ideas.

Chris Matts puts this well – The real aim is to build the knowledge in the minds of the developers and business users rather than focus on creating a document that may well not be used.

Having said that, there are real requirements to produce documentation as an end product such as for legal compliance or for product patents. Sometimes documentation is required for future support activities, but there are other better ways to handle these requirements.

“How do stories get documented ?”

At Thoughtworks, we have developed a practice to determine project /programme requirements in facilitated workshop sessions. Stories are captured during those sessions, typically by the clients themselves, using the following format.

As a .. <user who requires this feature>
I would like .. <goal>
So that… < business justification

The stories captured in this fashion supported by process flows and scenarios are sufficient for estimating and prioritising and thus, for further planning.

During the development phase, the stories prioritised for development are further corroborated with acceptance criteria, a set of conditions that the story must fulfill to be considered complete.My colleague Dan has an excellent article describing the intent and the anatomy of a story in detail.

“Are the story documents kept after its been implemented ?”

XP practitioners derive a catharatic experience from tearing story cards after they have been implemented.

In practice, the story documents are largely captured electronically either on the WIKI or in Word/ Excel documents or as automated acceptance tests – so does not make sense destroying them. The supporting documents that you build around documenting your business rules, UI design or implementation logic are useful even beyond the implementation.

We have used FIT tests as living documentation which live beyond the development phase. During development, they are useful for specifying acceptance criteria in a testable form and beyond that, they are useful as a regression test suite.

“How are the stories organized ?”

I like organizing my stories on the WIKI as that gives you multiple advantages. WIKIs allow you to have a view which lists the entire scope of development by listing the stories, their status, their priorities etc. You can have individual page/placeholder for each story which you can flesh out as they get prioritised for development.You can have another view where you can reference the stories by iteration. Multiple views are useful for all the stakeholders in your project including your client, project manager and the development team.

We have also used Trac , JIRA and Sourceforge for organizing the stories. Based on Thoughtworks’ collective experience in project management, our product division would shortly be launching an integrated project management tool called Mingle , which I expect to start using on our projects.

“How are they kept for future references ?”

As for maintaining documents for future reference, it is better for the existing development team to summarize the whole project as it has been implemented in a seperate document. Future reference requirements usually stem from the need to support or enhance the existing application. So important decisions taken during the project such as key architectural decisions, business rules, workarounds and UI flows can be very useful context to a new person on the project.

Support artifacts also need to be created in the context of the audience and the intent. For example, a working prototype of the application was extremely useful as a training tool for the internal staff instead of a wordy user manual in a previous project I worked on.

“If a feature is changed in the future is the same story accessed again and modified?”

The simple answer is no. You should create a new story for the same. You could mark the original story to say it has been superseded by a new story. Stories are also planning units and there are implications on project velocity* calculations and scope management that need to be considered here.

If your documentation is a living documentation like FIT tests, you will need to change the original documentation for it to stay relevant.

I would be interested in getting more perspectives on the above questions myself. So if you have wisdom to share, drop some comments here.

* – Project Velocity is the throughput of the team in a fixed period of time measurable in a unit of business delivery (Thanks Dan)


Seek value, not quantity

For my Quality Management course at the university, the professor used a restaurant’s business model to explain the concept of value stream mapping. According to him, value for the customer would come from reduction in the overall cycle time – From the time the order is placed to the time the customer gets the food on the table to the time the customer pays the bill and leaves. He argued that it was a win-win situation since the customer’s hunger is satisfied and the restaurant is able to maximise the potential of a limited resource i.e. seating area.

Now although it sounded logical, I did not completely agree with it simply because I have been in the customer’s shoes. The logic above would be true if hunger was my sole motivation. However, I have been to restaurants to have an evening out & spend some quality time with girlfriend/family/friends. I would have actually been annoyed at a rapid fire quick service because it would have given me lesser time to soak in the ambience, speak to people, sip my drinks and have a relaxing time. On an evening out, I would rather have the restaurant focus on catering to my needs rather than focus on getting me out of the door.

So how does the restaurant make money off me given that my longer stay at the table has reduced the total footfall ? There is another metric in the restaurant business called the ARPU – Average revenue per user. To balance the reduced footfall, restaurants maximise the average revenue earned per customer. This is achieved through consultative selling of higher margin products such as wines and desserts.

Now lets tie this back to the software development process. Stories, in the context of lean, often get treated as inventory and cycle time is defined as getting a story from definition to development. There is a focus on reduction of this cycle time because it maximises the number of stories that can be delivered. However, it would be foolish to believe that more stories delivered is the same as more value delivered.

Customers seeking to build custom software applications are often unclear about what they really need. They often go along paths which yield them wrong results. They often break their own model and come back to square one. Ultimately, they learn enough to know what they really need . It is these really valuable stories that finally get delivered. It may take them longer to get there and wasted effort may occur along the way, but the end result is far more satisfying and valuable. The more time and chance the customer has to discover what they need, the more valuable the end product is for them.

Its more important to focus on maximising business value delivered, not more stories.

Do you do emergent planning in your daily lives?

David Seah, a Boston based freelance designer, writes about personal productivity and time management on his blog. His charts and tools are available on his site for downloads.

The interesting bit is that his ideas seems to have a lot in common with the principles of Agile development. Check this post here which resembles release planning in Agile projects. Yet another post resembles iteration planning on our projects.

Anyone tried his techniques before in their daily lives? I am going to give it a shot to see how effective they can be.

Take the agility test

Michael Hugos has come up with an agility test for the IT executive.

Take it. Get your clients and colleagues to take it. Compare notes. Learnt anything new ?

What is the business case for a disruptive business model / technology ?

Imagine being commissioned by Chad Hurley to work on a project for “Online video sharing” or by Steve Jobs to work on a project to deliver “Digital Music Player with a cool interface”. What business case or strategy would they have shared ? There was no existing market for their inventions before they introduced it. They could share with us their vision as shaped by the trends , the changing customer behaviour and the enabling technologies , but they would have been hard to pin down on what they thought the business benefits case might look like. In fact, Youtube did not even have a revenue model till a year after its launch

What does this mean for the well-intentioned analyst who believe their job is to protect the customer’s stakeholder value by coaching them against investing in such projects ?

The truth is that it is not always possible to judge disruptive technologies or business models using traditional tools of analysis. In fact, Clayton Christensen, the Harvard Professor who coined the term “disruptive innovation” advises us of the following

“In highly uncertain situations that typify disruptive innovations, following the typical strategy-making process just doesn’t work. In these uncertain situations, some companies believe they need to fly by the seat of their pants. However, companies can follow a rigorous process by using an emergent strategy supported by a discovery-driven planning process that enables them to adjust their strategy when they encounter unanticipated opportunities, problems and successes.”

As a business analyst the role is more about supporting this emergent strategy development process and not just a resource allocation filter. Admittedly this requires a significant mind-shift. Michael Hugos gives us a very important mantra – Be strategically focussed and tactically agile to meet the needs of a customer in an dynamic business environment. Delivering business value is as much about offering flexibility for the business to change direction as it is about delivering incremental revenues or saving costs.

Pairing and productivity

Just been through a blog reading trail starting from Kathy Sierra’s “The Asymptiotic Twitter Curve” to Jason Fried’s “How to shut up and get to work” to Joel Sposky’s “The value of alone time” which all talks about individual productivity. The gist of those blogs is that we feel compelled to repetitively keep checking our email, IMing, answering phone calls, checking our blog stats etc without realising that it breaks our concentration and prevents us from going into a “zone”.

This led me to think about how Agile addresses the productivity issue. It was interesting to see that Agile considers colacated teams to contribute towards the overall productivty of the team. On the other hand, advocates of agile defend the loss of productivity when 2 people work on the same problem by offering reasons such as continuous code review, design brainstorming, risk mitigation against siloed knowledge etc. Based on the logic in those posts, productivity is actually helped through pairing and adversely affected by colocation.

One advantage of pair programming that my ex-colleague Ivan Moore mentioned to me was that out of politeness and respect for your pair,one would be less tempted to check for their emails or answer any phone calls or browse generally. The pairs keep each other honest and focussed on the task, and can crack on faster. Thus, what is lost in terms of having 2 people working on 1 problem is compensated by the increased and sustained focus on the task.

Agile advocates co-located team sitting in proximity for free flow of communication about project matters . Under such arrangement, a team member can potentially disturb another pair to ask about a problem that they might have tackled earlier. The benefit is that one does not have to solve the problem from scratch if other had done something similar earlier.

However, Joel observes that about 15 minutes of respondent’s time is wasted when they are interrupted in their work as they will now need to spend that time to get their concentration back. Joel’s observation is based on the fact that individual programmers have to juggle a lot of small details in a short memory and one loses that thread when they are interrupted. The process of re-organizing the data takes an investment of 15 minutes.

Now when I have been explaining a concept to someone and disturbed, I too have lost my thread and have had to repeat some of the things I may have already covered. However, if someone was listening keenly, they might be able to get me back on track again quicker without the need to repeat things.

Given this, my theory is that pairing would allow for all small details about the program to be jointly understood by each half of the pair and even if they get disturbed, they can make a faster recovery to get back on track and achieve their focus again. Thus, it might take only 7.5 minutes to get back to the problem for a pair who were interrupted as opposed to 15 minutes for an individual.

Has anyone got any empirical observations to back this up ?

Technical Debt Part 2

Following PierG‘s comments, I had further discussions with my colleagues and realised that he and I may be speaking about slightly different things. Technical debt can be generated by either of these

1. Technical debt generated from having poor safeguards such as bad testing, insufficient code reviews, no refactoring etc is bad (See Thomas Looy’s post )

2. Technical debt generated from having made a conscious decision about achieving short term fix that includes things like having stuff in a configuration file, static html page, manual workarounds etc.

One is a clear compromise on principles of working whereas other is a conscious well thought out short term fix. I suspect PierG had the former in mind when he talks about taking on technical debt as wrong.