Chris Sterling’s Blog

The thought of less documentation is appealing to many in the software industry. Reducing the specificity of our requirements can have tremendous value but some go too far. One of the big myths about Agile software development is that Agile means no more documentation. This was not the purpose of the Agile Manifesto value “Working software over comprehensive documentation”. The ideas was that only the documentation that supports the creation of working software should be created. There are many reasons for software to have documentation:

• Maintainability
• User manuals
• Training
• Communication
• etc…

Since the idea of Agile software development is not to remove all documentation from software delivery then what is the right amount of documentation? Of course, the answer is it depends but we won’t leave it at that. This article will focus on one area of documentation and that is specifying what users want.

User stories were popularized by Mike Cohn with his book “User Stories Applied”. Many people do not read and take in all of what is provided in Mike’s book before using user stories to capture user desired functionality. Instead they may be introduced to it on the Internet, through a seminar, or in training. The appeal of user stories is they are short statements that are easy to write and fit on 1 index card. The problem, as Ron Jeffries explains it in his article “Card, Conversation, Confirmation”, is the statement written on an index card is not enough information for the development to implement without providing some clarity through conversation with someone who represents the user’s desires. So the first way that user stories “GO WILD!” is:

Stakeholders expecting that a statement written on an index card is enough for the team to implement what they desired in the software

And this can be more generalized as:

User stories inappropriately used and becoming just bad specifications

Development must have conversations with people knowledgable about the user story domain to clarify the desired functionality. And as Ron rightly describes, conversations may be supplemented with documentation when needed. This could be in the form of an user interface design, usability study, business rules, acceptance tests, and any other format necessary for the development team to implement the user story satisfactorily. During the conversation new understanding and clarity of the user story specifics will be drawn out. The next way user stories “GO WILD!” is:

Acceptance criteria (or “Confirmation” as Ron puts it) for the user story is not captured and many times forgotten during implementation

It is important that the development team and domain experts write down important nuggets of knowledge about the desired functionality. This could be as simple as writing it down as bullet points on the back of the index card or creating skeletons of acceptance test cases.

Sometimes teams come across user stories that seem either too ambiguous or potentially mammoth in size. There are some great indicators in the user story text that I have found over the years while working with them over the years. Here are a few ways that user stories “GO WILD!” along with ways to deal with them when they do:

Using conjunctions within the user story statement

Whenever I see an “and” or “or” or “if” or “until” or… well you get the picture… this is an indicator to me that more than one piece of functionality is being described. This is an opportunity to split the user story into multiple user stories usually right where the conjunction was put in.

Describing too much about “how” to implement the user story

It has been common in my experience to see business analysts or technical writers creating user stories to support a Scrum Product Owner or stakeholder group in defining their desired features. You might see phrases such as “when they click on … button”, “drop down”, and “using Excel” which are specific about the “how”. In the past, these people were asked to describe the functionality so it was clear for programmers to code and testers to verify. Now we are asking them to write user stories that describe “what” the user wants and not “how” to implement it. This is a problem since they have used the “how” to make clear requirements for the development team to implement. Agile teams are expected to take more responsibility in defining “how” the software is implemented. This includes providing their input on specifying and designing the software. Many times I have seen specification of features get simplified or enhanced through the conversation between development team and domain experts. We want this to occur more often since these interactions will increase software features delivered and improve the product for its users.

Can’t describe the value provided to the user for the desired functionality

There has been more than one time that I have asked “why are we implementing this user story” and was not provided a satisfactory answer. I may have gotten an answer like “because it is defined in the requirements document”, “because it was prioritized high”, or “because the CEO wanted it”. All of these may be good enough reasons to implement the functionality but it is less likely that the development or domain experts will implement it masterfully given the lack of understand around what the user wants to do with this functionality. Understanding how a user story fits into the bigger picture is important since it will enhance the conceptual integrity of the software overall. When software is implemented by generating many disconnected components and integrating them together into a cohesive package the software tends to become inconsistent in its usage with lots of hidden but potentially useful functionality. Ask yourself when writing a user story why would the user want this functionality. You can use the user story template:

As a [user role] I want to [do something] so that [I gain some benefit or value]

By address the “so that” clause you will provide more understanding of how the functionality fits into the larger picture. Or you may find out that it is not useful at all. In that case, you can throw it away and work on something of more value to your users.

Writing technical user stories in which the value is not understood by the Product Owner or stakeholders

Technical user stories tend to get deprioritized since they are not easily assessed for value by most non-technical people. I have found that it is essential to describe why we have new technical needs or what the cost of not addressing a system’s quality attribute such as scalability could be. I have written an earlier post on this regarding a term coined by Mike Cohn called “abuse stories”. The idea is to capture the cost of not addressing an architectural or infrastructure need by describing it from an abuser of the software’s point of view. Writing technical needs in a user story format is not necessary but may be helpful while building trust between the development team and stakeholders. Sometimes project stakeholders believe that time and money is getting wasted on technical perfection. By writing the abuse story a development team can show that these technical needs are not just wasted efforts but provide value to the software. It is hoped that over time a team can just let their Product Owner or stakeholder group that there is a technical need that must be prioritized before particular functionality should be implemented and that will be enough.

Now that you know some ways that your user stories can “GO WILD!” it is time to tame those user stories so that your software can be the best it can be. Happy story writing! And, oh yeah, happy new year 2009!

4 Comments »

George Dinwiddie, a wonderful blogger on Agile software development, wrote on “Working Hard? Or Hardly Working?”. Please read his blog and then you can read my comment below:

Good day George,

I read your blog all the time and enjoy what you write about. This blog entry caught my attention and I had to respond.

The one problem I have with the judgment of “working hard or hardly working” is that is presumes that working hard is always good. A large issue in our industry is the “busy-ness” factor of all people involved in the organization from executives to managers to developers and operations. We all are too busy and therefore our hard work turns into adding more work than if we were to slow down and make better decisions about how to complete the work best.

So, even if someone could tell if another person is “working hard” or “hardly working” I am not sure this is a good measure of anything. In the end I would rather understand if the organization or product team as a whole is getting faster in their delivery of valuable software. If this is the case then all people involved are improving their capabilities for the good of the whole.

Coincidentally, Tom Perry wrote on “Measuring Productivity = Continuous Improvement?” where he makes many references to “monkeys” but also makes a conclusion that teams can create their own productivity metrics to help improve their own delivery capabilities rather than management defining problematic measurements for the team that create poor responses.

Both blog entries have great information and references in them and I hope that those of you interesting in “measuring productivity” take a look. I am still on the Alistair Cockburn side of the house when it comes to measuring productivity. I believe it cannot be done but that doesn’t mean that the team cannot create metrics that help them improve.

And finally I will just point you towards Ron Jeffries wonderful writing on “Productivity” from “Annals of Kate Oneal”.

No Comments »

In a Certified ScrumMaster course this week one of the teams created some haiku on the topic of user stories. Here is what they came up with:

INVEST your time in telling stories that give you meaning moving on…

Unique perspective inward and outward focused simply understood

The user wishes to do something on the site they do not know why

Start with a user a new or improved feature gives us the value

As a Scrum novice I want to learn this method so that we can plan

The confirmation makes conversation simple and settles details

Not bad for 5 minutes worth of work. Some of the items that were referenced:

• INVEST model
• User Story template
• 3 C’s of a User Story

1 Comment »

This is a bit off my usual topics on this blog but I am a heavy open source user and this article is something that I hope gets to more enterprise operations, managers and executives. I have been using and deploying production available applications using open source tools, libraries, and platforms for over 12 years now. Open source tools can do almost anything commercial products are able to do and have transformed the software industry in that time span. The list given in the article contains open source projects that I would recommend and have used in the past either directly or indirectly including *nix tools and libraries shown.

I would like to add to this listing with some of the tools I have come to use often:

• Maven 2.x+ (http://maven.apache.org/)
• JBoss (http://www.jboss.org/)
• Rio/Jini/Apache River (http://incubator.apache.org/river/RIVER/index.html)
• Apache Commons (http://commons.apache.org/)
• Subversion (http://subversion.tigris.org/)
• Apache Web Server (http://httpd.apache.org/)
• Bouncy Castle (http://www.bouncycastle.org/)
• Time and Money (http://timeandmoney.sourceforge.net/)
• Spring Framework (http://www.springframework.org/)
• Hadoop (http://hadoop.apache.org/)
• Ruby on Rails (http://www.rubyonrails.org/)

This is some of the open source that I have and still use on my projects. What are your favorites that were not on the list?

3 Comments »

For multiple years now I have thrown around the name “Executable Design” to describe Test-Driven Development (TDD) and how it is used for design rather than a test-centric tool. The name itself causes problems for those who are initially introduced to the technique. As a coach I was looking for a way to introduce it without stereotyping it as extra tests inhibiting more code getting delivered.

From my readings of multiple books, articles, and blog postings along with my own experiences with TDD the content of what I am about to distill is not new. This post is entirely about explaining the technique in a way that garners interest quickly. There are multiple pieces to “Executable Design” beyond the basic process of:

• Red, Green, Refactor or
• Write Test, Write Code, Refactor

These statements and the technique is the basis for practicing Executable Design but are not sufficient for describing the value and nuance of the practice. Not that I will be able to present it sufficiently in a single blog post but I want to present the basic principles.

While in a meeting with a team recently we were presented with a question I have heard often:

“Why should we use TDD?”

There are many reasons but generic reasoning alone is not sufficient. We discussed the safety net that good code coverage creates. We discussed the reason system tests do not take the place of unit tests. Then we started to touch on design and this is where it got interesting (and usually it does about this time for me). Before I can describe the rest of this discussion I want to present what lead up to this meeting.

A coach that I highly respect seemed a bit preoccupied one day when he wandered into my team’s area. I asked him what was going on and he told me that some of his issues with the current team he was coaching. He wondered why they were not consistently using TDD in their day-to-day development. The team had allowed a card saying “We do TDD” onto their Working Agreement and were not adhering to it.

I happened to know a bit about the background of this project that our company has been working on for over 2 1/2 years. There is a significant legacy codebase developed over many more years with poor design, multiple open source libraries included, and heavy logic built into relational database stored procedures. Also, just recently management on the client’s side had changed significantly and caused quite a shake up in terms of their participation and guidance of release deliverables. Yet the team was supposed to deliver on a date with certain features that were not well defined. This lead me to discuss the following situations that a coach can find their way into:

1. You could come into a team that has limited pressure on features and schedule and has considered the impact of learning a new technique such as Executable Design. Also, they have asked for a coach to help them implement Executable Design effectively. This is a highly successful situation for a coach to enter.
2. You could come into a team that has deadline pressures but has some leeway on features or vise versa and has considered the impact of learning a new technique such as Executable Design within their current release. Also, they have asked for a coach to help them implement Executable Design effectively. This is somewhat successful but pressures of the release rise and fall in this situation and may impact the effectiveness of the coaching.
3. You could come into a team that has deadline pressures and has not considered implementing Executable Design seriously as a team. Also, they have NOT asked for a coach and yet they have gotten one. The coach and the techniques they are attempting to help the team implement may seem like a distraction to the team’s real work of delivering a release. This is usually not successful and please let me know if you are a person who is somewhat successful in this situation because we could hire you.

The current team situation seemed to be more like #3 above and therefore the lack of success in helping the team adopt TDD did not surprise me. Also, I started to play devil’s advocate and provide a list of reasons for this team NOT to do TDD:

• At current velocity the team is just barely going to make their release date with the minimum feature set
• Not enough people on the team know how to do TDD well enough to continue it’s use without the coach
• The architecture of the system is poor since most logic is captured in Java Server Pages (JSP) and stored procedures
• The code base is large and contains only about 5-10% test coverage at this time
• It sometimes takes 10 times longer to do TDD than just add functionality desired by customer

This is not the full list but you get the picture. Don’t get me wrong, the list above begs to me the need for Executable Design but if the team does not have significant experience to implement it effectively it could seem overhead with little benefit to show for it.

After discussing this and more stuff that I won’t go into he told me about a couple of things that he can do to help the team. One of them was to work on minimizing the reasons for not doing Executable Design by discussing them with their ScrumMaster and actioning them on the impediments list. Some of those actions would go to upper management who get together each day and resolve impediments at an organizational level. One of the actions was to get our CTO and myself into a room with the team so they can ask the question “why should we do TDD?”.

Now we are in the room and most of the team members had been exposed to TDD through pairing sessions. Some of them had some ideas about where TDD was useful and why they thought it was not on this project. During the discussion one of the team members brought up a great discussion point:

“One of the problems with our use of TDD is that we are not using it for improving the design. If we just create unit tests to test the way the code is structured right now it will not do any good. In fact, it seems like we are wasting time putting in unit tests and system tests since they are not helping us implement new functionality faster.”

This team member had just said in the first sentence what I instinctually think when approaching a code base. The reason to do TDD is not just to create code coverage but to force design improvement as the code is being written. This is why I call TDD and its best known principles and practices of applying it Executable Design. If you are not improving the design of the application then you are not doing Executable Design. You might be just adding tests.

Some of the principles I have found to help me in applying Executable Design effectively are (and most, if not all, of these are not something I came up with):

• Don’t write implementation code for your application without a failing unit test
• Separate unit tests from system and persistence tests. (as described in this previous blog entry)
• Create interfaces with integration points in a need-driven way (as described in this previous blog entry)
• Always start implementing from the outside in (such as in Behavior-Driven Development and as described in this previous blog entry)
• Mercilessly refactor the code you are working on to an acceptable design (the limits of which are described in this previous blog entry)
• Execute your full “unit test” suite as often as possible (as described in this previous blog entry)
• Use the “campground rules” of working in the code: “Leave the site in better shape than when you arrived”
• Create a working agreement that the whole team is willing to adhere to, not just what the coach or a few think is the “right” agreements to have.

Try these out on your own or with your team and see how they work for you. Modify as necessary and always look for improvements. There are many thought leaders in the Agile community that have written down important principles that may work for you and your team.

And finally, now that I have filled an entire blog post with “Executable Design” what do people think about the name? It has worked for me in the past to explain the basic nature of TDD so I will use it either way unless others have better names that I can steal?

2 Comments »

To start with, I want to be honest about my knowledge of kaizen. I have not been in a workplace where the actual term kaizen was used to demonstrate improvements within our organization. I have researched quite a bit about it and found the book “The Kaizen Revolution” by Michael Regan was the easiest for me to read on the subject. This book also follows a fictional situation that creates examples of using the kaizen philosophy to change a company around. Through my research and discussions with others on the subject I have found that kaizen and the mindset shift is similar to that enabled by the Scrum framework.

Scrum is based on an empirical process control that focuses on inspecting the current state and adapting behavior to improve. This focus on continuous improvement through “inspect and adapt” is supported in the Scrum framework at 3 points in the process.

• The Daily Scrum meeting allows the team to focus on their commitment for the current Sprint and whether they are still on track to deliver on that commitment. If they are not able to meet the commitment then they are asked to adjust the Sprint therefore adapting to the situation.
• The Sprint Review meeting allows customers to view a potentially shippable product increment created by the Team and provide feedback that adjusts the Product Backlog contents and priorities. We are inspecting the product and adapting to a new understanding of the product.
• The Sprint Retrospective enables the Team to improve the process they use to delivery software each Sprint. The Team is expected to inspect their process honestly and thoroughly to figure out how they can adjust for improved delivery capability in the following Sprint.

The “inspect and adapt” focus allows for a Team and product to continually improve over time through seemingly simple mechanisms with sizable effects on the software delivered. An unexpected or additional effect of this focus on “inspect and adapt’ in Scrum is the organizational environment encompassing the team starts to adjust based on needs of the Team to improve their software delivery. Therefore a single team causes “organizational change” through small and incremental adjustments.

One of the main tools that a Scrum Team can use to “inspect and adapt” is the “impediment”. An impediment is:

“Anything that prevents a team member from performing work as efficiently as possible” - from Victor Szalvay’s article “Glossary of Scrum Terms”

In coaching teams I have found that capturing impediments is done casually and is not initially found to be as important as other activities the team is doing. Over time I have found a few simple “rules of thumb” for capturing impediments that have helped Scrum Teams:

• ScrumMasters keeping a physical impediment and action item list. As we teach in the Certified ScrumMaster course it is important to action impediments after the Daily Scrum meeting immediately. Many of our teams at SolutionsIQ setup an area to capture impediments written on post-it notes. Following the Daily Scrum meeting team members who are interested in taking action on the newly captured impediments stay behind to work with the ScrumMaster. An action item is created identifying “What” needs to be done, “Who” is going to make sure it happens, and “When” they will communicate progress back. It usually looks something like this:

Impediments getting actioned

• At least 1 impediment raised every Daily Scrum. It is my opinion that if a Scrum Team is unable to bring up at least 1 impediment at each Daily Scrum then the ScrumMaster is not supporting the team effectively. I want the ScrumMaster to find ways to extract at least 1 impediment during the Daily Scrum. In the past I have asked teams whether they were comfortable speaking about an impediment they are having in the Daily Scrum with me there. I have also pleaded with a team to tell me at least one impediment before we leave. One time the Daily Scrum was before a Sprint Review we were going to conduct in the afternoon. Each team member said they had no impediment and at the end I found out that at least one person was not ready to demo the software just by asking “Why are there no impediments today?”.
• Ask everybody on the team to write down at least 1 impediment. Sometimes a Scrum Team has improved significantly and begin to get into a flow. Although I want to celebrate significant improvements we should not sit on our laurels. The stagnation of “status quo” can catch hold quickly and we must find ways to break through this potential behavioral issue. Just ask all members of the team to write down at least 1 impediment and then work with the team to action those impediments immediately as shown above. One of my colleagues, Brent Barton, once wrote up on our white board the following saying that I have found to be helpful to me for fighting the “status quo”:

“The absence of conflict is not harmony, it’s apathy” - from article by Kathleen M. Eisenhardt, Jean L. Kahwajy, and L.J. Bourgeois III called “How Management Teams Can Have a Good Fight”

My own nature has helped me internalize the “inspect and adapt” mindset with Scrum. I used to think I was lazy but the fact that I would work additional hours to improve builds, mock frameworks, test cases, and other artifacts of projects I worked on seemed to contradict this. The reason I thought I was lazy was my tendancy to find ways to automate just about anything that could be so that I did not have to manually do it anymore. My initial splash into programming actually started this way while working in a data entry job. Each day my work was incredibly repetitive. One day a person who understood the software we were using to do data entry showed me the use of a cool macro that allowed fields to be automatically filled out. I was immediately impressed and asked them to show me how that worked. The language used was a Visual Basic knock off and the scripts I wrote following this allowed me to get 10 times faster in my data entry. I was able to focus on other activities including learning HTML, JavaScript, Pascal, and Java. The moral of this story is jump on opportunities to improve your environment including processes, facilities, and software.

Those additional hours I put in has decreased over time as I have found a much more sustainable pace for myself. I cannot say that I keep a fully sustainable pace even now but at least I am able to identify earlier when I am starting to tank. The “inspect and adapt” mindset should not cause our Scrum Team to adapt themselves into a culture of late hours and intravenous Red Bull drips. A Scrum Team should find ways to adjust within their timebox rather than adjust their timebox. This means that hourly, daily, and iteration timeboxes are important to understand and monitor for breakage. As a ScrumMaster I want the team to keep a constant pace as much as possible and sudden adjustments that cause time management problems should be identified immediately and monitored from there to resolution ASAP.

A continual need to improve my life and environment has become an addiction of mine. How can I better use the time I have with my family? How can I help make my work environment even more fun to work in? What can I do to improve my knowledge in all types of subjects? This addiction has been a valuable behavior for my entire life and I hope that others will find something that has similar effects for themselves. People reading this blog who are currently using Scrum please use the impediment to your advantage and find ways to improve your software delivery. It’ll be worth every minute used to “inspect and adapt” your Sprint, Product Backlog, and processes.

1 Comment »

On November 6th I presented an updated version of the Managing Software Debt talk at Agile Vancouver “Much Ado About Agile” conference. This is a link to the presentation deck:

Managing Software Debt - Agile Vancouver (PDF)

I was honored to present at this local conference and had a great time meeting up with old friends and getting to know some new ones. I hope that I can do this again soon. If you are interested in more information and other presentations at Agile Vancouver you can go to their home page.

No Comments »

Managing Software Debt
Continued Delivery of High Values as Systems Age

Many software developers have to deal with legacy code at some point during their careers. Seemingly simple changes are turned into frustrating endeavors. Code that is hard to read and unnecessarily complex. Test scripts and requirements are lacking, and at the same time are out of sync with the existing system. The build is cryptic, minimally sufficient, and difficult to successfully configure and execute. It is almost impossible to find the proper place to make a requested change without breaking unexpected portions of the application. The people who originally worked on the application are long gone.

How did the software get like this? It is almost certain the people who developed this application did not intend to create such a mess. The following article will explore the multitude of factors involved in the development of software with debt.

What Contributes to Software Debt?

Software debt accumulates when focus remains on immediate completion while neglecting changeability of the system over time. The accumulation of debt does not impact software delivery immediately, and may even create a sense of increased feature delivery. Business’ responds well to the pace of delivered functionality and the illusion of earlier returns on investment. Team members may complain about the quality of delivered functionality while debt is accumulating, but do not force the issue due to enthusiastic acceptance and false expectations they have set with the business. Debt usually shows itself when the team works on stabilizing the software functionality later in the release cycle. Integration, testing, and bug fixing is unpredictable and does not get resolved adequately before the release.

The following sources constitute what I call software debt:

• Technical Debt[1]: those things that you choose not to do now and will impede future development if left undone
• Quality Debt: diminishing ability to verify functional and technical quality of entire system
• Configuration Management Debt: integration and release management become more risky, complex, and error-prone
• Design Debt: cost of adding average sized features is increasing to more than writing from scratch
• Platform Experience Debt: availability and cost of people to work on system features are becoming limited

Software Debt Creeps In

Figure 1.1: A relatively new system with little debt accrued.

Figure 1.1 displays a system that has minimal amount of software debt accrued. A few low priority defects have been logged against the system and the build process may involve some manual configuration. The debt is not enough to significantly prolong implementation of upcoming features.

Business owners expect a sustained pace of feature development and the team attempts to combine both features and bugs into daily activities, which accelerates the accrual of debt. Software debt is accruing faster than it is being removed. This may become apparent with an increase in the number of issues logged against the system.

Figure 1.2: An aging software system slowly incurs significant debt in multiple functional areas.

As a system ages, small increments of software debt are allowed to stay so the team can sustain their velocity of feature delivery. The team may be complaining about insufficient time to fix defects. Figure 1.2 shows a system that has incurred software debt across all functional areas and components.

At this point, delivery slows down noticeably. The team asks for more resources to maintain their delivery momentum, which will increase the costs of delivery without increasing the value delivered. Return on investment (ROI) is affected negatively, and management attempts to minimize this by not adding as many resources as the team asks for, if any.

Even if business owners covered the costs of extra resources, it would only reduce the rate of debt accrual and not the overall software debt in the system. Feature development by the team produced artifacts, code, and tests that complicate software debt removal. The cost of fixing the software debt increases exponentially as the system ages and the code-base grows.

Figure 1.3: The aging system has accrued significant debt in all functional areas and components.

Software debt in the system continues to accrue over time, as shown in figure 1.3. At this point, new feature implementation is affected significantly. Business owners may start to reduce feature development and put the system into “maintenance” mode. These systems usually stay in use until business users complain that the system no longer meets their needs.

Managing Software Debt

There are no magic potions for managing software debt. Software can accrue debt through unforeseen circumstances and shortsighted planning. There are some basic principles that help minimize software debt over the lifespan of the product:

• Maintain one list of work
• Emphasize quality
• Evolve tools and infrastructure continually
• Always improve system design
• Share knowledge across the organization
• And most importantly, hire the right people to work on your software!

Maintain One List of Work

One certain way to increase software debt is to have multiple lists of work. Clear direction is difficult to maintain with separate lists of defects, desired features, and technical infrastructure enhancements. Which list should a team member choose from? If the bug tracker includes high priority bugs, it seems like an obvious choice. However, influential stakeholders want new features so they can show progress to their management and customers. Also, if organizations don’t enhance their technical infrastructure, future software delivery will be affected.

Deployed software considered valuable to its users is a business asset, and modifications to a business asset should be driven from business needs. Bugs, features, and infrastructure desires for software should be prioritized together on one list. Focus on one prioritized list of work will minimize confusion on direction of product and context-switching of team members.

Emphasize Quality

An emphasis on quality is not only the prevention, detection, and fixing of defects. It also includes the ability of software to incorporate change as it ages at all levels. An example is the ability of a Web application to scale. Added traffic to the web site makes performance sluggish, and becomes a high priority feature request. Failed attempts to scale the application result in a realization that the system’s design is insufficient to meet the new request. Inability of the application to adapt to new needs may hinder future plans.

Evolve Tools and Infrastructure Continually

Ignoring the potential for incremental improvements in existing software assets leads to the assets becoming liabilities. Maintenance efforts in most organizations lack budget and necessary attention. The International Organization for Standardization (ISO) standardizes on four basic categories of software maintenance in ISO/IEC 14764[2]:

• Corrective maintenance - Reactive modification of a software product performed after delivery to correct discovered problems
• Adaptive maintenance - Modification of a software product performed after delivery to keep a software product usable in a changed or changing environment
• Perfective maintenance - Modification of a software product after delivery to improve performance or maintainability
• Preventive maintenance - Modification of a software product after delivery to detect and correct latent faults in the software product before they become effective faults

Most maintenance efforts seek to prolong the life of the system rather than increase its maintainability. Maintenance efforts tend to be reactive to end user requests while business evolves and the technology decays.

To prevent this, attention must be given to all four categories of software maintenance. Budgets for software projects frequently ignore costs for adaptive, perfective, and preventive maintenance. Understanding that corrective maintenance is only part of the full maintenance picture can help an organization manage their software assets over it’s lifespan.

Improve System Design Always

Manage visibility of system design issues with the entire team. Create a common etiquette regarding modification of system design attributes. Support the survival of good system design through supportive mentoring, proactive system evolution thinking, and listening to team member ideas. In the end, a whole team being thoughtful of system design issues throughout development will be more effective than an individual driving it top down.

Share Knowledge Across the Organization

On some software systems there is a single person in the organization who owned development for 5 years or more. Some of these developers may find opportunities to join other companies or are getting close to retirement. The amount of risk these organizations bear due to lack of sharing knowledge on these systems is substantial.

Although that situation may be an extreme case of knowledge silos, a more prevalent occurrence in IT organizations is specialization. Many specialized roles have emerged in the software industry for skills such as usability, data management, and configuration management. The people in these roles are referred to as “shared resources” because they use their specialized skills with multiple teams.

Agile software development teams inherit team members with specialized roles, which initially is a hindrance to the team’s self-organization around the work priorities. Teams who adhere to agile software development values and principles begin to share specialized knowledge across the team, which allows teams to be more flexible in developing software based on priorities set by business. Sharing knowledge also reduces the risk of critical work stoppage from unavailable team members who are temporarily on leave.

Hire the Right People!

It is important to have the team involved in the hiring process for potential team members. Teams will provide the most relevant skills they are looking for, thus, allowing them to review and edit the job description is essential. Traditional interview sessions that smother candidates with difficult questions are insufficient in determining if the candidate will be a great fit. Augmenting the interview questions with a process for working with the candidate during a 1 to 2 hour session involving multiple team members in a real-world situation adds significant value to the interview process. Before hiring a candidate, teams members should be unanimous in the decision. This will increase the rate of success for incorporation of a new team member since the team is accepting of their addition.

Another significant hiring focus for organizations and teams is placing more emphasis on soft skills than technical expertise. I am not advocating ignoring technical experience. However, it is critical in an agile software development organization or team to have people who can collaborate and communicate effectively. Soft skills are more difficult to learn than most technical skills. Look for people who have alignment with the hiring team’s values and culture.

In Summary

As systems age they can become more difficult to work with. Software assets become liabilities when software debt creeps into systems through technical debt, quality debt, configuration management debt, design debt, and platform experience debt.

Applying the six principles in this article will lead to small changes that over time will add up to significant positive change for teams and organizations. The goal of managing software debt is to optimize the value of software assets for our business and increase the satisfaction of our customers in the resulting software they use.

References

1. Ward Cunningham - “Technical Debt” - http://c2.com/cgi/wiki?TechnicalDebt
2. “ISO/IEC 14764: Software Engineering — Software Life Cycle Processes — Maintenance” - International Organization for Standardization (ISO), revised 2006

7 Comments »

In the past I have blogged on software debt in terms of:

• Going beyond code to manage software debt in “Clean Up, Clean Up, Everybody Do Your Share”
• Problem with catching up on software debt in “The IT Manager’s Dilemma with Software Debt”

Although there are reasons defined or implied in those blog entries I think it is important to discuss it more directly. The following image will be the backdrop for the rest of this blog entry.

Software that has been around for some time usually gets difficult to work with. The value of the software depreciates as more debt creeps into the software because the costs have not went down and probably went up to keep up with maintenance. So the costs are rising to maintain the system and the value of the software is depreciating. A business asset that has these kind of characteristics is in jeopardy of getting scrapped. Sometimes the only thing stopping us from scrapping the software is that we have nothing else to replace it and the business processes it currently supports are dependent on it.

Depreciation of software is inevitable but when we manage software debt diligently with a disciplined approach we can significantly prolong the depreciation from going below the minimum acceptable value for the software. In fact, if we manage software debt in the process of adding business functionality we will increase the software’s value and therefore further prolonging significant depreciation.

I will be at Agile Vancouver speaking on “Managing Software Debt” on November 6th. This presentation has been given in multiple variations over the past year and a half. Although I won’t go into detail here, the following 6 principles are what I see as essential to managing software debt effectively:

• Maintain one list of work
• Emphasize quality
• Evolve tools and infrastructure continually
• Improve system design always
• Share knowledge across the organization
• And most importantly, hire the right people to work on your software!

I will post on more details of managing software debt in the near future.

No Comments »

Refactoring is an essential practice for teams developing solid software and continually evolving the design to meet new customer needs. From the home page managed by Martin Fowler, who wrote the original book on refactoring, it says:

“Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.”

On a project that is well tended in terms of its design and structure the act of refactoring can be elegant and liberating. It will allow teams to continually inspect and adapt their understanding of the code and the customer’s need as they gather feedback on the software.

On a legacy system the act of refactoring can seem overwhelming. Although the refactoring craft revolves around the mindset that continually making small incremental improvements will lead to overall improvement in the design figuring out where to start and stop in a legacy system is unclear. This blog entry is for teams who are unclear on where to begin refactoring and how much refactoring is sufficient day to day.

Where to Start Refactoring?

On all teams that I have worked with either as a team member or coach this question starts with the answer:

When the feature you are working on exposes the need to refactor

The problem is that this does not give a full answer with context and understanding for the particular project we are discussing. I like to answer the following questions before starting to refactor:

• Does this change directly affect the feature I am working on?
• Would the change add clarity for the feature implementation?
• Will the change provide automated tests where there currently are none?
• At first glance does the refactoring look like a large endeavor involving significant portions of the system components?

For the first 3 questions if the answer is “yes” then I am leaning towards starting a refactoring of the code. The only caveat is the answer to the last question. If the answer to this question is “yes” then I will use my experience with the system help me generate a gut feeling about the size of this refactoring compared to our initial cost estimate of the feature implementation. If the size of the refactoring is significantly larger than the cost estimate given to the Product Owner / Customer then I may decide to bring the refactoring up to the team for discussion. As we teach in the Certified ScrumMaster course:

The Product Backlog estimate is a mutually agreed budget. If the team is going to exceed the budget, it needs to escalate the decision. Otherwise, the team introduces scope creep.

When to Stop Refactoring?

Using the word “stop” is misleading since it means we just decide to quit at some point in time without regard for the situation. I would not suggest this at all. Instead the following conditions are what I use to figure out when I have finished a refactoring:

• Am I refactoring code structures that are not directly affected by the current feature?
• Are there any more code structures that are directly affected by the current feature I am working on that have not been refactoring sufficiently?
• If the refactoring is blowing our mutually agreed upon budget for this feature with the Customer then I should bring this up to the team to discuss further progress
• If we decide as a team that the refactoring can be absorbed into the current iteration without affecting delivery on our original iteration commitments then continue to refactor
• If the refactoring cannot be absorbed into the current iteration without affecting delivery on our original iteration commitments then we must bring it up to the Customer to discuss how to proceed

If the situation leads us to bringing up the refactoring to our Customer then the options for are to:

• Simplify the original feature - the team could bring a suggested simplification to the customer
• Remove the feature from the current iteration commitments
• Remove one of the other features from the current iteration commitments

Wrap Up

Starting a refactoring should be identified in the course of implementing a piece of business value prioritized by the Customer. We should gauge the size of the refactoring against the cost estimate given to the Customer. Wrapping up a refactoring should not be done without regard for the quality of our implementation going forward. The result of a refactoring should be improved code structure while preserving existing functional behavior. Use the questions, conditions, and options to manage your refactorings along with delivery of business value to your Customers.

1 Comment »