How much time should we spend on Automation and Refactoring?
Recently I was asked "How much
effort should we really put into automated tests and refactoring?"
For some context here is our testing
and refactoring process. Note for every new feature we build Automated Acceptance Tests, Unit Tests, and Refactor afterward.
On top of that we value automation so
much we incorporate an evaluation for automation in all ticket or bug tasks.
So why do we spend so much time automating and refactoring?
Here is a great chart on a 5 year
time savings of automating the task. (Source: https://xkcd.com/1205/
)
Using this chart we can extrapolate
time saving that unit testing, automated acceptance tests, and refactoring.
If we take this quote as a basis for
the argument:
“Indeed, the ratio of time spent
reading versus writing is well over 10 to 1. We are constantly reading old code
as part of the effort to write new code. ...[Therefore,] making it easy to read
makes it easier to write.”
― Robert C. Martin, Clean Code: A
Handbook of Agile Software Craftsmanship
Unit Testing
For example unit testing, we spend a
portion of all of our work creating unit tests. We have found that we can get a
very low bug rate and drive down the cost of development through this
practice.
The following is a short article on
the benefits of unit testing: https://dzone.com/articles/top-8-benefits-of-unit-testing
Sections 5,6, and 8 directly relate
to the time spent reading code. If our unit tests save us only 6 hours a
week, according to the above graph over a 5 year period we would spend 2 months
(per person) less in debugging time as long as we have good code coverage. In
previous environments I had seen bugs eat up many hours a day, in those
environments we could have seen over 5 months or more time savings just by unit
testing. Since unit tests provide documentation, expedite the debugging process
and increases quality we should be focusing on making this a strong skill.
Automated Acceptance Tests
The following is an article on the
top 10 benefits of automated acceptance testing: https://saucelabs.com/blog/top-10-benefits-of-automated-testing
Companies that need to do manual
integration tests have to have people dedicated to this task. This means we
save entire headcount in this area by automating integration level
testing. These tests also help developers quickly gain context as well. These
tests take more time to maintain so we cannot lose the unit tests by using only
automated acceptance tests.
Herd Immunity and Testing
One last part of this is my analogy
to herd immunity. Bugs can still creep into the code with the above practices
however when enough of the code is tested in this way we will isolate the side
effects of a bug to a recent change rather than many changes over time. If most
of the code is tested then the bugs that do come up are easy to solve, however
the fewer tests the less efficient we will be at debugging.
So the amount of time we invest in
writing tests and refactoring has an enormous payoff in the long term.
Especially if you take into account the lifetime of projects that had lasted 10
years or more.
Technical Debt
If any of these numbers seem small in
any way we couple that with technical debt and things become much worse. We
spend time refactoring because we want to keep the cost of change down. Here is
a chart on how technical debt can get out of hand:
Design Stamina Hypothesis
Finally there is an argument against
this which is the time to market, this is where the "design payoff
line" comes into play. In this blog: (https://martinfowler.com/bliki/DesignStaminaHypothesis.html)
Martin Fowler states that the design payoff line is hit within “weeks not
months”. (let alone years)
Based on the evidence I have and the
axioms I hold to be true, the level of testing we ask of our developers is the
right one. I hope this information is helpful.
What about the cost of the tests?
A followup asked was a question about the increase in cost in maintaining the tests. As for the tests being more code to maintain, I think you have to consider the refactor step in "Red, Green, Refactor". The team is asked to always refactor after the tests go green. This means removing duplication of code and creating single points in the application of where to modify it when the task is completed. Ultimately this means less, and easier to read code. We can only refactor safely if there are unit tests around all of the code. Those are the ideas presented in the Clean Code and Refactoring to patterns books. Ideally you will end up with less code when you write many unit tests. This means that the team needs to be really good at refactoring and unit testing.
Feedback and questions always wanted.
Please leave comments!
0 comments:
Post a Comment