Feedback Loops

It's only fair to share...Share on Google+0Share on Facebook0Tweet about this on TwitterShare on LinkedIn0Email this to someone

In a recent podcast Kent Beck was interviewed by Adam Wathon. The interviewer mentioned something that I was toying around in my head as well for quite a while now, namely, the importance of feedback loops.

It seems to me that there is a recurrent theme in the past years, in such a way that most of the new/hyped/recommended techniques and processes in software development have something in common:

Get more quality feedback in shorter feedback loops.

Business Value

In the clean code community, we have a basic value system against which new techniques and principles can be evaluated, and I am wondering if feedback loops shouldn’t just be one of these values. It doesn’t seem like one of the existing values implies the need for feedback loops, and they are no principle or technique themselves, so I suggest to include them as a core value to measure other techniques against.

How can we justify feedback and feedback loops as a core value? It’s actually pretty straightforward: Every business decision we make is based on the facts known to us. More feedback means we get more of these facts to base our decisions on, higher quality feedback translates to better decisions and shorter feedback loops mean that these facts are more timely, hence relevant. So, from a business perspective it makes perfect sense to rely on this core value (not exclusively of course) in order to make the best possible decisions.

Current state of Feedback Loops

In our daily coding routines, the trend to get more feedback sooner has been going strong for a really long time already. Let’s recap how far we have already taken this for a few examples:

Shorter compilation feedback loop

  • We moved away from slow monolithic compiles, where the whole bunch of source files was processed into a single binary file every time we changed one of the files towards a dynamic module system (DLLs). Feedback was shortened, because the generation of a DLL was much faster and your compiler errors showed up sooner that way.
  • We moved away from aggregating multiple source files into a module towards compiling singular source files, for example, translating a Java source file into its .class bytecode form. Again, compiler feedback was coming back to us faster than before.
  • We added incremental compilation into our IDEs in order to get compiler feedback even faster. Today, when we type code it is already compiled and the IDE shows us the errors of our ways immediately.

Test results feedback loop

  • We started testing our software by giving it to a QA department and having someone sit in front of the software, clicking on things.
  • Soon, we realized that automation speeds up this feedback tremendously and we had automated integration tests.
  • We added unit tests, because integrating and running slow and involved integration tests just wasn’t fast enough.
  • We started to run unit tests in parallel to get the results faster still.
  • Today, you can run your test suite on all your cores whenever you save a change to a source file and get almost instantaneous feedback.

Project state feedback loop

  • We used to write long and involved documents, including requirements, specification, architecture and design documents, and it took years for a project to finish.
  • Process standards showed up, which streamlined the more meaningful documents and gave us feedback on where we are within the project’s lifetime. Still, projects took a year or longer to finish.
  • Agile development started to value working software higher than comprehensive documentation. Again, we got feedback from the working software much faster than from the bunch of documents before.
  • Today, we refined these techniques (burndown charts, kanban, etc.) to get feedback more accurate than ever on the state of our project instantly.

Deployment feedback loop

  • Back in the days, deployment could easily take a week or longer. Only after that time did we get feedback on whether it worked at all.
  • Custom buildsystems, ant, maven, etc. came up to give us a way to create our deployment artifacts fast and with much better stability.
  • Continuous Integration made the feedback available even faster and deployment artifacts were always available.
  • Today, continuous deployment pushes code changes into production immensely fast, thus providing very quick feedback from the fields.

Operational feedback loop

  • Early on, we simply wrote text into files for debugging purposes. It took a long time for these files to get back from customers to the developers and their content was often rather chaotic.
  • We streamlined the logging with various libraries and software became configurable with regard to the level of granularity. We got more feedback that way and in a more uniform style.
  • With distributed systems we started to aggregate even more logs into a unified sink to be able to get feedback from all systems quickly and in one place.
  • Monitoring of production systems became possible in real time. Up-to-date feedback on memory and CPU usage of production machines became available.
  • Today, with see advanced tools like Takipi, which perform failure analysis and provide debugging capabilities and statistical evaluations right within the production systems.

As we can see, the duration of feedback loops has been steadily and significantly reduced throughout the past decades. It is also important to note that this does not only apply to software development, but transcends the whole process. From deployment, over operations, even up to market research and business analysis, the common trend that is continuing ever on is reducing the duration of the feedback loop.

Consequences

Given that we accept the importance of feedback loops in this way as a core value, what are the consequences? Essentially, this means that every decision we make can be evaluated with regards to its impact on the feedback system. Does it provide us with more feedback? Does it make feedback available faster? Does it give us better feedback?

As mentioned above, it clearly makes no sense to rely solely on the impact on the feedback system when making decisions. Nevertheless, I find that a lot of times we make decisions without consciously considering how they affect the feedback we will get.

We think about how we can improve quality, time-to-market, or speed of development, but how often do you actively think about how you could improve your feedback system? Realizing the benefits we reaped from that within only the past few years, I believe it’s about time we treat our feedback systems as first-class citizens of software engineering.

What next?

Often times it looks like we have already reached a zenith. After all, there is only so much point in reducing the time until you get your feedback. It doesn’t matter any longer if you get feedback within one second or half a second. In many of the loops mentioned above, we went down from getting feedback within hours to within seconds already. Is improving on that even possible?

Yes. Definitely. Getting back to the initial formulation of this core value, we realize that there are multiple dimensions to this: the time until we get feedback, the amount and the quality of feedback.

The above examples mostly described impacts on the feedback loop duration. For each of these points, we can still improve in the other two dimensions. For example, it doesn’t get much quicker than running your relevant unit tests as soon as you change your code, but we can get more and better feedback. More feedback might include not just running your unit tests, but also your static analysis tools: FindBugs,PMD,etc information updated on each character as you type. Better feedback quality may include highlighting code that has no test coverage – while you type.

Would it really make such a difference? I do believe so. We have all realized the value of unit tests and static analysis, but the feedback loop is still taking very long. Most projects I know of do run static analysis, but the results just go onto a Jenkins (or some other CI tool’s) dashboard. They sit there and wait for someone to look at them. One may report right back to the committer of course, but that would still take a good while. It requires that the developer first finishes his whole job to be able to commit/push to the CI system, and then it usually takes a whole test suite run and static analysis of most of the source code base, before the feedback is available. Having this information available in an instant is almost as good as having a senior developer sitting next to you, immediately giving you feedback on what you are just about to do wrong. Without actually having someone else sitting next to you, we could still get to the point of giving you the feedback just after you did it wrong. Before you went on to write the next line of code, before you continued on another file, before you ran your test suite, before you committed your chances, before you pushed them to the CI server and well before you come back from your coffee break.

[The feature image is CC-BY 2.0 by Noah Sussman via flickr]

It's only fair to share...Share on Google+0Share on Facebook0Tweet about this on TwitterShare on LinkedIn0Email this to someone