(Originally published by Lv Yi on May 7, 2020)
I often get this question – what do you measure while adopting Agile?
The conversation usually goes like this:
[me] What is it about adopting Agile?
[him] … doing Scrum.
[me] What effect do you expect after adopting Agile?
[him] higher efficiency.
[me] How would you define higher efficiency?
[him] … finishing more tasks.
[me] Why would adopting Agile lead to higher efficiency?
[him] Agile means higher efficiency, doesn’t it?!
The thinking is fuzzy, to say the least. I suggest to focus on reasoning before measuring.
Let’s take Specification by Example (SbE) as an example of a concrete agile practice. What do you measure while adopting SbE?
Firstly, we learn about SbE by asking 1) what is SbE, and 2) why do SbE ?
A good source of learning is of course the book “Specification by Example”.
1) What is SbE?
“Specification by Example is a set of process patterns that facilitate change in software products to ensure the right product is delivered effectively.”
Suppose that among those process patterns, we choose to focus on the below two at the beginning of our initial adoption. We dive deeper into them.
Specifying collaboratively
“Instead of relying on a single person to get the specifications right in isolation, successful delivery teams collaborate with the business users to specify the solution.”
Illustrating using examples
“The team works with the business users to identify key examples that describe the expected functionality. During this process, developers and testers often suggest additional examples that illustrate edge cases or address areas of the system that are particularly problematic.”
2) Why do SbE?
There are various benefits from the whole set of SbE process patterns. Let’s focus on the benefits from “Specifying collaboratively” and “Illustrating using examples”. In other words, let’s understand the purposes of these two specific practices.
“Specification by Example helps to improve the quality of software products, significantly reduces rework, and enables teams to better align analysis, development, and testing activities.”
There are three areas of benefits.
Higher product quality
“Specification by Example improves collaboration between delivery team members, facilitates better engagement with business users, and provides clear objective targets for delivery - leading to big improvement in product quality.”
Less rework
“Specification by Example helps teams establish a collaborative specification process that lowers problems in the middle of an iteration. Most teams have significantly reduced or completely eliminated rework that occurred as a result of misunderstood requirements or neglected customer expectations.”
Better work alignment
“Specification by Example enables teams to clearly define a target that’s universally understood and objectively measured. As a result, many teams find that their analysis, development, and testing activities became better aligned.”
The above reasoning could be summarized as:
Only after getting this clarity we can go to the measuring part.
“Practice has purpose, and we measure both the practice and the purpose.” I learned this from David Hussman in a private workshop for Odd-e in 2013 – one of the most memorable experiences with “the dude”.
We may come up with the following measures.
“The Book of Why” introduces causal diagrams as a tool to move from correlation to causation. Causal diagrams are similar to causal loop diagrams in systems thinking – they are both based on causation.
After some understanding of the practice and the purpose, we can reason further with finer-grained causation chain by identifying intermediate steps. Why does SbE lead to higher product quality, less rework and better work alignment?
The below causal diagram illustrates our reasoning.
Then, we can measure the intermediate steps, so as to learn the whole chain of causation via validation or invalidation. Note that some variables are harder to measure than others. For example, “amount of rework” is harder to measure than “internal defects”. We don’t have to measure all the variables. With a few intermediate ones, we build our confidence in this causation.
We may end up with measuring:
We need to reason about causes and effects, before getting into measures. On the other hand, measuring can also improve our reasoning in two senses: 1) thinking about how to measure increases the rigor in the reasoning, and 2) measurement results validate or invalidate the reasoning.
Notice that measuring here is for learning and improving. See my previous blog “Two Guidelines for Metrics” for further reference.
RSS feed