When we decided to open source Watson – our CLI time tracker – we had really positive feedback from the community via GitHub, Twitter, and people using it around us. From the Day One of this project, we had in mind that Watson would not stay single for too long: we called it Watson because it had to be paired with Crick (the storage back-end). So we came up with the idea that Crick could be a standalone, affordable service focused on IT teams activity.
Instead of going headlong to the wrapping of this service (we had almost everything ready to be shipped), William insisted to validate the idea with the Running Lean approach. To be honest, I was so convinced that this service would become popular that I was reluctant to loose time benchmarking the idea-of-the-year™ with a so-called scientific method that only takes on savvy marketing. But, I’m curious by nature. So I played.
It could be fun after all. And, it will only prove that I am right: Crick.io will be our first standalone commercial product. – Julien, Feb. 2016
- developers need to stay highly focused on their work, being disturbed by a co-worker is a real pain,
- managers need an objective report on the time spent to achieve a particular task, and,
- developers need to increase their productivity/meeting ratio.
We decided to focus on two customer segments: developers and project managers, and our early adopters were supposed to be part of the Watson’s community.
We wrote two scripts for problem interviews, one for each customer segment.
After having sent a bunch of emails to people around us, we had several days of problem interviews with developers and managers of local IT companies. We were glad that most of the time, people were quite willing to have a quick chat with us (I don’t have to tell you that Auvergne is the best place to live in right? You already know that).
For each interview, both of us (William and myself) were present, and we alternatively took the role of interviewer and external reporter. Being two is a crucial point to be fully objective in the post-interview debriefing and the notation phase.
Interview after interview, I was more and more disappointed: key problems were not first-world problems. Whatever the team sizes, developers or managers already had a custom or not perfect but sufficient solution to solve them.
After conducting problem interviews, you should transform qualitative data into (mostly objective) quantitative data (scores). If your interview reaches a threshold score, you know that the problem you address sparks interest. We decided to follow the lean analytics recommendations.
William and I scored each interview independently, and we compared our results. Even if our scores did not converge all the time, none of them reached the required threshold.
So with scores came the verdict: we did not achieve our goal to pursue this path. We had to face the truth. I was terribly wrong and blinded.
In parallel, we designed a landing page for Crick and announced the upcoming launch of the service. Like many landing pages, there was a pre-registration form. Numbers proved that only too few people were interested in this service. What a bummer.
We spent 10 days working on our idea and interviewing people around us. Somehow, it’s quite a lot of time, but applying the Running Lean method saved so much more time and energy we were about to spend in a predictable failure. In our case, this method worked. Day 1 to day 10 period was necessary to progressively accept that this idea was not sustainable. The landing page experiment also proved this fact.
We believe that our main problem relied on the fact that we wanted to tackle illusive problems that came from our brains (and not real people); all of this because we already had a solution that we wanted to make valuable. That is probably something you read/hear everywhere, but it is more difficult to notice than one may think.
This lean experiment has been really exciting and even if the result is not what we expected, we learned a lot in driving it. One should not consider this as a failure but rather as positive experience to get back on track.
Last but not least, you might ask yourself: “Why did they publicly share this story?” Answer is: we try to embrace “discomfort” because, believe us, it is complicated to let the world know about TailorDev internals when things did not turn out the way we wanted them to. Yet, we do not consider ourselves delusional people and we trust in transparency and honesty.
If you want to react to this post or share your lean experiments with us, you can ping us on Twitter!