What Build-Measure-Learn Is Not

Any framework or structured way of thinking is an opportunity for some structured abuse. Business frameworks like Build-Measure-Learn, which is at core of the Lean Startup Movement, are no exception. Just to be perfectly clear, I like Build-Measure-Learn, and I think it’s a valuable tool for startups and mature companies to generate more value and to find a viable business model.

I am using Build-Measure-Learn as a vehicle here not because it’s easy to pick on, but because it’s easy to understand and navigate. It offers a convenient platform to illustrate how these abuses can creep in and hijack the business objectives the framework was designed to help achieve.

Build-Measure-Learn Gone Awry

Build-Measure-Learn cycle was introduced by Eric Ries. It’s a straightforward and elegant concept illustrated below.

Startup-feedback-loop1

This state machine helps startups and mature companies execute in such a manner as to minimize resources and maximize the value of what is being created. The benefits of this virtuous cycle have been extensively analyzed.

I’d like to think of Build-Measure-Learn as a very special case of a general business process. It’s a particularly good special case because I believe it’s driven by the right business metrics. One, however, must never confuse a tool with a solution.

Remember, guns don’t kill people, people kill people, so it’s all about how Build-Measure-Learn is applied in practice. Without a carefully selected set of business metrics, i.e. what we measure, and without an analysis of the things that a business has to care about, i.e. what we learn, we have a very little chance of building a real business.

Any tool in the wrong hands can end up a powerful weapon of destruction.

On Disproportionately Powerful Engineering

Having both an engineering and a marketing background, I often find myself looking into the muzzle of two guns. This experience gives me a special, first-hand appreciation of the power balancing act which keeps the organization running smoothly.

To say that marketing and engineering is at odds at most organizations is a great understatement. In my case, these recurring conflicts are at the core of how business is run and where it runs into trouble. That is not to say that engineering is somehow inherently evil and will try to wrestle power away no matter what.

I believe Build-Measure-Learn machine is ought to be turned by marketing and business side of a company (and I’ll get to that point below). The problem is that a strong engineering team tends have its own vision of how Build-Measure-Learn out to be applied.

When paired with an understaffed and overworked marketing organization, these engineering visions can be extremely harmful. If the engineering actually manages to marginalize marketing and other business-focused functions, there most certainly trouble ahead for the entire organization.

How to Combat the Hijackers

Before combating the hijackers one needs to be able to identify the threat. Perhaps combating is not the best analogy to use here because it assumes some sort of armed conflict. I am not advocating that business guys take up arms in their fight against engineering.

The point is that weakened and overextended (there I go again with military analogies) business folks tend to buy into these grand engineering vision of goals and what the progress toward these goals is. As far as I am concerned, that’s a real danger that looms large at any high tech organization with a strong engineering DNA.

Here is how to spot the balance shifting:

Architecture

In most cases architecture hawks cannot be faulted for wanting to engineering the software right, but they do tend to get a little out of hand. I’ve always been skeptical about the value of iterating over an architecture because it seems an entirely engineering-focused activity with no obvious business side-effect.

Investing into a high-performance, scalable, and maintainable architecture is paramount, assuming your product actually needs one. While there are plenty of cases when these investments pay off, there are also tons of cases where engineering drives over-investment in architectures.

This can come in many guises too. Expressing a requirement to standardize naming conventions different products and APIs, and expressing a requirement to reuse every bit of code that we’ve ever written before (even when reuse is costlier and more bloated than a complete redesign) are both a sure sign.

Iterating on the architecture may seem deceptively like Build-Measure-Learn as long as your customer is your own engineers. In most cases, it’s not.

Bug-fixing

Have you heard the refrain that there are still hundreds of bugs in our software? It’s usually followed by an argument about investing into bug fixing to make sure that the product is really solid. On the surface, this discussion really focuses on building better software, measuring bug density, learning how and which bugs are recurring and building processes around the releases.

On the surface, bug-fixing looks like a perfectly reasonable process. The obvious issue here is that if this process is disconnected from customer needs and requirements, it fails to advance the business interests of the company.

Do the bugs affect our bottom line? Do they drive customers away? Are they in any way detrimental to those outside of engineering? When these questions are glossed over, you know something is off.

Re-factoring

Re-factoring is really my pet peeve with engineering. Don’t get me wrong, when re-factoring achieves a business objective, it’s the best thing since sliced bread. For instance, refactoring embedded code to reduce code size of system libraries, thus freeing up more space to the application in a highly resource-constrained embedded system is a good way to attract customers.

Re-factoring for re-factoring’s sake is an aberration. If you engineering organization constantly argues for the need to re-factor code in the name of bettering the architecture or some other abstract notion that doesn’t have an immediate business benefit, the alarms should be going off right away.

Yes, we are building better software modules, yes we can measure them being better since the functions are getting smaller and easier to debug, but what are we really learning in the business context? Is this in any way a substitute for finding our market or building a better product, or are we just iterating in place and getting a product that is progressively better as far as engineering is concerned?

Early detection is key

All these are markers, much like genetics markers that highlight an increased risk for a particular condition. The chances that something is afoot just go up when you start hearing a lot about architecture, integration, harmonization, optimization, refactoring, streamlining and the like. These things are generally benign, but a certain critical mass of buzzwords will most certainly belie a bigger issue.

Arguably, the lines can be blurry. Some purely engineering activities have lasting business benefit and most would help the startup transition between building, measuring, and learning. The ultimate question to startup survival is what percentage of engineering activities is helping the business case, and what percentage has not tangible business side effects.

Put 2 and 2 together

If you find yourself a victim of hijacking (err, disproportionately strong engineering leadership), chances are your company is no longer optimizing for business value creation, and the Build-Measure-Learn cycle has been taken for a ride. At a startup, this most certainly would spell disaster. At a large company, the smoke and mirrors can help everyone escape unscathed, but that’s a pretty pour outcome if all you wanted was to get better at running your business.

I welcome your thoughts and comments.

Advertisements

3 responses to “What Build-Measure-Learn Is Not

  1. I +1’d the main site.
    I recommend setting up +1 button nearby upper-left corner of it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s