Have you ever had this cereal before? It’s delicious. When I make it, I usually prepare two servings, using hot water. It’s got the right amount of everything—raisins for a light sweetness, nuts for protein and fat to get you to lunch time without feeling your stomach trying to eat itself, and grains for a great taste and energy. There used to be a cute story on the packaging of how “Bob” (the Bob of “Bob’s Red Mill”) discovered muesli while traveling with his wife in Europe. While he had never tasted it prior to that trip, it was very common in that country where he was served breakfast that morning. Whoever invented cereal this knew what they were doing… or maybe they didn’t, and it took just the right amount of trial-and-error to get a nice mixture.
There’s another story where cereal plays a prominent role—in Goldilocks and the Three Bears, the main character intrudes upon a home and proceeds to act as though she owns the place. She even eats the owners’ porridge, and the story is famous for her own trial-and-error in picking a suitable meal: one bowl was a little too hot, one was a little too cold, and the third one was just right.
Goldilocks was pretty smart in her actions, if you think about it. They embody what has become dogma in startups and well-run large companies alike: fail fast. The idea is pretty sound. Goldilocks didn’t commit to eating an entire bowl of porridge, not knowing in advance that any particular bowl was the one she’d desire. Instead, she sampled until she found the one that suited her needs.
Did you ever work on a project where leadership committed to the whole bowl of porridge before even taking a bite? Did you ever work on one where leadership couldn’t seem to commit to even one spoonful before switching bowls? I’ve worked in both types.
I remember one company I worked in. There were significant monetary incentives tied to releasing the product on-time, and so everyone was all-in. It wasn’t just the management that wanted to see the product ship on-time, each engineer also had a stake in it. The company had adopted a modified Scrum process. The product had a yearly release cycle with incremental updates in the mid-term for follow-on releases for things that couldn’t make it in the earlier planned releases. That is, plans were laid out an entire year in advance, and a product release date was set.
As it turns out, the release cycle that year was such that the features required a significant change in the product’s software architecture. In fact, there were four significant changes planned for the release, and each one was a departure from the way the prior releases had accomplished the same things. The same features on the prior releases had their limitations, and the planned changes would address those limitations while operating at parity with the prior release.
I said that the company had learned to adopt a type of Scrum, but that’s really only partially true. We had jokingly called the process “Scrum-fall,” poking fun that we had all of the trappings of a Scrum process (sprints, estimated work to fit in sprints, retrospectives for discussing what went well and poorly in the sprints, planning sessions for the sprints, backlog grooming, etc.), but not the primary benefit of adjusting plans with the latest information. That is, the entire effort was planned, beginning to end, and teams were expected to synchronize as to when they had needs from each other.
A process built on good intentions and trust only goes so far, though. In the weekly meetings with the VP of the business unit, each team lead would report their status. If any team reported at all that they were off-track, this resulted in stern words from the top leaders, and this eventually caused engineers and leads to leave the company. I can only speculate that there were disciplinary measures threatened as well, but I’m not certain.
In line with the theme that each change in a software development system has knock-on effects elsewhere, there were two observable changes (if one was keen to observe):
- All transparency was lost into the majority of most of the project efforts, and no significant slips were reported again.
- The project grew later by the day, until about 3 months until the intended release date, it was obvious that the date was going to be missed by at least a year.
At this point, I want to pause one second to say: as leadership in an org, if you do not have visibility into your projects’ progress, you have at least one disaster waiting to happen. That might be obvious. But the trick to this is that you’ve got to collect this information in a way that gives dignity to your developers in the process. If you don’t, just wait for top talent to leave amid accusations of “micro-managing” flying.
Back on topic, on my team, there were two people who prided themselves in their ability to speak “Agile.” They knew the vocabulary, and how Scrum would prescribe things to be done. However, they’d often miss the forest through the trees, insisting on inconsequential things like phrasing work descriptions in the ticket system in terms of user stories (I think this actually has its place—for determining what are all the constituent parts for a given feature to work as intended. I.E., it is an aid in defining requirements, but this intention was lost on these two developers). I won’t forget something that the one developer repeated several times, each time the schedule impossibility was discussed:
It all comes down to the fact that we aren’t following Scrum. Scrum is antithetical to the idea of a firm date with fixed functionality.
This may be true, but it’s also not very helpful. It doesn’t make distinctions between what’s possible and what’s not possible. It’s a complaint without a prescription. It’s a really easy thing to say.
When reality comes calling and you realize that your original plans aren’t working out as intended, you can:
- Do nothing, expecting things to work out if people just work harder.
- Find the pain points and determine if principled fixes can be applied to make up the difference.
- Adjust the due date to the right.
- Adjust the definition of “done” for the due date.
The first is the definition of insanity and is a recipe for turnover if you do it often enough. The second should be an ongoing activity in any event, and it’s often unclear that the potential gains for the expected effort of a fix will appreciably alter the projected timelines or will instead serve as a distraction. The third and fourth options are probably obvious as to the implications.
At this point, the Agile afficionados I have spoken with over the years will say “so there’s the answer, you don’t make firm commitments at a fixed time.” See how easy it is to say that? The advice is literally “just don’t do something so obviously sub-optimal.”
If it really is that easy, it’s a good sign (though not a guarantee) that it’s not a practical possibility.
Let’s get pedantic for a minute. The point of a for-profit business is to convince others outside the business to commit to transferring ownership of capital, and products or services are used as the leverage to accomplish this (among other things, but it’s the primary thing). On the other side of that deal, the party who is relinquishing ownership of the capital (you know… the customer) has expectations on the utility they expect to derive from this transfer, and these expectations are expressed in terms of functionality and dates (again, only primarily). The nature of the deal dictates the flexibility in either of these, as do the market conditions. If the customer believes that someone else can credibly deliver an adequate balance of utility, time, or cost, you had better expect that customers will respond.
That’s a long way of saying that sometimes, you don’t really have the option of choosing your functionality or time frames. Other times, you might. But pithy points pertaining to Agile methodologies aren’t going to appreciably affect that either way.
This job I’ve been speaking about was where I learned the magnitude of the importance of the intersection between engineering process and business goals. In effect, the long-term success of a business (at least long enough to have success, as defined by business goals) is dictated by the business’s ability to
- consistently identify the correct problems to solve to deliver sufficient utility to the customer.
- produce solutions within the cost and time bounds that the customer is content enough with.
These two have interesting interactions with each other. Choosing problems to solve for customers is influenced by the utility that customers obtain for their exchange of capital, and the feasibility of the solutions to these problems dictates that utility (in terms of function/time/cost). However, the feasibility of the solutions is dictated by the business’s ability to execute on the plans that meet the utility constraints dictated by the customer and market.
Even beyond this, once a problem and solution have been chosen for target delivery to market and the engineers take over to design and build the solution, these two questions never really go away. They need to be applied in each decision that the engineers make. A colleague of mine was fond of saying “engineering is about trade-off decisions.” That rings true. There are myriad ways to design and build any particular solution, and on the way, there are myriad problems to be solved. The tricky business is to identify the correct problems to solve and to staff them appropriately to solve them.
So the question then is how to know that the business is on a path to delivering a solution within the utility constraints. This always, always comes down to planning, estimation, and measuring.
Each of those topics (planning, estimation, measuring) could have a book dedicated to the topic. In this post, I’m primarily discussing planning, but I’ll make comment on each of estimation and measuring since these are all connected:
- There’s a right way and a wrong way to estimate. The wrong way is to stare at the definition of the work to be performed, to envision yourself implementing the work, and to formulate a gut feel about how long it would take to produce the code. This is almost never even remotely accurate (but it is better than nothing and may be all you’re left with if your planning ecosystem doesn’t permit better). The more information you’re able to feed in to this process for comparison purposes, the better. You need to find similar efforts in related areas that had similar architectural requirements, similar testing requirements, similar problem types, that you have actually run through the most-representative approximation of your development and testing process and determine an estimate based on these, in addition to what you know and don’t know (i.e., a “confidence interval” of sorts).
- Measurement is poorly used and abused. Choosing what to measure is difficult. As a consequence, the chosen metrics might have very little to do with the actual business goals, and then those metrics effectively replace product goals. The results of doing this are a lot of thrashing, as plans drastically change (note, they don’t evolve or grow in this case, they simply change) when the metrics change. And this is a natural consequence, because the point of the engineering metrics is to tell you how suitable your solution is to the customer’s problem, not to tell you when you’ll ship it. What is being measured should correspond directly to how the plan is progressing, and not be a “meta-metric,” where by you conflate engineering-oriented metrics with planning-oriented metrics and engineering-oriented metrics become planning-oriented metrics (there are times when this is OK—if you’ve intentionally engineered the system against cascading requirements, or of this is so internalized by the entire team that it is as if it has been engineered that way, then it can work to combine the two; but I don’t think most companies are honest about their knowledge of their teams’ disposition in this regard). It’s kind of like architecting software: put the proper abstractions in place so that concerns don’t leak between layers. What should you choose to measure? That’s the rub, and that’s where this often goes sideways. Determining what to measure is as much a requirements-gathering effort as is designing the system to meet user goals. You either know these up front (not likely), or else you need to decide on them iteratively. To minimize thrashing, you must have a history of execution against a plan to make intelligent decisions as to what your metrics will be. In effect, don’t choose the metrics by which you’re judging an effort’s success until you’ve got a working plan.
How do you get a working plan? The message on planning isn’t really any different from the other two. You build a plan. You determine your progress in short intervals (say, 2-4 weeks). After each interval, you re-plan each effort. Why do you re-plan? Because you have certainly learned new information in that time. If you’ve discovered a mountain of new work, or you found that your estimates for the work that you’ve completed were off, this is vital information that must be used to adjust the project plans! How do you know when you’re centering on a longer-term plan? When the iterative plans over time returns nearly the same plan, with only minor extensions, you’ve found your long-term trajectory. And now it makes sense to forecast your effort.
Essentially, this iterative sampling/planning forces you to only plan in the near-term against the things with which you have high confidence, to build a database of past efforts to compare the remaining roadmap against, including the rate at which you create work that is not part of the roadmap. Of course, all of this requires time and effort (and honesty). It changes the culture of a company or team, and makes it more deterministic. Is the requisite effort worth the results? I think this is ultimately a question of profit margins and profit goals in view of time frame and staff expansion abilities. It isn’t worth the effort if you cannot afford to pay staff to do the work and have no plans to grow. But you plan to grow or if your projects become large enough, at a certain point it will become mandatory, or else you will fail (but hey, sometimes failures turn out well for some involved parties—what are the incentives your engineers and their leaders have?).
Bluntly, this is an example of weighing risks—I’ve seen that most companies aren’t interested in weighing those risks. And this is why most projects come in massively over budget. As a tangent, this lack of care for building a sustainable process to obtain transparency in goals leaves leads and managers to conduct their professional efforts so as to maximize their incentives—and as we saw with the earlier example with the weekly VP meeting on a failing project, if the incentives are not for transparency, or if there are anti-incentives for being transparent, then you can expect to see overruns without clear explanations, and without the ability to head them off.
Back on topic; amusingly, you’ll find that as you add people to an effort, you usually won’t see that the remaining work on the roadmap shrinks proportionally to the number of people on the effort. Instead, your roadmap will likely expand. This observation is different from Brooks’ Law. Rather, it’s that the more people you add to an effort, the more people identify the shortcomings of the planned work, and they add new work to fill the gaps they see on the roadmap (either because the roadmap actually has gaps, or else simply because people approach work in different ways). If you’d like, you can call it Kemp’s Law (at least until I learn that this was documented somewhere else and remember to edit this post). This phenomenon is good and right; this is where you learn the honesty of your estimates and the accuracy of your plans. But it requires very strong technical people leading the projects with the business interests in mind to understand the trade-offs in selecting which work to accomplish since you likely won’t have time for all of it. They also need the clear authority to make these calls (side note, this is a large part of what a Technical Product Manager does), with lead engineers working with these people, not outside of them. If there is a social hierarchy within your org that undermines this clear authority (which shouldn’t mean dictator-like authority), that’s a cultural problem that the top leadership needs to address, or else you can expect to never know how close you are to shipping a working product (did you ever hear those stories of a product being released that was clearly not ready?).
So, honest planning is planning that produces highly-confident and accurate plans for the near term, in view of the realities of what was already accomplished compared to what was planned, and constantly updates future plans in the same view. Don’t commit to eating the whole bowl of porridge before sampling it and knowing what you’re getting your team in to. Also don’t undermine the execution of the short-term plans that you have made. Make honest assessments of whether something is truly worth bringing in to your short-term execution as unplanned work vs. putting it off until the next planning iteration. There is no amount of setting aside space for unplanned work that can deliver results as well as actually planning the work. In effect, Porridge Planning is just right planning. Not too long to see failures and learn from them, not too short so as to perpetually be working on unplanned work. If top-level leadership is not capable of building such a process from their own skills, then it’s clear where the next hires need to be.
There is more to discuss on this topic at some point—like working with the momentum provided by Conway’s Law to ensure that each team’s effort is not wasted even if another team cannot deliver in time for the product, and employing Quality Engineering (as opposed to Quality Assurance) mechanisms to ensure that quality concerns are addressed as a part of planned efforts, not unplanned efforts that cause schedule overruns. However, this post is long enough already. But there is one aspect that still needs to be covered.
One of the first posts I had put up was about ensuring that what is being built is what the customer actually wants and will use, not simply what they say they want and will use. This will necessarily cause re-work in plans, and the important thing is that the feedback is coming early enough so as not to drastically disrupt the momentum that a development team is building. More over, you do not want to sink significant effort into something that the customer won’t even use. You need to have validation of your solutions as you’re working on them, not after. This can mean more meetings with clients, A|B testing, mockup walk-throughs, or anything else that will give you a means to collect this information. Not collecting it puts your plans at significant risk, which in turn means you can never actually know how close you are to having a product.
And lastly, the payoff of all of this is that when (not if) your plans are going off-track, you have a realistic starting point to get opinions from customers. The longer you wait to get the feedback, the more strange your adjustments may look to the customer. If the changes appear too far away from the agreement, it comes as a shock, and this breeds mistrust. Mistrust breeds sour relationships. Sour relationships breed lost sales.
One job of mine had me reporting to our executive staff (of a company with a sizable engineering presence in multiple countries across the world) multiple times, and to joint meetings with customer executives almost as many times.
My observations were that any time we reported to our customer that the prior decisions were leading to undesirable results in some way, as long as we presented the potential mitigations and explained why we had confidence that these were the best options, the customer would accept that these were decisions borne in competence and good faith, even amidst the tension of the moment.
A different time, there was one lingering question whether we had a workable solution for an important part of the customer’s product line. There were, of course, trade-offs that could be made. It’s simply speculation, but if we had shared these concerns with the customer early, this would have bought time for us to conduct trade-off experiments and to perform more market research in these specific areas to demonstrate that our solution was admissible. Instead, this information was shared relatively late in the process, and our suspicions that the customer had continued conversations with another vendor were confirmed when they announced that they wouldn’t be choosing us for the next product line.
My point in this is that software development systems are just that—systems. The business side is not disconnected from the engineering side. Engineers can’t create utility without consulting an authority as to what paying customers will exchange capital for. This can work for a time, for start-ups, but eventually the magnitude of the business problems become too large. Likewise, if the business side does not have trustworthy visibility into the engineering activities, they are fundamentally disadvantaged, unable to properly cultivate relationships with customers, built on trust and competence.
Porridge Planning and development is leverage. A business that builds a solid Porridge Planning process is building its own leverage.
Trevor Kemp 