Be Intentional

My first day of work at my first professional job was probably the most memorable of my career. This might be true for many or most people, but it seems to me like it might be more true for me than others.

I had just graduated from my Master’s program about a week before. While my classmates were going to commencement, I went to work. I was excited to join a company and to start helping them on a permanent basis. I moved to a new city, and it felt like there were limitless possibilities.

None of that is what makes the day memorable.

When I arrived that morning, I was shown my office (and man, do I miss working in an office with a door that closes. I haven’t had this in over 8 years). Soon after, I had my first meeting with my team, or at least most of it. The team that day consisted of me, a senior programmer who had been with the company for 5 or so years, and the project’s lead programmer.

The project lead was the founder and majority owner of the company. He lived in another state and had arrived the day before to help with this project, since it was expected that it would be a hard push. The senior programmer was there to hang on for dear life, on each wild ride this company took him on. I was there to learn what I had gotten myself in to. My boss wasn’t there at all. He was out coordinating with the customer the work that we were going to start that day. We were going to receive a call from him in a while to explain what it was we needed to do.

While waiting for the call, we started some preliminary planning. The other two developers had a basic understanding of the work that needed to happen, and really all we needed from my boss was some additional information that only affected the work of the lead programmer, since he was going to build the UI.

We hashed out a plan. The lead was to create the UI, as stated. The other developer was to incorporate the existing DirectX-based media capture dialog into the new UI, which basically meant disentangling and extracting the code from the last project, which to memory was highly coupled to the UI (but I could be wrong on these details, since I wasn’t involved until that day). They hadn’t assigned anything to me, so I asked what I could do to help.

I don’t remember quite how it happened, but the next thing I new, I was in charge of the storage layer for the application, and we were going to use a SQL-based database. At the time, I was the only one in the group who had ever written a SQL query, so I was in charge of choosing a database technology, setting it up (including the schema), creating a set of code interfaces around this database, and writing every query we needed.

Then the question came, from one of us to the lead, who had by then had his call with my boss.

“How long do we have?”

The answer came back, “2 weeks.”

Next came uproarious laughter from the senior developer.

I thought that he was sarcastically laughing because of the mountain of work before us. I didn’t think it was all that funny. I didn’t think we’d be able to get such an amount of work done in 2 weeks.

It turns out, I was wrong. He was laughing because they had an ongoing joke that any project could be done in 2 weeks, as that was essentially the standard length of time they quoted customers for any project.

The lead said, “no, we have 2 weeks.”

There was a demo scheduled after this period. The other developer asked about whether we would have to work over the weekends, and this clearly bothered the lead. The annoyed reaction seemed like a dismissal that combined a sense of bother over a lack of dedication with bother that we already weren’t trying get the work done in the time given.

Over my time at the company, we would have that conversation many times with various people. “It doesn’t matter how long it will take; this is how long we have.”

It wasn’t as bad as it sounds, or at least it wasn’t for me. I didn’t mind putting in a ton of extra time, I really liked most of the people I worked with on a daily basis, I believed in the product, I wasn’t used to anything different from school anyway, and I was engaged in all of the work I did. I didn’t even mind that I was to work on the database, something that I really didn’t care for too much.

To this day, I don’t know why we chose to use a database. Using directories to structure the data on the file system would have been much simpler and served our needs perfectly well, but we didn’t go that route. I remember going home that evening, excitedly searching the Internet to learn how to insert images into a database. I learned that most databases have an option to insert a binary blob, but also that binary data was often encoded in base-64 for storage and transmission. These are small things, but it was fun to learn something, and it was exciting to be a key part of a team.

There are a ton of stories I could tell from these first two weeks. There were a dozen or more things that I have never seen at another company (for both good and bad), that I’m unlikely to ever see again, and probably 4 of them happened in the beginning. To their credit, their software projects were typically big bets for such a small company, and their products meant something to every employee; they built a culture around serving the customer, not a hyper-aggressive company-first culture where employees are fighting each other for money and influence. As I saw from the first day, this company was unique in the way everyone they hired got involved and invested.

But what wasn’t so unique in my career was this company’s position on planning and supervising projects. In the time I was there, we never delivered anything on time until the final month I was there, which happened to be one of our “2-week” projects.

The first project was 2 weeks later than intended, and we scrambled to get the demo into the field 2 weeks later than anticipated. It had a couple bugs (none in the storage layer, thank you very much!) that made the demo difficult. I remember being an hour from my computer, on a Sunday in June, getting a call from my boss about how to work around an issue. However, it went well enough that the CEO called to thank us for our effort 2 days later.

But what wasn’t clear at the time was that the demo’s success was only by outward appearances. There were some problems coming out of this demo, some of which took months to come to light:

• The customer was displeased with the UI. It was not as easy to use as they had in mind, and during the demo, there was no way to review the data they had collected.
• The data was structured to make it easy to automate the generation of paper reports, which we made no attempt to generate for the demo.
• There was no way to demonstrate getting the data out of the device. In fact, because all of the data was stored in a database, we couldn’t even give them a means to access the data they had collected. We had to have the field representative perform a dump of the database and upload it to our FTP server so that we could re-create the database and extract the data.
• Because of the format the data was stored, it was difficult to do anything useful with it along the lines of how the customer would normally use the data they collected through their process. Essentially, we just didn’t fit into their workflow. Typically, their engineers would hand-build reports and substantiating data, and upload these into a global database in their org.
  • The engineers are highly-trained people, nationally certified, and highly-qualified for the work they do. Their reports were succinct and had an inherent high quality, since the data was hand-selected by these engineers from the larger corpus of data they built during an inspection.
  • As envisioned, our product would never be able to generate reports to compete with these engineers’ reports. The generated reports would be more verbose and “noisy,” since the engineers would not have the option of hand-building a succinct report. Alternatively, they would need to collect exactly and only the data that they needed to substantiate a report, which is too much to ask of a user.
  • In an alternative suggestion, we would extend the UI with an additional editing interface, and the engineers would need to sit and edit reports on a screen the size of a small greeting card before clicking “generate.”

Most of these issues came to light before we started the second iteration about 8 months later, but we were only putting effort in to one of them, and it was naturally the one the customer had told us about. We spent the remainder of our time with this customer working to build them the UI that they wanted (and the reason this process failed is a really interesting story for another day).

Our program manager was the primary one defining requirements for us on this project. He was a really interesting person, one I wish I had been able spend more time with outside of the project (but too bad, he lived in another state). He was an expert archaeologist with unique view on how to use technology in this work, and the product was, essentially, his idea. He was uniquely suited for this position, but neither he, nor my boss, nor any other person involved with the effort were equipped to do the requisite pre-work on the project to determine what problems the customer had that we could appreciably improve. Nor did anyone think that there was any risk that we didn’t understand the problem (which would come back to bite us later).

• Our ultimate desire was to become the go-to tool for this customer’s data collection.
• Our immediate goal with this project was to automate the generation of reports for inspectors.

We never determined that our immediate goal was actually on the path to our ultimate desire. We had never undertaken an exercise to build and validate plans that took us from our humble beginning to our ultimate desire. The assumption was, because our program manager was a subject matter expert and we had a person on the customer end agreeing with our program manager, that there were merely details for these people to work out, and we needed to just take one step at a time. Nobody had the realization that this formulation of the project could never work (more anecdotes to come on this at a later date).

Nobody asked and answered the question: once inspectors are able to automatically generate reports with our device, what makes it so that this is something they will desire to do?

Nothing—nothing—comes for free. Plans take time to build. They also take time to validate. They also take a tremendous amount of energy to continually groom and re-validate. Our leadership was right, that if we took the time to build and validate a multi-step plan, this would probably push us past the timelines that the customer desired for this demo. This alone is probably strong justification for not building an elaborate plan to get us to our ultimate desire at that time.

It does not, however, mean that it was appropriate that this exercise never be done at all. In actuality, we were optimizing our actions to balance our perception of two different risks:

• Extremely visible and acute short-term risk that we would miss a customer’s timeline and lose an opportunity.
• Much more opaque, but no less severe, long-term risk that the customer would not even have desire to buy our product because we were not solving problems for them.

Our habit was to pay mind only to the short-term risks, and to never invest time and effort into the longer term. In effect, we paid 100% of our short term risk by accepting 100% of the long-term risk. We moved our risk debt from one credit card to another, with no plans to pay off the latter.

It is a real possibility that a company will dump time and effort into a plan that does not end up working out. My recommendation is to do it anyway, as soon as is possible, for two reasons:

• To have the opportunity to drive the decision to continue or discontinue a project.
  • Do not let sunk cost fallacy force you to do the same old thing, with no difference in output. It is OK to terminate a project and re-balance resources to other efforts. It is much more painful to release staff after having lost a fortune on a product or project that got out of hand.
  • Do not let failure sneak up on you. Have the ability to make educated forecasts on ROI. Make sure an effort continually makes sense to pursue.
• To have the ability to conduct a postmortem when a project or product fails.
  • It is extremely difficult to objectively determine where your plans went wrong if you do not make the plans formal.

Remember, the easiest part of a successful software product is writing the code! If the easiest part of the process yields results that are unexpected, that is a really strong sign that it’s time to review the plan, to determine if our understanding of the risks to our ultimate goal remain accurate.

How do you know a proper balance between investing more in planning and risk discovery vs. implementation? Try this: when you are no longer caught completely by surprise by an outcome, even if it is not a desirable outcome, you are probably planning your projects enough. If you are never surprised by the outcomes, and you also seem to miss opportunities, you might be working too hard to uncover and mitigate risk.