Recordings Are Coming, I Promise

I've given this Clean Code presentation of mine at work a couple of times. It's pretty simple, just a sort of intro into Robert Martin's book, and it's mainly targeting junior developers. But it's been very well-received at work and so I figured I'd do a screencast of it to share more widely. Indeed, I've been wanting to do screencasts of other things as well, so this will be a good first step into that medium.

As it turns out, this is really bloody difficult to do. I've been recording take after take, always finding something wrong. Editing isn't exactly a perfect process either, since I'm not satisfied with it if it doesn't seamlessly line up. So I continue to try these takes over and over.

A phone rings in the background, the kids come into the room and require attention, I stammer or make a mistake, the battery on my presentation tool runs down, I try taking the laptop somewhere else to do a quick recording attempt but the battery runs low, other system notifications pop up, etc.

This is hard. By comparison, actual public speaking is a piece of cake. You just do it and it's done. But privately recording something for public consumption, that's a whole other beast entirely. When standing up in front of an audience, a stammer quickly becomes a forgotten thing of the past or a mistake can be corrected. On a persisted medium like a video, that's not acceptable. It's there forever for all to see. Perfection is more critical.

This is going to take longer than I expected.

Addresses in a Post-Google World

It comes up a lot in business applications... How should we design a form to accept addresses? We see all kinds of examples. There's the baseline simple approach:

• Address
• City
• State
• Zip

Then of course people want to add a second address line, so you end up with the very poorly-named variant:

• Address1
• Address2
• City
• State
• Zip

And then, almost inevitably, it spirals out of control from there. Should we separate street number from street name? Street type? County? Country? Should we pre-populate from a table of zip codes? Where do we get that data? How do we keep it up to date? What about other address elements? What about international addresses? Strange middle-of-nowhere addresses that don't fit this model? How does the post office do it? Can we copy their format? Can that format work without the miles of red tape and bureaucracy to support it?

It goes on and on. And, thinking about it now, it's all terribly silly. Why are we still trying to solve this same problem over and over? Every company ends up with its own formats, every format gets broken by some edge case at some point, etc. It ends up being a lot of work, and what's the gain?

Is there a simpler, more universal way?

Well, yes. Google already did it. (And Bing, and MapQuest, etc.) The form is actually quite simple and intuitive:

• Address

Done. That's it. The user knows how to type their address. They've done it a million times. You don't have to instruct them on the various components that you think their address should have. Just let them type what they've written for years. And you have their address. The address they use as they know it.

But what about the components? What if I want to report on this data somehow and filter records by state or zip or some other metric? Well, that's your problem. It's not the user's problem. Don't give them an ugly form that might not even work for them just because you have some back-end requirement that isn't meaningful to their user experience.

This doesn't mean you can't get this granular data. It just means that you don't need to get it from the user. There are plenty of address locating and geocoding services out there which will give you a hell of a lot more structured data than whatever random business user you assigned to this project would have designed in their bad UX form. Get this data behind the scenes. Don't trouble the user with it.

For example, let's look at the data you get from Google when searching an address:

{
   "results" : [
      {
         "address_components" : [
            {
               "long_name" : "1600",
               "short_name" : "1600",
               "types" : [ "street_number" ]
            },
            {
               "long_name" : "Amphitheatre Pkwy",
               "short_name" : "Amphitheatre Pkwy",
               "types" : [ "route" ]
            },
            {
               "long_name" : "Mountain View",
               "short_name" : "Mountain View",
               "types" : [ "locality", "political" ]
            },
            {
               "long_name" : "Santa Clara",
               "short_name" : "Santa Clara",
               "types" : [ "administrative_area_level_2", "political" ]
            },
            {
               "long_name" : "California",
               "short_name" : "CA",
               "types" : [ "administrative_area_level_1", "political" ]
            },
            {
               "long_name" : "United States",
               "short_name" : "US",
               "types" : [ "country", "political" ]
            },
            {
               "long_name" : "94043",
               "short_name" : "94043",
               "types" : [ "postal_code" ]
            }
         ],
         "formatted_address" : "1600 Amphitheatre Pkwy, Mountain View, CA 94043, USA",
         "geometry" : {
            "location" : {
               "lat" : 37.42310540,
               "lng" : -122.08239880
            },
            "location_type" : "ROOFTOP",
            "viewport" : {
               "northeast" : {
                  "lat" : 37.42445438029150,
                  "lng" : -122.0810498197085
               },
               "southwest" : {
                  "lat" : 37.42175641970850,
                  "lng" : -122.0837477802915
               }
            }
         },
         "types" : [ "street_address" ]
      }
   ],
   "status" : "OK"
}

That sure is a lot of very structured data. Was your random business user who defines the requirements for that form going to be able to get all of that out of the user? What would that form have looked like? It even gives back a "formatted address" in response to something otherwise unformatted based on what it found. So even if the user puts in something that looks messy you can replace it with something cleaner. You even get coordinates for God's sake.

Seriously, stop having your users enter their address components separately. It's as bad as giving them multiple text inputs for the components of a phone number. You don't need that level of granularity in your data, and if you do there are much more comprehensive and much more reliable services to give you that data based on what your users enter into the form.

Pick Any Two Constraints

Most of us are familiar with an industry adage:

Faster, cheaper, better... Pick any two.

The idea is simple. You want something soon (faster), you don't want to spend a lot (cheaper), and you want it to be a quality product (better). However, you're not going to get everything you want. Reality doesn't work that way. And so you choose the two that you want more and settle for those.

• If you want your software delivered quickly and to be of high quality, it won't be cheap.
• If you want your software cheap and of high quality, it won't be delivered soon.
• If you want your software delivered quickly and cheap, it won't be of high quality.

I've encountered plenty of people in the industry who are familiar with this, but for some reason it never sinks in with customers. Maybe project sponsors are just too accustomed to getting everything they want that they can't imagine having to pick something to sacrifice? That's a cop-out answer, so no. Or, perhaps we're not communicating the concept in a way that they truly understand? This is a much more productive approach, because it puts the onus on us to better facilitate that communication, which is our professional responsibility.

So instead of choosing which two you want, instead think of it like this:

Schedule, budget, quality... Choose the two by which the project is constrained.

Unless the project leadership has a complete and unerring understanding of the entire project (which I've never seen happen, so let's just assume for a moment that it's not realistic), the project will require some "wiggle room." In this triangle of constraints, it's going to move along an axis somewhere:

The good news is, you get to decide where:

So which two of the three axes will be fixed for the duration of the project, and which one will be allowed to fluctuate? That's for the business to decide. The most important part is that this be communicated explicitly and without confusion before work begins. It's easy for assumptions to be made when communication isn't explicit, and attempting to scope and document those assumptions is a fool's errand.

Ah, but there's a catch. (Isn't there always?) One of these three axes is a lot more difficult to quantify than the other two. How do we, as an industry, measure software quality? What metrics do we employ? It's a bit of a difficult question, isn't it? Now take it a step further and try to answer that question in a way that a non-developer can understand. We can talk about unit tests and code coverage and continuous integration all day long, but a project sponsor isn't going to hear us. "It's all Greek to him" and MEGO Syndrome sets in quickly.

This is a fair response. After all, how much attentiveness and interest is the developer going to be able to maintain if the project sponsor steers the discussion toward budget forecasts and marketing predictions? I can assure you that the moment somebody breaks out some spreadsheets to talk about finances, my attention span is entirely dissolved. (Some of you may enjoy such topics, of course. But I'm sure there are others which thoroughly glaze your eyes and put you to sleep. So the point remains.)

Ay, there's the rub. And I wish I had an answer right now, but I don't. Not yet anyway. So the challenge is ours to defend our axis of quality. However, the decision itself still belongs to the business. Just because we can't quantify it doesn't mean we should sweep it under the rug.

If you constrain the project by schedule and budget, quality will slip.

Plain and simple. The stakeholder(s) will ask us to quantify that, and right now we can't. (If you can, please speak up. The industry needs to know what you have to say.) We can quantify schedule, and we can quantify budget. Those are pretty clear and understandable metrics. And so (unerringly in my career so far) the business will choose those two metrics as the fixed axes. Those are what they understand, so those are what they choose to control.

The quality axis isn't in their scope of control, so choosing that one would rely them very heavily on you to control it. It would leave them with no guarantee that they're controlling any more than one axis, which is no comforting situation at all. So, again, the onus is on you to communicate and "sell" this idea. At least for now.

The critical piece, however, is that the idea is at least communicated. The business has every reason and every right to sacrifice quality in order to meet schedule and budget. It's unfortunate, and it leaves a sour taste in our mouths as engineers, but it's their decision and not ours. Don't make the decision for them. Don't speak only in terms of two constraints.

As professionals it's not only our responsibility to craft a quality product, it's even more importantly our responsibility to communicate the risks to that quality when discussing the product with the business. And, conversely, as professionals it's the responsibility of the business to understand the reality of what we're communicating (provided we can communicate it effectively.) Or, to put it another way... They won't be mad if you set the expectation. They'll be mad if you make their decision for them.

Observations of an Offshore Team

There's certainly no shortage of offshore teams in software development. The stereotype of an Indian sweatshop of programmers and the garbage code they produce is well known throughout the industry. And I'd be lying if I said I hadn't seen that stereotype in action again and again. But I can't let that sour my perception of all offshoring in general. Particularly because my company now has an office in the Philippines for providing low-cost development work.

First of all, the Philippines is a far cry from India. The country and the people are vastly different in every way. I'm not going to compare apples and oranges, and besides that's outside the scope of these observations anyway. I was in the Philippines, not in India. So I can't speak to the latter. But hopefully what I've learned in working with the former can be applied somewhat universally.

The difference between working remotely with an offshore team and actually going there is profound. The perceptions, from both sides of the proverbial fence, change entirely when sitting in the same room. And I can't recommend the experience enough for any company which partners with an offshore team. Naturally, it would be cost-prohibitive to always work face-to-face in this situation. But sending at least one or two representatives to work directly with the team makes all the difference in the world.

If any company or team expects that they can simply toss requirements over a fence and expect production software to be tossed back over at a specified date, they're deluding themselves. It's an absurd notion to begin with, and years of industry headaches have confirmed that. Just being able to have casual conversations over lunch or dinner regarding the business domain and intent of the project can save untold costs in what would otherwise have been software that didn't capture the actual business intent. Understanding the why is vastly more important to software development than meeting a set of defined specifications.

There's also the psychological aspect to it. A team is more of a team when they know each other. Prior to this trip, everyone in the Manila office was a name and sometimes a voice, nothing more. An IM window, an email, a conference call... That was the extent of it. Now they're not just names on a form somewhere. Now I dare say I can call them friends. Or at the very least colleagues. A forced workplace joke over a conference call pales in comparison to a Friday night out to grab a drink or two. The social aspect of the team helps us to communicate and to understand each other more readily. The "fence" is torn down.

(As an aside, I regret to point out that I wasn't able to socialize nearly as much as I had hoped. So the fence is still partially there. Oddly enough, I ended up socializing more with members of adjacent teams than with my own, which is a bit unfortunate. But even going out to lunch with my team from time to time was still markedly better than IMs and emails.)

But even just the perception from "our side of the fence" entirely changes when you go there and meet the people on the other end of the emails. It's one thing to memorize someone's name, but it's another thing entirely to shake their hand and have a conversation with them. It's one thing to be numerically aware of the time zone difference, but it's another thing entirely to work the same shift as them and experience what it's like to have time-shifted hours. (After all, it's easy for us to label a rowdy background din on a conference call as being unprofessional. But honestly, what do you think an office of software developers who are all friends with each other is going to be like after 9:00 PM?)

Then there's the change in perspective on the little things around the office. What does the office look like? How is it laid out? What sort of equipment are they using? What assumptions do we have about our own "normal" office environments actually translate to that office, and what assumptions don't? You might be surprised.

For example, there's a labor/tools anti-symmetry between office environments in the US and many office environments abroad. In fact, this anti-symmetry is specifically why companies in the US delegate work abroad. To put it simply, and hopefully not disrespectfully... people are cheap. In the US if there's something slowing down a team and it can be solved by the purchase of some technology, the choice is clear. Don't let anything get in the way of the people. Tools are disposable. But in many offices abroad, tools are expensive (more expensive than they are here, actually, due to varying market forces). But people are easy to find and easy to replace.

It's harsh, and it's distasteful to me, but it's a reality that pervades the industry. I like to think that my company is different, and we're aware that it's going to take some time to convince our new team members of that fact. As I've said many times before, I hate referring to people as "resources." People are individuals. They are contributors. They are team members. They are not "resources." And actually meeting your team and getting to know them helps a lot to make that distinction.

I really like our team. And I truly believe that there's real development talent there just waiting to be nurtured. I don't want to simply delegate tasks to them. I want to work with them. And that difference, I believe, will make all the difference in the resulting software. Maybe not on the first project, maybe not on the second, but in the long run.

The 90/9 Rule

We've all heard of the 80/20 rule, right? It feels like it's one of the most commonly-cited principles in the software development industry, in my experience. And there are always subtle flavors of it when cited. More often than not, it seems like it's in reference to "getting bang for your buck" on a software project.

And getting "bang for your buck" is a rather important concept in business software. Purists like myself are always compromising to the almighty dollar. And the reason for that is simple... We don't own the software we write. The person who writes the checks makes the rules, plain and simple. And that person is very interested in quantifying the value of what they get for the money they spend.

To help illustrate "bang for your buck" in the world of software, I've come up with a rule of my own. I call it "The 90/9 rule." And it's essentially like Zeno's Paradox of Achilles and the Tortoise applied to software development. Think of it as such:

• For a reasonable expense, you can achieve 90% of your intended goal.
• To achieve 90% of the remainder (thus, 99% total), double that expense.
• To achieve 90% of the remainder (thus, 99.9% total), double that expense.
• ad infinitum...

A key thing to notice is that it never reaches 100%. No matter how much expense one incurs, one will always have a compromise somewhere. The only question one needs to ask is where one is willing to draw that line between expense and intended results.

This isn't to say that we're not going to deliver complete and valuable functionality. It just means that there will be some things which the business intended or imagined that won't quite fit into the project. It always happens. And I dare say that a primary root cause of delays and shoddy results on a project is when the business won't let go of that full 100% intent. They won't let the reality of what's happening alter their perception of what they want to happen.

Considering the 90/9 rule in a project brings that part of reality to light. By accepting the balance of expense vs. intent, the business can (should) more effectively manage expectations and focus the spending of that expense on the 90% that's realistically feasible within the scope of that expense. And, if more is needed, the business can balance that against the added expense.

As an illustration of this rule, consider the simple matter of system availability, or uptime:

• 90% (one nine) is pretty easy to achieve and definitely within a reasonable expense.
• 99% (two nines) is still reasonable, but requires a little more expense. A proper server vs. some resurrected old laptop for example.
• 99.9% (three nines) is going to require considerably more. That's less than half a day of downtime per year. Consumer hardware at all probably won't cut it, nor will a consumer internet connection on which the server sits, etc.
• 99.99% (four nines) is getting really expensive now. That additional small amount of availability (a difference of a few hours of downtime in a given year) is going to require redundancy across multiple sites.
• 99.999% (five nines) will require solid redundancy. The expense to guarantee this level of availability is no small matter at all.
• and so on...

The exponential growth in expense is easy to see. What's also easy to see is that 100% isn't on the scale. 100% doesn't exist. No amount of expense is going to account for every possibility. You can get very close to 100%, but you won't reach it. There will be a risk somewhere.

The illustration of high availability focuses on hardware, but the same holds for the availability of the software. How solid, validated, and bug-free does 90% reliable software need to be? How solid, validated, and bug-free does 99.999% reliable software need to be? You can imagine the amount of planning, testing, validating, refactoring, and overall hardening of the system that would be required to achieve that level. And you can imagine how expensive it's going to be, bringing in experts from every field used by the software (architecture, infrastructure, database, UI, etc.) in order to achieve that level.

Think about the 90/9 rule on your current project or your next project. How well does it map to your expectations vs. the project sponsor's expectations?