Details Are Important

There's a potential for a runtime error, so I should log it...

This is in a catch block, so I also have the additional context of an exception. So let's add another function argument to use the overload...

:::sigh:::

The Only Way to Go Fast is to Start Well

As happens from time to time, there's a cross-blog debate stirring in response to something somebody posted. It's not quite as widespread as The Software Craftsmanship Debate from a year (or two? has it been that long) ago, but it's spreading. The catalyst this time was a post by Robert Martin entitled The Start-Up Trap.

The article is somewhat emotionally charged and comes across as being a bit dogmatic, which is Martin's style in general. (I'd be lying if I said I didn't exhibit a similar style.) But looking past the tone one can find the point pretty plainly... Businesses often delude themselves into believing that they don't need to write their software well. They're not "at that stage" or they need to focus their efforts on other things.

(What's worse, many times it's not the business which actually makes this decision. Many times it's made by the developers on behalf of the business without the business actually knowing. But that's another issue for another time. The point there is about professional behavior, and Martin has a whole book on that called The Clean Coder.)

Reacting to the dogmatism, there have been various responses to his post which try to focus more on pragmatism. One such post by Greg Young caught my attention as being one of the more equally-emotionally-charged responses. Another by Jimmy Bogard boils down to a much more level-headed statement, which I think really captured the pragmatism effectively.

And so, to reply to accusations of favoring dogmatism over pragmatism, Martin wrote a response. I feel like this settled the dust a bit, which is good. After all, we're talking about pragmatism. Do what's in the best interests of the business, that is the bottom line. There is no room for dogma on the bottom line. Or, to put it another way:

One developer's desire to be right does not outweigh the business' desire to reduce operational costs.

But in all of the back-and-forth, I think there's a very important issue to which we have only been alluding. And this issue is very much at the core of the communication (or, in many cases, the miscommunication) between all of the advocates on this debate. That is, if the question is:

When does TDD slow you down?

Then the answer is:

When your code isn't designed to be tested.

This, I think, is the great misconception around whether TDD speeds you up or slows you down. Young's article alludes to this when he describes his business venture and the application they prototyped. Martin validated this when he talked about a small one-off utility he wrote for one-time use (where he "fiddled with it" more than "designed it"). Neither of these applications were meant to be tested.

One can argue about the business sense of whether or not an application should be designed to be tested. Young uses an argument which I feel is very often overused, making it difficult to discern when it genuinely applies (which it indeed may have in his case, as I said it's difficult to discern). The argument that "it worked therefore it must have been correct."

Yes, many businesses succeed with crappy software. (Just look at Microsoft. Sorry, couldn't resist.) So, if they succeeded, how could the software have been bad? Evidence (the success) suggests that it was good, right?

This argument is just that, an argument, because to my knowledge there are no real numbers on either side of the equation. There are no double-blind studies, no usable case studies which aren't affected by tons of other variables. Did the company succeed because of the software, or did they succeed through the blood, sweat, and tears of their tireless employees despite the software? Most importantly, could they have been more successful with better software? How do you define and quantify success without anything else to compare it to?

I'm moving into a tangent. Sorry, I do that sometimes. Back to the main point... TDD only works when it's applied appropriately. It's not a magic wand which always makes all software better. This sounds a lot like the pragmatism being proposed by the various responses to Martin's original post. Which makes sense. But it's only part of the message.

Many times, when somebody says that they don't have time for TDD or that it will slow down their project, it's because their code wasn't designed to be tested, which itself is probably the deeper and more important issue here. The developers who make this argument often find themselves faced with the same difficulties when trying to write tests for their code:

1. The code needs to be re-designed to accommodate the tests.
2. The tests are slow to develop and not very effective, not providing any real business value in and of themselves.

Well, when faced with these difficulties, of course you're not going to want to write tests. They're making things harder and who wants that?

The problem here isn't that TDD is slowing you down, it's that your code is slowing you down. (Again, I fully recognize that sometimes you just need to fiddle something together until it works, and the various bits of what we'd call "good architecture" aren't necessarily required.) You're decrying TDD because it doesn't work for you, and you're assuming that TDD is to blame for this.

TDD is not just something to put on a checklist. It's not just a line item added to the project plan.

• Is code written? Check.
• Are tests written? Check.
• I guess we're done, then.

No, this is an oversimplification. If the code is not testable then what value are the tests going to provide? You're writing tests (or trying to, anyway), but you're not practicing TDD. So to decry the merits of something you haven't even tried is a bit misguided. (I'm reminded of religious zealots who decry science with no understanding of the scientific method. You know, speaking of dogmatism.)

One does not simply "write tests" to practice TDD. One must also write testable code. Without that, the tests will indeed appear to be a hindrance. This isn't because writing tests is slow or because TDD is all dogmatism with no pragmatism, this is because the code is already rotting. The misdirected attempt to add tests and claim that it's TDD is suffering as a result of that rot.

TDD is not alone in this. Any good practice, when one attempts to apply it as little more than a checklist item to an already rotting codebase, will appear to be a hindrance. After all, it's hard to fix something that's broken. It's especially hard when we can't admit that it's broken or don't understand why it's broken. Of course, if we don't fix it, it will only become more broken.

This is where the communication between the opposing sides of the debate tends to break down.

"Writing tests adds work to the project, it slows us down!"

"No, writing tests speeds you up, you just don't know it!"

In between these two statements is the difference in the codebase itself. And since the code is the one and only real source of truth, we should stop arguing and go look at the code. Any time we have this argument, the best thing we can do is sit down over the code and pair program. Maybe one of you is right for a reason unknown to the other one. Maybe the other one has a point the first one didn't previously see. Discover it in the code.

There is no Agile without Delivery

Colleagues and clients alike have been debating the nature of "agile development" lately. Everybody seems to want to "do agile" but for some reason it's just not working. Why is that? We're creating user stories; We're assigning story points; We're arranging work into iterations; and We're holding meetings every day which clearly have the word "Scrum" in the Outlook meeting invite. So what are we missing?

Clearly we're just not doing agile hard enough! Right?

It's a fine effort, but it's missing the point entirely. Calling something "agile" doesn't make it "agile." Let's enumerate a few facts about this very common series of mistakes:

1. Agile is not a meeting that you have.
2. Agile is not a project plan that you create.
3. Agile is not the terminology that you use.

So then what is agile? If it's not the meetings we set up in Outlook and it's not the project artifacts that we create, well, what else is there?

Agile, at its core, is the way in which we deliver the product(s) that we build. And for most enterprises it appears to be a frighteningly fundamental shift in how they expect the product(s) to be delivered. Because, ultimately, the biggest change is the delivery itself.

I'm talking, of course, about continuous delivery. The notion that each iteration delivers a complete and working product each and every time. Now, naturally, there will be bumps along the way. Nobody is perfect. So you can expect some iterations to fall short. And that's where the learning, adjustments, and... well... agility come into play.

But the words "complete" and "working" are key here. And they represent the fundamental culture shift in the delivery of the software which proves to be a significant stumbling block in the adoption of this process within a corporate culture. "Working" isn't difficult to understand, it just means that the product should essentially be close to defect-free and should functionally perform as expected. But "complete"? How is that even possible?

Therein lies the fundamental change... Defining (or, rather, re-defining) the word "complete." In this sense, "complete" changes with each passing iteration. I understand that the business has in their minds an image of what the "complete" solution should look like. What it should do, how it should behave, etc. They may have even spent a lot of time and money documenting every nuance of what the "complete" solution should do. And that's awesome. We can use that.

However, we can't use only that. Let's assume that, for all intents and purposes, it's going to take a team of people a whole year to develop this solution. The business agrees, the developers agree, everybody is optimistic that this is a good estimate. (Let's assume for a moment that accurately estimating an entire year of work is even possible, which I contend is not, but that's another issue entirely.) Then what? Should the developers just hide away in a cave for a year and eventually emerge with a complete solution that matches the specifications?

Clearly not. This would carry with it the enormous risk that the developers misunderstood the specifications, or the specifications misrepresented the core business need, etc. In this drastically waterfall process this would result in wasting an entire year of expensive effort on something that didn't meet the needs. No, we want to identify issues early (and often). We want to know when something is wrong as soon as possible so we can correct it with as little impact (and cost) as possible.

After all, as much time and money went into that specification, I guarantee that it will have flaws. There will be vague notions open to interpretation, or logical inconsistencies and mutually exclusive requirements. It may be difficult for the business (or certain members therein) to accept, but their gold standard document is not what's going to be built. It's just a document. And it's a document that was created at the earliest possible point in the Cone of Uncertainty.

(I'm kind of reminded of a quote from Steve Jobs: "Everything around you that you call life was made up by people that were no smarter than you." If you've ever done government contracting or, God help you, military contracting than you've encountered this before. Tons of time and effort and money was sunk into creating that gold standard specification. It was designed and approved by Top Men. So it must be flawless, right? Hardly.)

So we turn to agile. In agile we want to keep the business involved. After all, this is their project. They're spending a lot of money on this and they should be able to control it. (Not micro-manage it, hopefully. We've all seen that before as well.) If something is going in the wrong direction, they should have the reins in hand to steer the project in the right direction quickly and easily. That, naturally, is agility.

So how do we achieve that level of steering control from the business? (I know! Let's have regular meetings and call it a "Steering Committee"! It has the word "steering" in it, so it's what we need to do, right?) Well, in order to do that we have to give the business the useful information they need. We have to deliver something to them, every step of the way. Based on those deliverables they can track progress and make sure we're still going in the right direction. They can steer our course into that Cone of Uncertainty.

So... What are we delivering?

This is where that fundamental culture shift takes place. Traditionally, we're delivering project plans and status reports. We took that year of work, broke it down into chunks, broke those down into pieces, broke those down into bite-sized morsels, and so on. Then we arranged them into iterations or sprints or whatever we feel like calling them. Each one was assigned a weight, and we distributed the weights across the iterations. This gives us our burn-down which tracks our progress. We now have numbers to show our progress.

So... What are we delivering?

Can the business really steer us based on just those numbers? They're great numbers, don't get me wrong. (Until they lie, and charts and reports can always be made to lie.) But what information do they give the business? They tell the business if the project is delayed vs. on-schedule, and that's important. But they don't really address the uncertainty, do they? What happens when new information is discovered and adjustments have to be made? What happens when things become blocked and get shifted around and now the bite-sized morsels are out of order and dependencies aren't in place when they need to be? More to the point, what happens when specifications need to change as our collective understanding of the product grows and evolves with the business? Can we reflect that in numbers?

Perhaps, but not accurately. And not in a way that really helps with the uncertainty. We're just delivering documents in response to that original gold standard document. What we're not delivering is an actual product that the business can sink its teeth into and really, truly know what's happening.

We missed a step.

Arranging work items into iterations was a step in the right direction, but it didn't get us where we really needed to be. We also needed to define deliverables for those iterations. Business-visible deliverables. Consider these two iteration outcomes:

1. In Iteration 4 we completed the data access layer for the domain object repositories.
2. In Iteration 4 we completed the form for submitting an order to the system.

Perhaps both of these components had the same weighted values. They required the same amount of effort. They provided the same numeric impact on the burn-down. But which of the two can the business actually use to track the progress of the product? Not the progress of the project, the progress of the product.

Not the progress of the project, the progress of the product.

The deliverable for the data access layer is important and it needs to be done, but in its entirety it doesn't give the business anything they can use to track the product. It's too horizontal to the overall system, and the business really needs things to be more vertical. They need features to be completed, not technical implementations. The deliverable with the order form completes a feature and gives the business something they can see.

Therein lies the "completeness" of the deliverable. If all we built was the data access layer, what are we delivering to the business at the end of the iteration? Nothing. It doesn't work. It doesn't do anything. Deliverables like this keep us in a perpetual state of being "almost done" with the product. (Which can very easily lead to a perpetual state of being "almost done" with the project, and that's a very bad place to be.)

We can't just keep pushing forward on every feature across the entire horizontal board of the product. We need to complete individual features through the vertical of the product and deliver those as discrete complete working products.

What is "complete"? Complete is a product that somebody can use. Complete is not a system component that the developers claim is correct. The business knows that, at the end of the year, they want a "complete" working system for their new line of business. They know that in said system they should be able to register a user, log in, browse the catalog, submit orders for products, track the statuses of orders, submit requests for help, administratively manage the products, etc., etc. That's "complete" at the end of the year.

So what's "complete" at the end of the iteration? First they get a system where they can log in. (Of course it's just with a single test user account, since there's no registration feature yet.) Does it work? Does it look right? Do the mechanisms for logging in perform as expected? Sweet! Next iteration...

Now they get a system where they can register and log in. Still on track? Cool, let's keep going...

Next iteration they get a system where they can register, log in, and browse the catalog. (Of course it's just with a handful of test products, since there's no product management feature yet.)

Next iteration they get a system where they can register, log in, browse the catalog, and submit an order.

And so on, as opposed to this...

First they get some architectural framing in place. We're creating the business objects which contain the logic of the system and adding our interfaces for the various peripheral components of the system. There's nothing to test yet.

Next iteration they get the architectural framing and the login functionality, so they can at least log in to the system, but there's really nothing to see yet. Some of the pages have been started, but there's no data persistence so don't expect them to work yet.

Next iteration they get the architectural framing, the login functionality, and more work on the various forms. We plan to write the data access layer in the next iteration, then stuff should start working.

Next iteration they get the architectural framing, the login functionality, the data access layer, and we're on average about 15% done with any given form. So there's still nothing they can really be expected to work.

And so on. In this case we're not really delivering anything. We have a burn-down chart and lots of numbers to assure everybody that we're still on track, sure. We're on track with the project. But how is the product doing? Does anybody know?

This strictly horizontal approach leads to a host of other problems as well. Stop me if you've never heard any of these before:

• "We just discovered a problem with implementing that feature. It doesn't really fit into the architecture. We need to adjust the feature because we've already put too much effort into the architecture and it can't change now."
• "There's a cross-cutting dependency on an external system. Nearly every part of the application needs it. We've been developing all of the partial components across the whole application while we wait for that system. But we just found out that the external system is going to change. They went with a different vendor. So we need to re-write a lot of this."
• "Development was on schedule, QA went very well, but UAT isn't going so well. The application meets the requirements, but the users just aren't happy with it. Some of them are complaining that it breaks their workflow or that it doesn't meet their operational needs. Shouldn't all of that have been outlined in the requirements?"

Why did we build a rigid architecture across all of the features before we examined those features for more details? Why did we deliver partially-complete features without a required system on which they depend? Why did we wait until the very end of the project before letting anybody see or interact with the product?

Instead, we should be delivering a complete and working product (not year-complete, iteration-complete) each and every time. The architecture should be the simplest possible to support the implemented features; The features should be complete with all dependencies accounted for (push up the priority of that decision on a vendor, or wait to develop the features that require it); and The users (or business desicion-makers of some sort) should be in the system every iteration to ensure the progress of the product.

The technical team must be willing to break apart the product into discrete deliverable features, and the business team must be willing to accept those discrete completed features on a regular basis. The more regular, the better. If the technical team wants to hide away in a cave while developing the whole system, or if the business team doesn't want to be bothered by the product until the whole thing is year-complete, then who is responsible when expensive changes are made (either through shifting requirements, discovery of new details, etc.) late in the project?

An iteration isn't complete without a delivery. Assess, prioritize, build, deliver. Every time.

Dependency Injection and Leaky Abstractions

Over the years I've generally made use of three basic methods of dependency injection...

• Constructor Injection
  

  public class Widget
  {
      private WidgetDependency _dependency;
  
      public Widget(WidgetDependency dependency)
      {
          _dependency = dependency;
      }
  }
  

• Setter Injection
  

  public class Widget
  {
      public WidgetDependency Dependency { get; set; }
  }
  

• Late-Bound Injection
  

  public class Widget
  {
      public WidgetDependency Dependency
      {
          get
          {
              return ServiceLocatorObject.Resolve<widgetdependency>();
          }
      }
  }
  

Equally, over the years, I've unilaterally preferred the third option. Though I've never really given it a whole lot of thought or attempted to articulate why. It was just a preference. To some (and indeed often to attendees of my What Is Dependency Injection presentation) it can certainly seem like the more complex choice. Sure, it does present a little more code and a little more round-about way if getting what you're looking for, but it carries benefits. And I've recently found a compelling benefit that previously had just sort of tagged along for the ride without explicitly being known to me.

I started thinking about it a couple days ago when I answered a question on Stack Overflow. While answering the question I have to admit that I may not have fully understood what was being asked. But now I think I do. Looking back, it seems to me now that the communications failure between the asker and myself was specifically in regard to the above dependency injection options.

The asker was most likely supplying his dependencies by means of either constructor injection or setter injection. And, since I generally stay clear of those methods, it just didn't occur to me. Instead, again, I prefer the Common Service Locator as a pattern where a single object, bootstrapped at application start, is responsible for, well, locating services. Or dependencies, as it were.

The concern about which he was asking was how to write unit tests without first knowing the implementation of the code being tested. He's seen that Test Driven Development is preferred throughout the industry but couldn't reconcile how one can effectively write tests without knowing some details about the implementation. Specifically with regards to dependencies. And, furthermore, as the implementation changes and the dependencies change, he asserts that the unit tests would have to change as well.

He's right. Because he's using one of my non-preferred methods of dependency injection. And therein lies the problem. Let's assume for the moment that he's using constructor injection. In the example, then, he ends up with an object like this:

public class MyMathObject
{
    public MyMathObject(MyOtherClass1 dependency)
    {
        // implementation details
    }

    public int Add(int addend, int augend)
    {
        // implementation details
    }
}

Looks clean enough, right? But there's a problem. There's a leaky abstraction. His assertion was that, prior to implementing the class, the only known footprint is this:

public class MyMathObject
{
    public int Add(int addend, int augend)
    {
        // implementation details
    }
}

And that it's the implementation details which change the footprint to the previous one with the injectable dependencies. Thus, one ends up in a cycle. Define the object, write tests, implement the object (thus changing its footprint), re-write the tests to match, make updates to the object later in the software life cycle (maybe further changing its footprint), re-write the tests again to match, and so on.

He's right, that is a vicious cycle. And it's all stemming from that leaky abstraction. The problem isn't in the tests or in Test Driven Development at all. The problem is in a flaw in the design of the object. That constructor is making implementation details externally visible. It's not only allowing but requiring that all external neighbors to it know about its implementation details on some level in order to supply those dependencies. And, of course, as those details and dependencies change then all neighbors must also change.

This is tight coupling. If a change to Object A requires that Object B must also change then Object A and Object B are coupled around that feature. In this case those changes are internal implementation details, and so this coupling is of course wrong.

And this is precisely that subtle reasoning I've had behind my preferred method of dependency injection which has never explicitly occurred to me before. With something like Common Service Locator (or, more commonly for me, a home-grown implementation of the same pattern), the dependency injection within the object instead looks more like this:

public class MyMathObject
{
    public int Add(int addend, int augend)
    {
        var dependency = ServiceLocatorObject.Resolve<widgetdependency>();
        // more implementation details
    }
}

Externally to the object there is no knowledge of the dependency. The coupling is eliminated. So things like tests (or any other neighbors to this object anywhere in the system) don't need to know anything about its implementation details, just as it should be.

Appy New Year

Be sure that I haven't forgotten about this blog. Be sure also that I've been inordinately busy lately. This seems to have been a theme in my life in 2012... Take on as many things as possible. Or, more colloquially... FILL ALL THE TIME! This blog, much like many of my reading/writing activities, has simply been pushed down the list of priorities. This list ebbs and flows, really. But just to give you an idea of what's currently on the list, in no particular order:

• A big project at work becoming increasingly stressful as it pushes toward a release date
• A pro-bono side project where I get a chance to practice various skills and get in on the ground floor on a business opportunity
• Websites and small projects that allow me to keep tinkering and learning
• Windows Store apps
• Increased family commitments
• A relatively new hobby that takes up a little time and a lot of money (Revolutionary War re-enacting)

I'm indeed quite behind on my reading, though I continue to build my backlog of books at the standard rate. (One new tech book every two months.) Someday the tide will swing back to reading a lot and I'll catch up, I'm sure.

Of these items on this admittedly incomplete list, the one I'd like to bring attention to at the moment is:

• Windows Store apps

As you're probably well aware, most of my development takes place on the .NET platform. I'm a fan of C#, I guess you could say. Though fanaticism isn't really my thing, nor would devotion to a language or particular tool seem in any way healthy in my opinion. Perhaps it's just a series of coincidences throughout my career, but it is what it is. And if you've kept up with the churnings at Microsoft recently then you're aware of this new RT thing.

Microsoft is changing its platforms. Probably not as fast as they want to, since they still have to support everything else they have. But it is a bold new era for them, or at least they're trying their best to make that be the case. And so they have this new Windows Store development paradigm. You've seen those Windows 8 commercials, right? Touch-friendly interfaces, "fast and fluid", and so on.

Well, recently I became possessed of a Surface RT tablet. ("Recently" as in maybe two weeks ago.) I like it. Sure, it's no iPad, not yet anyway. But it's something. It fills a space in the tech gadget world. And, more importantly, it provides me with a platform on which I can test Windows Store apps as they are intended. Until now I've had only my work laptop and VMs on my personal laptop, which respectively provide less and less of the intended Windows 8 experience.

I have a couple of simple ideas for apps. Nothing major, though I hope that one of them can evolve into something major. I've always had a desire to write these on one platform or another, but that ebbing and flowing list never seemed to have enough space on it for adopting an entirely different platform just for some tinker-level apps.

Then came Windows Store apps. They're mobile apps like any other app store out there, but they're a hell of a lot closer to the .NET space than anything else. Hell, I can even write them in JavaScript, which is something I thoroughly enjoy practicing already. This lowered the barrier to entry significantly. Sure, I'd still like to target other platforms, but now I can get in some early practice and cut my teeth on a more familiar platform.

So as a proof of concept for myself, I've written my first Windows Store app. Indeed, my first app on any app store. It's nothing special, not terribly complicated in any way. The whole thing weighs in at about 200 lines of code (not counting the dressing that Visual Studio puts in by default) and 19KB when uploaded to the store.

All it does is turn your device into a photo frame. (Which I use at work when it's idle, basically like a screen saver that's also family photos on my desk.) What makes it a little more unique is the added feature that you can enter the network address of an Apple TV device (which I have at home and thoroughly enjoy) and the slideshow of photos will be pushed there as well.

Again, it's all terribly simple and I can't be humble enough about it. But I'm proud of it nonetheless. I'm proud of it because it's my first. It's something. It's a thing that I created and published. And that makes it special to me. I'm well into my next app, which is a bit more ambitious. But this one provided me with the momentum.

So I guess in some software geek sense, I'm "published" now? It doesn't feel like the term fits. But, again, it's something. And it's something I'm thoroughly enjoying. Here's hoping my next (more ambitious) app is a huge success.