Monday, April 4, 2011

Source Control and Inverting the Motivation

Sorry I haven't written in a while.  I've been starting a new job (yes, again) and, you know, a transition sort of consumes a lot of time and effort.  Also, this one involves relocation.  But so far I've been unquestionably happy with the results.

Anyway, I've been working at a client site for a few weeks now as we define and document a project.  One of the subjects that needed to be addressed is one of source control.  They were previously hosting somewhere off-site, and they're looking to us for recommendations on what kind of setup they'll want as they move forward.  Not a problem for us, of course.  We specialize in helping clients.  So, for the purpose of this client, it's pretty cut and dry.

But the experience as a whole got me thinking a lot more about source control.  Sure, we all have our favorite products that we know inside and out (for the most part, anyway) and it often just boils down to Yet Another Religious War among geeks who all think their preferred product is better.  That's our culture, it's how we think.  But, thinking about it, there's a key aspect of source control that I feel is all too often overlooked.  The component of developer motivation.

Let me explain...

"Source control" is more than just a repository for code.  It's more than just a repository which tracks in minute detail any changes made to code.  The name itself implies a greater purpose.  "Control."  For it to be successful or useful by any measure, it needs to be a controlled point of access to code changes.  Not just track them, but insist on tracking them.  To this end, source control needs to motivate developers to use it.  It's something we've all touched upon before.  A source control system should facilitate what we're doing, not get in our way.  It should never create a barrier to what we're doing.  It should never motivate us not to use it.

Consider an example...
  • System 1 has a source control repository which holds the code.  There isn't really anything closely resembling continuous integration, nor is there even an automated deployment process.  Deployment to any environment, be it test or production, is done by a developer.  He compiles the code locally, sets up the configs locally, and copies the output to the server.
  • System 2 has a source control repository which holds the code.  Upon check-in (at least to the main repository), the continuous integration system compiles the code and runs its automated tests.  Once everything passes, it automatically deploys to a target environment.  Maybe not the official QA environment, maybe it's just an internal shared Dev environment.  But it's deployed somewhere and is immediately live.
Now, given these two systems, which one has source "control"?  Well, think of it from the point of view of a developer.  If you wanted to make a change and see the result of that change, each system has a shortest path to make that happen.  And any manager or professional of any kind will tell you that the shortest path is the path you should always expect people to take.

In System 1, the shortest path is to make the change, compile it, and copy it to the server.  In System 2, the shortest path is to make the change and check it in to source control.  The difference is clear.  System 2 motivates the developer to use the source control system.  System 1 inverts the motivation.  Such a setup actually provides incentive for the developer not to use source control.

Sure, the developer knows the importance of source control.  He wants to use it.  But you know how it goes... He's just going to skip it this one time because he just needs to quickly see the result of this change in the test environment.  He'll update the source control later.  Really, he will.  You would, right?

Both of these systems host the source code in a central repository controlled by the business.  Both of them provide tracking to any code changes made in the source control system.  But only one of them controls the source.  Only one of them acts as a gatekeeper.  Yes, in System 2 a developer can still circumvent the source control.  But it's a hassle.  And his changes will be over-written on the next check-in.  So why would he?  System 1 sits outside the path of development.  It's an extra step.  System 2 is the path of development.

And it happens more often than we think.  Ask yourself... How do you track changes to your database schema/procedures/views/etc.?  That's source code too, after all.  Those are a major part of the overall system and their changes absolutely must be tracked.  But even in System 2 above, how are database changes tracked?

I've seen plenty of System 2's out there which treat database changes exactly like System 1.  (Hell, I've even helped design and painstakingly maintained such a setup.  So I know about the motivation.)  It's a question I ask when I encounter a new setup at a new job or a new client site:
  • Me: "How do you track your database changes?"
  • Them: "Oh, just like our code.  We keep it in the source control repository."
  • Me: "How are the deployments handled?"
  • Them: "Simple, we run the scripts against the database."
  • Me: "Manually?"
  • Them: "Ya, why not?  Developers check in their change scripts and then run them at deployment time."
  • Me: "Doesn't that invert the motivation?"
  • Them: "Doesn't that do who with the what now?"
  • Me: "Well, the physical act of recording the change in source control is itself the extra step.  There's no motivation to actually do it.  The target database doesn't know the difference.  It's source, but it's not controlled."
It's not enough to just keep a repository of your source code.  It's not enough to track changes to that repository in minute detail.  It's not enough until you've created an environment where the shortest path from developer to server is through source control.  Not as an extra step, not as a to-do item on a deployment checklist pinned up next to the developer's desk, not as something off to the side which needs to be remembered.  That's just a source repository.  Source control needs to be in the path.  Not in the way, mind you, because that just provides the developer with motivation to go around it.  It needs to be part of the path of least resistance.


  1. I see the value of using two tools accomplish this, a CI and a CVS. I use TeamCity and SVN and it works pretty well.

  2. Honestly, that's my preferred setup as well. I'm sure it has something to do with my familiarity with it from back at BGC. But it also has something to do with the separation of concerns I get from those products. The source control, the CI server, the build scripts, etc... all distinct and discreet entities.

    At my current job we use TFS for everything and I just don't like it. The upside is that TFS is a one-stop-shop for everything. The downside is that it's not best of breed in anything. Personally (and professionally), I value the latter over the former. It'll take me a while to get used to TFS, and I'm hoping it will grow on me.

    These days SVN gets a lot of flak because distributed systems have taken center stage. Now, that flak isn't inaccurate, but for me personally I just have to ignore it. I'd _love_ to use a shiny new DVCS, but in my situation the problem is that nobody else does. Everyone at every job I go to is perfectly happy with their existing system and there's zero incentive to go with a DVCS. I've used a couple of them personally (though for some reason I just don't get the hang of Git, not sure why but it's just not smooth for me), but using a DVCS by one's self kind of defeats the purpose and renders the entire exercise moot.

  3. We are using TFS (Tub Full of Shit was recently used) and it really doesn't seem to get better. I would much rather use a series of popular top OSS tools for each of it's functions. They are probably going to be better, more mature, and more easily integrate with others.

    As for the SVN VS Git, I really like DVCS simplely because it allows me to work how I what to then push to the server. If something takes me a little longer or is a little more involved I can easily start a branch locally. It opens up a lot of workflows, but you really need a team who is willing and capable. Of course it also integrates with other major OSS tools. So if you are running SVN at work you can always give Git a shot for yourself.

    Also I find that since it keeps track what revisions are actually applied it can perform more intelligent merging.

  4. There's also the fact that it seems to be the most popular in the OSS world. At least you should be familiar it seems.