The Infinitely Configurable System

It's a subject that comes up from time to time in many places where I've worked.  Business users want something they can control, and they don't want to have to rely on developers for the business.  (Are we really that scary?)  I suppose their intentions make sense on a theoretical level.  But what about in practice?  Well, what's the difference between theory and practice?  In theory, there is no difference.  In practice, there is.

So in theory the business wants something they can configure.  They don't want to have to work with the developers for every simple little change.  But how far do they want to take that?  I've seen it be as simple as having some look-up values which business users can edit.  That's fair enough.  But many times I've seen it be something so horribly complex that the theory and the practice are just too distant from one another to ever be realized.

I'm talking about The Infinitely Configurable System.  When business users want a rules engine that they can completely configure to suit their ever-changing needs.  They want to design their forms, define how those forms interact, define their fields, define the requirements for the inputs and the format for the outputs, and so on and so on.  They want a framework that allows them to design the business.  After all, they're the business people.  Defining the business is what they do.  Why would they want to involve a developer when a framework can do the job just fine?

It's a nice dream.  But do they really know what they're asking?  (We live in an age of frameworks now more than ever before, and the subject seems to be coming up for me a lot more these days.  I don't know if that correlation implies causation.)  Does the business user, who is an expert in that particular business, know what it takes to design a framework?

First of all, let's take a look at what this means to the up-front requirements of the software.  If you want to define a rules engine, you need to know up front all of the rules that the engine is going to support.  Consider an example.  Let's say this engine comes with a form designer.  Users can create input fields, label them, arrange them, and save them to the database.  That's exactly what they wanted, so that's what they got.  They use this engine to create a couple of simple forms.

Some time later, after the system is in production, they decide that they really need that form designer to support the ability to further define how fields interact with one another.  Field B should be defined as being conditionally required depending on the value of Field A.  Or Field B should be able to change its potential input values or input type based on the value of Field A.  The list goes on and on, and gets big fast.

If these were normal forms created by a developer, then the rules could be thus codified.  The models would have the new business logic added, the UI would be updated, and so on.  The forms as the business sees them can fairly quickly be adjusted to suit the new rules.  But we, the developers, aren't working on the forms.  We're working on the framework used to create the forms.  Suddenly the effort just got a lot bigger.  There are a lot more edge cases to consider; There is a lot more testing to be done; This work is going to take a lot longer.

Had the up-front requirements been complete for what the business is going to need, this wouldn't have been such a big change.  But ever-changing needs defining an ever-changing framework are difficult to implement.  Just look at the frameworks out there today.  They have a defined scope.  They know what they do; They define what they do; They make no pretense about their purpose and their limits.  But is the business willing to accept limits based on its original definition of what its software is going to do?  Up-front requirements are never complete.  The only constant is change.  But by going the route of a framework instead of a custom application, change becomes far more costly.

Now let's take a look at what this means to the subject matter experts.  As an example, let's say we're defining this software for a health services company.  They have subject matter experts who know all there is to know about health services.  Doctors, nurses, pharmacists, care managers, etc.  These subject matter experts are prepared to define what health services software needs to do.  They have a vision of how they're going to use it to help their patients, manage their business, and in general reduce their operational costs.

But what do health services experts know about software frameworks?  Remember, we're not designing an application for health services.  We're designing an application which a non-developer can use to design an application for health services.  This software isn't about helping patients, managing a business, or reducing operational costs.  This software is about providing a means for the business to define these things.  This is a framework.  It's a rules engine.  It's a logic system, not a health services system.

Are those subject matter experts prepared to define requirements for a rules engine or a logic system?  Are they experts in software design?  Because that's what the business wants.  They want something they can use to design their software.  This is going to cause a pretty significant disconnect between the business and the developers.

The developers, in this model, don't need to know anything about the business.  After all, the entire purpose behind this was that the developers should be removable from the equation, leaving the business users to maintain the application.  There's no domain specific language here.  The business users are prepared to talk about health services, but the developers are prepared to talk about rules engines.  If they're not on the same page and not talking about the same thing, don't expect to get very good requirements out of it.  Without any knowledge of rules engines, the subject matter experts are going to define very poor requirements.  Without any knowledge of health services, the developers are going to deliver a very poor product.

Then, even after all of the requirements have been defined and the design has been approved and all the work has been finished on the application... It's still not done.  Not even close.  Is the business expecting a health services application at this point?  I sure hope not.  They wanted a framework, so they're getting a framework.  Now it's their job to define their business and create their application.  That sounds like it's going to be a pretty difficult pill to swallow.  All of the money that was spent, all of the time that was taken, and when the product is finally delivered to the business it's not ready to be used.  Now it's time for the business to design their actual software.

Finally, what will be the recourse for the business when something goes wrong?  Do they submit a bug to the developers?  Not so fast, buddy.  That all depends on what is wrong with the system.  Is it a defect in the rules engine, or is it a defect in the logical rules that the business created within the engine?

If it's a defect in the framework then that's fine.  The developers will update their tests, update the software, and deliver it back to the business with the defect fixed.  Of course, now the business needs to start adjusting their use of the framework.  The defect is fixed, but their application might not be.

But what if it's a defect in the rules defined by the business?  That's not the developers' problem.  It wasn't part of the requirements and has nothing to do with the product that the developers have built.  It's the business' problem.  The subject matter experts need to analyze their logic in their rules engine and fix it.  The business users need to debug their software.  Again, that sounds like a really tough pill to swallow.

I've been in this situation enough times to know that they won't swallow that pill.  They'll push it back to the developers to figure out the root cause of the problem and fix it.  After all, the business users aren't software developers.  It's not their job to design and maintain this stuff.  Even though the original intent of the project was for the business users to be able to design and maintain this stuff.

This leaves us in a situation where tons of effort has been put into creating a user-definable framework which isn't being defined by the users.  More cost went into creating it; More cost goes into maintaining it (because it's a lot more complicated then it needs to be); And developer morale has taken a serious hit because they're caught in the middle of this nonsense.  All of this could have been avoided by just writing a custom application that does what the business needs it to do.

Don't chase that holy grail.  The infinitely configurable system is a dream.  Let the business users stick to the business processes that they know and love, and let the developers stick to writing simple and maintainable software.