Sunday, January 14, 2007

Configuration Files and Revision Control

Do you check your configuration files into your revision control system? What is the standard practice for this? This problem keeps cropping up for me and I have tried different solutions. I don't particularly like any of them, so I thought I would throw this out and hope you tell me how you deal with this situation.

Currently I am using Ruby on Rails for a Data Analysis project. Rails has a standard directory structure which you are supposed to follow. This includes support for separate development and production databases. It seems like what I want to do is check in this entire directory structure and then have one checkout for each developer working on the project and another checkout for the production release.

Here are my problems with this. First, it feels wrong to check passwords into revision control. This just doesn't feel very secure. Second, configuration information can change without the project changing. Mary is using database A for her developing and I am using database B for my developing. If every time Mary does an update she gets my database configurations along with my code changes, either she'll be annoyed because she has to update her config file, or I will be because she didn't and she changes my instance of the database unexpectedly.

Don't check in config filesSo the obvious thing to do is to not check in the configuration files, but there are problems with this too. When Sue gets added to Mary's and my project, she can't just check out the repository and be up and running. She needs to create all of the config files that are missing. I guess this can be part of the same documentation that includes where the repository lives, but this still seems inelegant to me. A bigger issue is deployment. When DataAnalysis version 2.1 is deployed and on Saturday night the server running it has a disk crash it would be great if Dave, who's on call and not involved with the project, could restore DataAnalysis without having to call a developer (particularly me). We, of course, have tagged the project in subversion and there is documentation telling Dave how and where to check out the tagged version, but what about the config file? If he has to actually edit a config file of an inhouse developed application, he may be tempted to call me up just to make sure it is done right. It's bad enough Dave's Saturday night was disrupted, why should mine be? How can we make deployment one step? (Rule 2 of Joel's 12 steps)

OK, so I've rambled long enough. How do you deal with this scenario? Please leave a comment and let me know, I'd love to hear it.


William said...

I ran into the same ting trying to use CVS from within Netbeans. Netbeans provides fantastic support for CVS, but it only allows CVS to import the entire project: references, configurations, code, etc. This becomes a problem when some people work in a Windows environment and some in a Linux environment; Windows paths don't work in Linux and vice versa. Leaving out the Netbeans project configuration information by making the initial import manually was definitely the thing to do.

Michael Haddox-Schatz said...

Couple of comments. First, I believe that Windows will correctly handle linux style slashes, i.e. ../foo/ will work in both Linux and Windows. Of course, absolute paths look very different, but you really shouldn't check absolute paths into a repository.

Second, I have no experience with NetBeans, but in Eclipse you can choose what gets checked in and what doesn't, and you can even tell CVS or Subversion to ignore certain files so it knows to never check them in. I'd be surprised if you can't do that in NetBeans as well, but I have been told Eclipse is better so it might be true. :)

Third, absolutely. I worked with another developer who used Eclipse and I didn't, and suddenly I had all of these eclipse configuration files checked in which had nothing to do with the project, from my perspective. I wish there was a well defined, well documented process for what should get checked in and what shouldn't, and that tools like Eclipse and NetBeans would follow this process by default (with ability to override, of course). I don't want my own local settings checked into the shared repository. On the other hand if I was using a tiered repository system (which I've never used, so I might be making this up), maybe I do want to check in my local settings into my "local check-in", but then I never want to push those files into the shared repository.

FungibleResource said...

Maven and Ant include the concept of filters, a way to dynamically fill properties within files. What you can then do is check in template configuration files and then use a filter file in a user home directory, or in an environment directory, or something similar, to control the properties.

Michael Haddox-Schatz said...

I have used the maven filter approach with some success. On a file system where only the appropriate people have permission, I have a file with the specific configuration options. In the documentation for the project it tells you what options to give maven to get it use that filter file when the deployment is built. However, on my Ruby on Rails project, there is no build step. This advantage of rails takes away the place for me to put in the hook for filtering.

I guess, since RoR's config files actually do replacement at runtime I could set it up so that the config file uses an environment variable to find the "real" config file. Then you can point it at different databases with one environment variable change similar to how RoR does development vs. production deployments. This seems like a hack to me, though, but maybe it's what I should do.

ger said...

yeah.. also thought about that for a long time. Not sure about ruby or any other framework, but in general, here is a solution:
there should be a version controlled config file which contains all the config params with their default values. If someone adds some feature which needs a config, he updates this config file with the new param and its default value.
There also should be a local config file (per checkout) which isn't added to the version control at all. Here you may re-defile values for config params.
This idea could have lots of framefork-related implementations (config arrays merging, config objects subclassing, etc).
If some change/fix/etc was made by changing the config file this should be documented to let others know what happened.

Anonymous said...

Can anyone recommend the top performing Script Deployment utility for a small IT service company like mine? Does anyone use or How do they compare to these guys I found recently: N-able N-central change management
? What is your best take in cost vs performance among those three? I need a good advice please... Thanks in advance!