Config Files in Python, with a splash of dynamic programming

I got some nice feedback from a recent post about scientific programming, and the thing that seemed to resonate most with the good folks out there, was the suggestion of using config files.

I want to go into some details of this in python only because I think it's a pretty neat way of getting into dynamic programming (also meta-programming).

I bring this up because, as a scientific programmer, I was fed on a stringy diet of imperative programming (Fortran, and C), and bypassed the twisted power of Lisp (the mother, or meta, of all dynamic languages). Fortunately, the guys behind Python have sneaked some lisp-like dynamic features back into python. And these features make it very easy to do config files.

Here's how I might abstract out a config file. Let's say I've written a routine to run a simulation. First, I wrap everything in a function:

def simulation():
   pass

if __name__ == '__main__':
  simulation()

Then I pull out all the parameters I want to use

def simulate(pdb_file, n_step, force_field):
   pass  

if __name__ == '__main__':
   simulate('1jbc.pdb', 10000, 'OPLS')

With Python, I can collect all these parameters together into a dictionary:

parameters = {}  # a dictionary
parameters['pdb'] = '1jbc.pdb'
parameters['n_step'] = 10000
parameters['force_field'] = 'OPLS'

def simulate(parameters):
  do something with parameters['pdb']
  and parameters['n_step'] etc. 

if __name__ == '__main__':
  simulate(parameters)

This seems like a major complication, but look what I've done. I've collected all the parameters into one thing – a dictionary. I can pass this whole dictionary to any other function, maybe

def analyze_results(parameters):
  blah

or

def display_trajectory_in_a_viewer(parameters):
  blah

This is really easy with Python's dynamic typing, because the values in the dictionary can be anything, be it a string, or an integer, or a float. And when you save the parameters to a text file, it becomes, voila, a config file:

f = open('config', 'w')
parameters_string = repr(parameters)
f.write(parameters_string)

So what does the file 'config' look like?

{ 'pdb': '1jbc.pdb', 'n_step': 10000, 'force_field': 'OPLS' }

This is a legitimate piece of python code in text form. If you want, you can pad extra spaces and carriage returns and it won't make a difference.

So now for the magic. Once you have saved your parameters in the text-file 'config', you read it back later in your program:

f = open('config', 'r')
config_string = f.read()
parameters = eval(config_string)

The parameters dictionary is as good as new.

The magic is in the function eval(). eval() turns a string of python code into pure python code. Turning code into strings, and back again is one of the cornerstones of meta-programming.

This allows you to save your parameters into a config file, and read it back again, with very little work. And now in another script, you can loop over any of the parameters as you please.