Our Python Toolset

I thought I would diverge from the ramblings for a post and talk about the toolset I have put together for our server-side python bits. I'm really happy with the way it is all coming together and wanted to brag a little bit. it also might be nice to have something that I can look back on in a few weeks when I'm cursing the toolset we are using... so here goes:

• Repo - We are using Git, which I have come to love. nuff said.
• Documentation & Unit Test - We are just using pyunit & pydocs, the batteries included are great! We have been making big efforts to fully use these tools.
• Builder - I setup SCons as a central point for all of the tools. its kinda nice because when I forget a tools name I just tell scons what I wanted to do, ie. scons test (sudo apt-get install scons)
• Dependency Checking - I wrote a custom python script that wen searching for dependencies. its not perfect by any means and will need to get better. but its enough for now.
• Interactive Shell - IPython is pretty kick-ass. if you write python and haven't tried it, you really should giver a go. I have gone to the many hours of trouble setting up a bootstrap file which I think is gonna be like heaven to play with. (sudo easy_install ipython)
• Test Manager - It's more of a test finder / runner, but it does the trick. Nose can read settings out of your setup.cfg and away it goes looking for and running test files. (sudo easy_install nose)
• Test Coverage - This is a nice little strap-on to Nose called Coverage. You use it to tell you how much of a class your unit test covers. kinda nice stuff to know. (sudo easy_install coverage)
• Static Analysis - Just a warning, this bad mother tool is gonna hurt your feelings. its called PyLint, and when you recover your self esteem you'll realize that despite being wickedly noisy it kicks ass. (sudo apt-get install pylint)
• Cyclical Complexity Testing - So I know this one sounds kinda scary, and a lot of it is covered in PyLint, but there are a couple of scripts floating around the interweb that give you an arbitrary number representing how hard your code is to maintain. They're pretty neat. (python wrapper, original tool)

Thats it for now. tomorrow's mission is to integrate Tornado and either the google protocol buffers or a json parser.

Repo (GIT) Checkout the project to giver a try. git clone gitosis@unit21.ca:eda.git Builder (SCons) - sudo apt-get install scons Running scons or scons all from the eda_server directory will run all of the tests on the code Dependency Checking (Custom) - Just python Running eda_server/build/env_check.py orscons check will test your system for the development tool dependencies. Interactive Shell (IPython) - sudo easy_install ipythonRunning ipython bootstrap.py or scons ipython will open up a ipython shell and fill it with all of the imports and a bunch of test data. more on this in a second. Test Manager (Nose) - sudo easy_install nose Running nosetests from the root of eda_server or scons test will run all of the unit tests in the project. currently the copy constructors and datastore stuff fails. Test Coverage (Nose with Coverage) - sudo easy_install coverage Running nosetests --with-coverage from the root of eda_server or scons test_coverage will show you the coverage of our unit tests. Static Analysis (PyLint) - sudo apt-get install pylint Running pylint --rcfile .pylintrc * from the the root of eda_server or scons pylint will analyze the package. currently there are a lot of very nit picky things wrong, and again problems in datastore land. Cyclical Complexity (complexity.py and complexity) - Follow instructions here: http://gdub.wordpress.com/2006/07/09/cyclomatic-complexity-for-python-code/ - Scripts live here: http://www.journyx.com/curt/complex/index.html (change sort -n [+4|+5] to sort -n -k 6) Running complexity /.py, scons complexity_full, complexity.py /.py, or scons complexity will give you a list of the files and their cyclical complexity.