Software Development is a Garden

On my drives to and from work every day I sometimes puzzle over the why of it all. Recently, I have been doing a lot of reading around how to create an environment for success to be more likely. I chose those words carefully because I do not believe there is a perfect recipe for success because fundamentally the conditions that exist that gave rise to success are in the past and are impossible to duplicate perfectly (by perfect I mean every single atom in exactly the same place at exactly the same time). I believe that success is the byproduct of its environment (both external and internal factors contribute to that).

While driving through the frozen fields of corn on my way home I thought of a neat analogy between software engineering environments and gardens. What came to mind was specifically a square garden. I was exposed to square gardening during my undergraduate education when me and a few friends organized a large AIDS day benefit concert to raise money for a South African AIDs clinic one of my friends had spent some time with while volunteering in Africa. In my opinion square gardening fundamentally is about arranging a small plot of land in its optimal configuration to provide a variety of crops that all help each other to grow and fight off pests.

So, how is a square garden analogous to creating an environment for programming success?

First, most square gardens start off with a general plan so you look at your plot of dirt and then measure off small grids. This is analogous to starting to create a new product, service or package. That stage where you have the general idea of the need you need to satisfy, you’re given some resources, and now you’re trying to figure out what you need to plant in order to be able to satisfy your need (providing food). Next you may being planting some seeds, you think you know what they are but maybe not. The seeds could represent people, ideas, frameworks etc. Then you provide a bunch of encouragement and tend the garden and hope that it will eventually be what you want it to be.

Tending the garden could mean a lot of things. You could shout at the plants but that doesn’t do much (though some seem to enjoy it). You could grab a ruler and measure a plants growth rate but that only informs you enough to know which plants to keep and which ones might need some help or need to be pulled. Or you could move a plant to a better spot but it can be risky. You could also pull up a plant and check its roots too though I would not recommend it. Sometimes some plants grow large and shade out other smaller plants so they cannot grow so you have to make a decision, either to trim down the large plant, move the smaller ones, or even remove the larger plant if the small ones might yield something better. Some plants require some infrastructure to grow on (another plant or some twine, beans) so you could take time to provide that care. After a period of time you may find yourself with a bountiful varied harvest but sometimes you just end up with a bunch of corn, two angry old men, a boy who can see ghosts and a lion but you make do with what you’ve got.

I may have written a paper doing a statistical analysis of changes to the relationships between the characters based on speech acts in the script of this movie. Its script may or may not be seared into my brain.

The key takeaway is that you do not actually know what the seeds are going to turn into, or how they are going to interact and the harsh reality that the amount of control you have over the end result is minimal. Sometimes no matter what my wife and I have done some plants in our gardens have failed. Sometimes I feel like some plants were amazing in spite of the best efforts of my children to destroy them. Sometimes my kids did destroy them. Software engineering is not a science, engineers and projects are all different and require different things to thrive. Also, once you’ve harvested you start all over. You may be able to reuse some of what you knew about the previous garden but time has moved on and things are different. For example, the seeds you plant (even taken from successful plants from last year) may be totally different this year. The next project is still risky and will be different. The point is that everything is cyclical and unpredictable. People joke about the accuracy of weather forecasts where we have bazillions of datapoints, decades of experience, incredibly complex modeling abilities. The same people turn around and want accurate predictions for when something will be “done” that has almost no data and no well performing algorithm for prediction.

Not everyone feels this way. An engineer once shared how Ebay at one point in time (thankfully in the past) tried to create an environment where each project was given a certain amount of ‘seats’ on a train (they called their releases trains). The process boiled down to ideas were funneled all the way down to an incredibly detailed specification and ROI analysis before the work was started. Each step required a large amount of effort and when the work (actual coding) was to be done engineers who had not estimated the work, been involved in the requirements definition, discussion or anything were then given a specific amount of train seats (three weeks/1 engineer) to produce the product. The engineer shared how that system destroyed moral, performance and the success of the company for a period of time.

I think that on paper such rigorous planning is an honest attempt for people to be ‘wise’ with their money but I feel it is a pound wise and pound foolish because you could minimize the plan/validate stage and spend those resources failing faster (all things in moderation). There is also the enormous cost to your culture and people’s moral when you take away choice. Google did a major change to their search algorithm (the Panda change) and about two weeks before they released it someone finally asked “Is this going to affect our bottom line?” (see How Google Works). They did a quick analysis and even though they predicted it would affect short term profitability (they had downgraded many many horrible ad sites that used Google’s adwords system) they stayed true to their core principle of improving users’ search results.

As a gardener or leader I’ve had the greatest success when I have guided and encouraged people to in most cases do what they thought was the most important thing at the right time while I’ve acted as a buffer between them and everyone else. I have arrived at how I work or manage others because when I’ve been thrown into a pot of soil with others I’ve had the greatest fulfillment, dedication and happiness when I’ve been respectfully guided and nourished (emotionally, fiscally etc.). Perhaps that is just me but I’ll use my human prerogative and assume most people are like me (not true). We all have our gardens to tend, either our own direct work or even the garden we ourselves are in (I tend towards the attitude of “we’re all in this together so lets get it done”). So, if you want “success” start with the seeds, tend the garden, and see what bears fruit.

Andy Hunt and Dave Thomas of The Pragmatic Programmer fame apparently also like the gardening analogy for programming. That’s what I get for googling my cool thought when I got home from work.