I was inspired to write this blog post in response to a post I came across today on LinkedIn about sizing software projects. (link below). Sizing software projects is the thing that most everyone gets wrong, its hard and almost impossible to get an accurate estimate, why is that?
Well apart from scope creep, budget changes and all the usual business variables mentioned by Patrick in his blog post, developers and product teams will never be consistent in their output, not even if you average it out over a long time which is what SCRUM/other agile methodologies suggest when trying to establish velocity – that simply does not work, it is a fallacy. Writing software is an iterative and creative process so “how someone feels” will change output, and I am not talking about how someone feels about the project or work, I am talking about how someone feels about their cat dying, or their wife being pregnant or political changes or the weather, or the office banter or how unwell/well they feel today, in fact “life” guarantees this. So I am going to be a bit outrageous here and suggest an alternative way of thinking about this. Let us start with asking the most important question – “what is the point of estimating”? there are only two possible answers to that question….
1. You are going to undertake some work which you will charge your client for so you need to know what to charge them.
The only possible way you can give your client a fixed price for work that is essentially a creative process is by substantially over-pricing the work estimate and giving yourself lots of fat in the deal to give you the best opportunity of making a profit at the end of it. If you think that you can ask a team of developers to tell you how long it is going to take so you can make a “fair” markup you’re deluded. The best option you have in this scenario is to work backwards, you need to understand the need the client has at a high level, then you need to establish the value that your customer is getting from the thing you would deliver, then you put a price on it, you are looking for the best possible price the customer is willing to pay, you should not at this point be trying to establish “how much will it cost”, you must be asking the customer “how much are you prepared to pay”. Once you have a number, now you can work with your developers, but instead of saying “how long will it take” you are asking “can it be done in this timeframe…”, that may seem a subtle difference but it is actually huge because in answering that your developers will take “ownership” of the delivery commitment and that is what you need to stand any chance of being successful. The risk you are taking is on your team, not the project – if your team succeeds then you and your business do, if they fail so do you.
2. Your organisation wants to know how much and how long this new software thing is going to take/cost so they can “budget” and control and cost overrun.
The reason to budget is because managers and finance people (and he people that own the actual money that gets spent) generally need to *know* how much it costs in order to feel like they are doing their job. This really comes from an age where output was quantifiable (manufacturing for example), but creative IP output is much harder to quantify because there are so many variables that are outside of your control.
Think about this, you wake up one day and have a great idea to write a piece of software that will change the world; you believe it is going to make you your fortune. You are so confident that you leave your day job, set up your business and your first day you sit down and start to change the world – what is the first thing you are going to do?
I am going to bet that you will NOT crack open Excel and start to do a time estimate and a budget – Instead you will most likely start making your software. So you get on with it, now project forwards, you have created your software and you start to sell it things go well, in fact so well that you have to hire a manager or two, then you go and raise some funding to go global with your idea. Now something important happens, instead of spending your time making that thing you believe in, now the people who invested money (which may well be yourself) will want to protect that investment so they put management controls in place, and when the question “To get this thing to the next level, what do we need to do” is asked of you, and you answer “We need to build X, Y and Z” the dreaded question comes – “How long will that take?” which roughly translates to “how much will that cost”, this is because the people asking that question are in fact trying to protect cash and de-risk the investment – they don’t believe in the thing you are building in the same way that you do, the thing is just a means to an end – profit (which by the way is a good thing).
Going back to that first day, if you had tried to budget and determined it was going to take you six months before you could sell your first one, and after six months you realise you were not even half way there and you had another 12 months to go – would you stop? Well you would make sure that you could pay the bills and survive of course, but if you decide to stop, it would not be because of your budget, you would stop because with hindsight you no longer believed the idea was as good as you first thought – otherwise you would plough on regardless right?
So back to the boardroom then, and the “How long will it take”? question. Well the answer to that question should be, “I have no idea – how important is it”? Because either its important and it has to get done or its not that important.
You would be a little more committal than that but you get the idea. If you assume that an acceptable level of estimating effort was going to be 25% of the overall development effort (which has been my experience in software development) and if you have a thing that needs to get done because its strategically important for the business to flourish – then how long it takes is irrelevant, its either strategically important and it gets done, or its not. So if it “just has to get done” what on earth are you trying to estimate how long it will take for – just get on with it and use that 25% for making the thing you are trying to make – just like you did in the first six months of your enterprise.
You need to ask the same question about how important is it and what is it worth to the business, this is the question that the people trying to de-risk are not wanting you to ask, because they will find that question as difficult to answer as you will trying to answer the “how long will it take” question. Of course for trivial stuff like defects and general maintenance/tactical incremental development work this does not really apply, but for big projects that have strategic importance the “how long will it take?” question is a nonsense question to ask because any answer you get will be either fictitious or grossly over estimated, and both of these are bad.
If you want to get something strategically important created, hire a great team and empower them to get on with it – if you are making them spend their time telling you how much it will cost to develop instead of developing it then you are failing – not them. As a manager, entrepreneur, director or investor, hire software developers to do what they do best – make software, it is your job to take and manage the investment risk, if the team fail then you either hired the wrong team or you did not manage the business well enough to sustain the effort required to make it happen, asking them for an estimate is just a way of getting a stick to beat and blame them when things are not going well.
I have been managing (arguably very loosely) software development projects and a software business for the best part of 20 years, and I have learned a few things along the way. Perhaps more importantly I have been doing this largely investing my own money, so I think I know both sides of the “How long does it take” question very well.
The article I responded to
https://www.linkedin.com/pulse/software-project-estimation-broken-patrick-austin
This content is published under the Attribution-Noncommercial-Share Alike 3.0 Unported license.
There is an old rule about software project time estimation: Whatever you come up with, multiply by PI and change unit to one magnitude bigger. So far always worked.
Hi Jacek,
Indeed thats sort of the point, developers will never know how long something will take beyond some vague approximation, asking them for anything other than “their sense of scale” is pointless and a waste of their time. Thanks for the comment
Gerry
Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.
— Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid
Gerry. First off, welcome back to your blog. I missed you.
You are so dead on in many ways, especially in the way you descibe software development projects and the bulls**t games that are played.
Your story makes me laugh out loud. Hardware development is no different. I am so glad to be retired from most of it. However, as a consultant off and on, I have to re-engage just so I can overbill my client.
You are a refreshing alternative and a fun venue as compared to the crowded eevblog. It has unfortunately turned into a childrens playground where the head master keeps the childen in check with videos while he keeps the home fires burning. Sad model, but in his case, necessary. Hope you can keep yours going without comercializing it like Dave has. Nice bloke, just dont like his business model.
Best regards
Bob
Hi Bob,
Thanks for your comments. I am really over-committed at work so my blog activities have to take second place, I have not gone anywhere just struggle to find the time to create content. No plan on commercialising the blog, my day job pays the bills, I do the electronics stuff for the love of doing it.
Gerry
https://www.youtube.com/watch?v=BKorP55Aqvg
Certainly you have seen this one? If not, I think you might get a kick out of it
I laugh everytime I see it. Enjoy!
Hi Bob,
No I had not seen this before, duly posted inside my organisation, should give a few people a giggle.
Gerry
gerrysweeney,
Hi from ugly Russia 😎
Аm 50 years old. I was a developer in past. Now it’s my hobby.
Your blog very curious and professional.
It’s a shame if you do not write any more.
With hope …
Hi Alexey,
Thanks for writing, yes I know I have been very very lapse in writing/blogging, I wish I had more time but its been manic for me. I will get back to writing I promise. Thanks for reading
Gerry