First, the bad news: Software projects fail at a high rate. One 2008 study concluded the failure rate was nearly 60%, and a 2016 study suggested the failure rate was as high as 68%.
Now, the good news: Project success seems to be improving overall. A 2017 Pulse of the Profession report from the Project Management Institute (PMI) shows that success rates for all projects, not just for software, are finally on the rise.
There’s more good news: Most software project failures can be avoided by commonly known, but not so commonly practiced, software project management principles and practices.
I combined my research, our team’s experience, and input from dozens of IT pros to compile a list of the twelve most common reasons that software projects fail. Then I asked IT professionals to rank them by order of impact, highest to lowest, on software project success. I’ve summarized each cause for readability.
#1 Defining Requirements
Too often, gathering requirements looks more like creating a wish list than thinking through the business process, how the software fits into it, and the impact on the customer experience.
In fact, a current state vs. future state business process exercise is rarely done and, when it is, the work gets bogged down in excruciating detail. To make matters worse, the work is usually missing important people and steps.
This is what causes ‘unstable requirements’. Others get wind of the work, typically after development has started, and spend political capital (or withhold it, in the case of a customer) to get their pet features added.
As I mention in #2, #5, and #10 below, be sure to get the right people and process steps represented in your requirements-gathering phase. Then prioritize the functionality and features while keeping things as simple as possible. Use an Agile approach to define and develop a minimum viable product (MVP, or what we call at Level 12 the Roller Skate) and execute in short, focused iterations to deliver value early and often.
Click here to learn how you can identify and avoid 5 common mistakes when defining project requirements.
#2 Selecting People
Assigning the wrong people or understaffing a team at the beginning of a project, then replacing or throwing more people at the project when the timeline slips, is an age-old reason software projects fail.
Getting the right people on the team from the beginning of the project is critical. The domain knowledge, emotional intelligence, and competency of the people on your project team should be your first priority, and it should include a well-rounded representation of committed power users and stakeholders.
If your project is mission-critical or especially complex, you’ll need to staff your team with the best people. “Best” is a mighty subjective word, I know. We’ll define it further when we dig deeper in the drill-down posts to come.
If it’s not a mission-critical project or you simply don’t have the luxury of getting the “best” on your team, there are other options. Again, more to come in future posts. (Hint: It’s about leadership.)
Click here to discover simple, yet effective strategies to get the right people on your software project team.
#3 Designing the Solution
The design phase is often where the wheels fall off a project. Here’s why:
The requirements might seem simple enough, but lurking beneath the smooth surface of optimism and simplicity of the problem is the monster known as “solution complexity.” The problem can be very simple, but the solution might be very complex.
Keeping the business process and software requirements simple DOES improve the odds you’ll find a simpler solution (more in #7), but don’t be too quick to commit to your design or your estimate. The design process can be surprisingly complex, enough that it might expand the original estimate of time and money by several orders of magnitude (more in #5).
We’ll look at more things to consider in the upcoming drill-down posts on this topic, like anticipating future requirements in your current design and designing for portability, usability, and maintainability.
#4 Support by Executive Leaders
Leadership is the art, science, and skill of helping others move from where they are to a new and better place.
If the journey to the new and better includes getting hard things done, like completing a software development project successfully, it requires leadership and sponsorship by a respected executive.
The executive sponsor must have the ability to inspire the team with a vision for the future, made better by the result of the project team’s efforts.
S/he must also have the authority to remove obstacles that threaten the project’s success and, ideally, the skill and finesse to manage change, build strong relationships, and get committed participation and buy-in across the organization.
In the drill-down posts on this topic, we’ll explore how sponsors and teams can work more effectively together, whether you should develop a project charter or just use a statement of work, how to conduct a kick-off meeting, and more.
#5 Estimating Cost and Time
“The software doesn’t work, we’re behind schedule, and we’re over budget.”
These dreaded words cause acid reflux for any business leader writing checks for a software development project.
The problem here is that developing software is exploratory by nature, and estimation often happens too soon and by the wrong people. Their intentions are good, but they come to conclusions without a solid understanding of the problem and expectations (or requirements) of the software.
If the estimates are being done by upper management (or by anyone other than the users of the software and the developers who will create it), you’ll have indigestion soon after the project starts.
Invest the time up-front to bring together the right stakeholders, operators of the process, and users of the software. Get the folks with the domain knowledge together, then ferret out a true understanding and agreement on the current state and future state business process.
Once that’s done, define the requirements (as described in #1 above) for a minimum viable product (MVP) that better enables the process to perform to future state expectations. You can add other features later, and you’ll do separate estimates for those projects.
Now estimate your time and materials investment for the MVP. Get it done. Then repeat this process to add features over time.
#6 Managing the Project
If you’re using a more linear or Waterfall approach to managing your project, you’ll likely have a traditional project manager.
If you choose a more iterative Agile approach, which we use here at Level 12, then project management responsibilities might be divided among a scrum master and product owner instead. I’ll use project manager (PM) here to encapsulate these responsibilities and keep it simple.
Your PM should be very good at leveraging best practices (harvest the mistakes and lessons learned in the past and apply them in the present).
They define team roles and responsibilities and lead project planning and tracking. They perform risk management, communicate to the project’s executive sponsor and team, and promote both internal and external client involvement.
They also mentor others, develop good working relationships, and take the right actions at the right time to ensure the right things get done right.
For software development projects, it’s also important for the PM to have a thorough understanding of the software development lifecycle (SDLC), along with the tools, technologies, and methods used by the project team.
To ensure your project is a success, be sure your PM can demonstrate the chops to handle these critical areas of responsibility.
#7 Complexity of the Solution
Creating quality software is hard. Most of the work is exploratory and intellectual, some is creative, and a little is clerical.
And for every 25 percent increase in problem complexity, there is 100 percent increase in the complexity of the solution. Be sure to simplify and take the waste out of your business process before you write or enhance the software that enables it. Never paint over rust, and never write good software to automate a bad process.
Also, just because something can be done doesn’t mean it should be done. Simplifying the development process is just as important as simplifying the business process the software will enable. Developers should never over-engineer the solution just because it scratches their engineering itch.
As Einstein is often quoted, “Everything should be made as simple as possible, but no simpler.”
Make things as simple as possible while delivering value early and often.
#8 Selecting Technology and Tools
It may take years for an individual to develop an expert level of competency in a chosen technology stack and development technique.
By employing standard practices and common technology, you will stand a better chance in recruiting team members that fit together, as well as making all products developed more portable and maintainable.
So it’s critical to decide what your technology stack and development methodology will be and to stick with it—to a point.
While it’s important to know what you’re good at and play to your strengths, your problem and the best software solution might require techniques and tools outside your team’s current skill level.
Start with the problem, then make an intellectually honest assessment of your team’s skills as you consider the selection of technology, tools, and design. Have candid and crucial conversations with your executive sponsor about how to proceed.
You might decide this an opportunity to update your tech stack and for your dev team to learn and master new technologies. Or it might be better to reduce the scope and complexity further, augment your team, or hire an outside development firm. Either way, do this before the project gets underway.
#9 Writing Clean, Efficient Code
Designing and coding are different skills.
The handoff from design to coding can cause problems because it’s often not the same person doing both. You might structure your team so the same person designing also does the coding, or you might keep them separate.
Both approaches can and do work. Just remember that handoffs introduce opportunity for errors. You must be sure both individuals collaborate closely and frequently to understand what units must be coded, how they’ll fit the design, and how to catch errors quickly.
It’s important to mention competency again. Consistently clean and efficient code, with very few programming or stylistic errors, is critical.
You get what you pay for here. It’s true that our basic instinct is to rush when selecting team members, which often results in a bad hire or selection. Another mistake is to select the right programmer and rush them. Either way you’re making time to do it twice, rather than doing it right the first time.
#10 Testing and Validating the Software
Why would anyone design and build, say, an airplane and not test its parts and units before assembling and flying it for the first time? It doesn’t happen in aviation because human lives and millions, even billions, of dollars are a stake. When a plane goes down in flames, it’s all over TV and the internet.
Regardless of your development methodology, it remains true that fixing mistakes early usually costs less time and money than fixing them later.
When it comes to writing great software, test as if your life depended on it. Implement a Test-Driven Development (TDD) culture.
Be fanatical about it. Make it non-negotiable. Automate your testing and do it as you write your code. Aim for having 90 percent or more of your code covered by sound testing.
#11 Reviewing and Inspecting Code
Pobody’s nerfect (yes, that’s a joke)
Even if you test 90 percent of your code using automated tools, there will be errors: code reviews and inspections by humans are important. Testing should not replace inspections, and inspections should not replace testing. Do both.
Also, after-action reviews, code reviews, and “retrospectives” help capture valuable learning that comes from making and correcting mistakes. Be sure your development process includes an effective review process (by humans, especially users) at appropriate milestones and certainly after the project is complete.
#12 Maintaining the Software
It’s easy to overlook and underinvest in activities and processes that are related to the ongoing maintenance of your software’s code base.
For example, it’s important to document work as the code is written. This way, when you look at code months or years later, you’ll remember what you did and why.
Also, just like a lawnmower, a jet engine, or a bicycle, good design, construction (coding) and documentation will make maintenance much easier.
And like the real-life examples above, you’ll want to budget money, time, and people for scheduled maintenance. If you ignore this, there will be a compound effect of deferred maintenance called technical debt, which will add stress to your development team, your budget, and your users.
Maintenance isn’t just fixing mistakes in your software. In fact, it’s mostly adding features and enhancements to your existing software.
Be sure to design and build your software so it’s easy (or at least easier) to enhance and maintain over time.If you to avoid these mistakes and keep your project on track.
If your project has gotten off track and you need help getting it back on, or want more information on how you can keep your project from failing, drop us a line or call us at 502.495.3936.