Lately I’ve been thinking about software development as a form of production, and the ways that different organizations manage it. Most mature IT organizations have evolved over time to managing software projects according to structured project management methodologies. More sophisticated organizations use a highly rigorous process of planning, organizing and controlling resources, resulting in elaborate Gantt charts, forecasted delivery dates, and tight sequences of pre-ordered and estimated tasks. It’s comforting. It’s logical. And completely wrong.
Why? Traditional project management (or PM) activities arose out of large scale, civil engineering projects such as dam and bridge building. Especially in the latter part of the 20th century, these projects became increasingly more complex, involving increasingly large numbers of people. A rigorous planning process was deemed essential to ensuring these expensive projects could achieve their well-defined objectives (i.e. dam a river, build a building) without running over budget.
Traditional software project management practices have evolved to mirror those used in the construction industry, with highly structured and sequential requirements gathering, design, construction and testing phases, making software development look suspiciously like a large public works project.
The table below shows how standard operations theory categorizes projects, job shops and batch processes. Projects are generally defined as “one of a kind” efforts that are characterized by a high degree of customization and the use of unique resources, processes, and equipment. Job shop processes produce customized outputs but reuse resources, processes, and equipment since the work is generally similar from job to job. Batch processes are less about “creating” output and more about “assembling” it, as batch processes rely on the reuse of resources, processes and equipment to produce larger volumes of output with greater standardization.
What’s wrong with traditional management of software development? Software development is not a “project”. As the table shows, projects are typically temporary in nature, uniquely different from each other, and produce “one of a kind” outputs. They marshal a diverse set of resources into a team that achieves a “one-off” project objective, uses project-specific tools and solves a unique set of problems. Project teams are often disbanded once the objectives have been accomplished.
Civil engineering projects may meet these characteristics, but IT “projects” rarely do. Instead, most software development generates a variety of different but similar outputs, and uses shorter processes and standardized tools (such as data integrations, software components, test suites, and GUI prototyping software). The outputs are usually somewhat “standardized”, but are also significantly customized to the meet the requirements of the particular application (such as custom adapters to third party systems, implementation-specific filtering and business rules, conformance with approved presentation styles, customized security access, etc.).
IT software projects more closely resemble “job shop” processes, where a wide variety of outputs (software components) is produced using a standard set of tools, relatively consistent organizational structure, and long term perspective. In other words, software development does not generally produce the “one of a kind” outputs that are the hallmark of project processes.
What happens when you run software development operations as “project” processes? Mismatching process types and output characteristics hurts; the inefficiency of using the wrong process type to run your operations manifests itself in the form of higher costs. Applying traditional project processes to a job shop environment, like software development, results in over-planning, too much effort spent creating uncertain forecasts of limited value, and heavy-handed control over materials and labor that stifles creativity and is ill-equipped to deal with inevitable planning variances. The traditional planning ignores human fallibility in accurately forecasting projects with intangible (and invisible) interim outputs, and unclear dependency and resource constraints.
If traditional project management isn’t the right way to run projects, what is? In a future blog post I’ll cover why you’re much better off running most IT software development using the Agile framework. The more flexible nature of this methodology is better aligned with customization of process outputs and the reduction of technical uncertainty that is required by software development.
The key takeaway here is that most IT “projects” aren’t really projects at all: they’re jobs. Smart organizations match process management styles to the type of production being managed. For software development, the appropriate techniques are NOT those of traditional project management.