Economies of Scale
Economies of scale arise when unit costs (that is, costs per single unit) fall as output increases. This can happen for many reasons, some obvious examples are: greater productivity through specialisation/division of labour, bulk purchasing/transportation, cheaper financing (lower interest rates due to lower perceived risk) and increased application of technology (i.e. investing in more advanced machinery with higher output).
The core concept of economies of scale is that the marginal cost of producing an additional unit is less than the average cost of production of all other units. Economies of scale provide a natural competitive advantage that can protect an incumbent from new-comers, and are the original realisation of the virtuous cycle/‘flywheel’.
On the flip side, it’s also possible to have diseconomies of scale, where the marginal cost of production is more than the average cost of production.
People
Applying this to the world of people, economies of scale means that the marginal output per person increases as the number of people increases, and diseconomies of scale means that the marginal output per person decreases as the number of people increase. In principle, it’s entirely possible for the marginal output per person to be negative, that is, adding an additional person causes the output of the entire team to shrink, rather than to increase.
To take a concrete example, imagine a world where the first person produces 100 ‘units’ of output, and the second person produces 90 additional units of output and so on. The average output per person as the team grows will look like:
And the overall output of the team will look like:
Note that even though the average output per person constantly declines, the overall output of the organisation still grows up until a certain point, and then it begins a precipitous decline. Note that, if you assume that every person has the same salary, costs will still grow linearly with the number of people even as output diminishes.
What causes the marginal output per person to decline?
If each additional person adds work onto every one of the existing team members, then adding each additional person will necessarily reduce the output of all other team members due to the capacity that this work consumes.
What causes this additional work?
Communication - communication is expensive.
As a team grows, the number of lines of communication within a team grows super-linearly, with O(n2). Even if the communication overhead per person is small, the net effect can be devastating as group size increases.
Software Engineering
Designing software involves creating a mental model of what the software will look like. Designing software in a group of people involves creating a shared mental model of what the software will look like. This inescapably involves communication. Documentation doesn’t avoid this problem, documentation is just a means of of communication, and can be cheaper or less cheap than other means of communication depending largely on the preferences of those involved. This design work doesn’t stop when code starts being written, as programming is itself a design activity[1] and thus still carries some level of communication overhead.
Fred Brooks quote that “What 1 programmer can do in 1 month, 2 programmers can do in 2 months”[2] is not a statement about programmer laziness, it’s an insight that:
Collaborating whilst building software requires the communication of large quantities of information.
Inter-person communication is incredibly low bandwidth.
Creating a requirement that information be shared adds substantial overhead (it quadruples the work involved in this case, from 1 engineer-month to 4 engineer-months).
It should be clear from this examination that software engineering suffers from diseconomies of scale. More broadly,
all design work suffers from diseconomies of scale
. The more people that you have to convince of your ideas, the slower you will be, and the less you will produce on an individual basis.
That design work suffers from diseconomies of scale is evident in all other industries, ranging from automotive design to marketing campaigns to legal arguments. In all of these industries, design work is typically performed by either individuals or small teams in order to maximise output.
How can we scale engineering work?
In order to continue to make economically viable progress as an organisation grows, the problem space of the organisation must be decomposed into sub-problems that can be worked on by relatively independent teams.
Amazon Example
Up-to-date counts are hard to come by, but the most recent estimates are that ‘Amazon.com, Inc.’ employs approximately 70,000 engineers. So, how do they avoid diseconomies of scale?
In 2002, Jeff Bezos issued a mandate to all Amazon development teams that read[3]:
All teams will henceforth expose their data and functionality through service interfaces.
Teams must communicate with each other through these interfaces.
There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter.
All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
Anyone who doesn’t do this will be fired.
Thank you; have a nice day!
Jeff is also oft-cited for his “2 pizza team” rule that:
Every internal team should be small enough that it can be fed with two pizzas.
This set of edicts are a precise reaction to the diseconomies of scale of Software Engineering, and is a way of continuing to make progress at larger scales without being trapped by terrible economics. These rules intentionally limit both the burden of intra-team communication and the burden of inter-team communication.
Implications for System Architecture
The optimal system architecture intimately depends on the number of people that will be involved in building and maintaining the system. Monolithic architectures offer a lot of benefits to small teams due to their low cost of cross-modular change, however as an engineering organisation grows past 100 engineers, monolithic architectures can become unviable. Whilst it is impossible to know the exact future size of an engineering organisation, knowing the direction of travel can be crucial to making good architectural decisions.
Summary
Economies of scale arise when unit costs fall as output increases.
Diseconomies of scale arise when unit costs rise as output increases.
Communication overhead can cause diseconomies of scale.
Software engineering, and indeed all design work, requires large amounts of communication and so suffers from diseconomies of scale.
In order to escape diseconomies of scale, the problem space of an organisation must be decomposed into relatively independent teams.
The optimal system architecture depends on the number of engineers that will work on a system (amongst other factors).
References
Jack W Revees (1992). Fall '92 issue of C++ Journal.
What is Software Design?
Fred Brooks (1975). The Mythical Man-Month. Addison-Wesley. ISBN 0-201-00650-2.
CIO.com (2017).
What you can learn from Amazon
.