The past, present, and possible future of software architecture

First published to my blog.

It is important to understand the ways in which software may be conjured into the world for the simple reason that software and information technologies more generally have immense social and economic impact. The genesis of such process has been called software architecture, and there have been various attempts over the years to define the term and corresponding activities precisely, specifically in relation to the elements, forms, rationale, and constraints involved (“What Is Your Definition of Software Architecture?,” 2017).

A well-defined understanding of software architecture is critical to its practice and perhaps the word choice alone has helped sustain a certain nature of software architectural practice akin to that undertaken for buildings. The definitional approach taken by Perry and Wolf (1992) is typical of the traditional genre, drawing parallels with the precursors of hardware and network architecture, and their forerunner, the architecture of the built environment.

Samuel Butler (1912) observed:

Analogy points in this direction, and though analogy is often misleading, it is the least misleading thing we have.

To what degree are we misled by the architectural analogy? And might we find a better analogy, that is one that’s less misleading?

Buildings are made from atoms. Their purpose may be captured up front as a set of requirements that will most likely correspond in good part to the requirements upon completion. They are comparatively inflexible for the simple reason that rearranging atoms entails significant investment of energy. They may be simple or complicated, but never complex (a building cannot have unpredictable moments of emergent behaviour other than in terms of its own software systems). And a building’s interface with its environment, utilities for example, is easily prescribed in terms that may remain unchanged for many decades.

On the other hand, software is intangible, and while it may be simple in the most trivial instances and complicated at any noteworthy scale, it is also complex. It’s ever-changing, even during initial assembly, because the context in which it is operationalised is fast- and ever-changing. Interfacing with its own kind adds additional complication and potential for complexity, and it is I feel this potent complexity that informs more recent attempts to define software architecture.

Making a distinction between traditional (1990s) and modern (21st Century) definitions, the Software Engineering Institute gravitates towards the definition by Bass et al (2003):

The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.

This emphasis of the interrelationships of structural elements appeals. The reference to visibility may be better phrased perhaps in terms of the ease with which other software architects and systems may sense structures — their properties and dynamics — for potential interconnection and extension. At this point one must return to the start of the definition to re-encounter “program” and “system”, singular, and question such singularity. Such delineation is not then a function of the software system per se but of the sociotechnical system in which software architects (of both concious / deliberate and unconcious / emergent kind) operate. As Conway noted (1968):

any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.

Modern software architecture is self-bounding by a desire to understand “how the system will behave” (“Software Architecture,” 2017) and the context in which the architecting happens. The latter exhibits complexity while the former is intent on avoiding it. This juxtaposition is, in my opinion, the germ of the next evolution in conceptualising software architecture — a zooming out to consider both the software and the people involved in ways that treat the two more as a unity. In the organisational context, this is the domain of enterprise architecture (EA).

EA is about aligning an enterprise’s IT assets (through strategy, design, and management) to effectively execute the business strategy and various operations using the proper IT capabilities. … it aims to reduce IT costs through technology reuse and eliminating duplicate functionality. (Lapalme, 2012)

As I have noted previously, the EA process is considered the glue between business and IT. Unfortunately, EA in IT terms has not kept abreast of EA in non-IT terms. Enterprise IT architecting is mechanistic, a servant to the business strategy and, as Lapalme notes, it fails too often to consider IT part of the holistic and dynamic whole. In other words, it falls short by defining and pursuing a plan rather than striving to support and drive continuous change.

Service oriented architecture (SOA) has emerged in response, defined by Microsoft broadly as “a loosely-coupled architecture designed to meet the business needs of the organization.” (“Service Oriented Architecture (SOA),” 2017) SOA is a philosophical approach to building systems from autonomous services, and yet it has been plagued by the weakness of its acclaimed strength.

The SOA Manifesto (2009) describes SOA’s primary ambition as seeking to secure intrinsic interoperability over custom integration. And yet, no matter the technological instantiation, services are too frequently reusable and not especially useful, or useful and not especially reusable. Just as pertinently, SOA is fundamentally a top-down approach by nature, at odds with the bottom-up emergent response required for organisational agility (Morgenthal, 2009).

But all is not lost. Adrian Cockcroft at Netflix is amongst the pioneers of “fine grained SOA” known as microservices, in which the granularity is married to more rigorous decoupling (Fowler and Lewis, 2014). Fowler and Lewis describe it as:

an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms.

We have then journeyed from so-called monolithic code in the pre-architecture times, through the static schemata of 90s software architecting and the dynamic structural components of “modern” software architecture, to the mass profusion of smaller and smaller structures combining and recombining as required.

The current challenge with microservices is the corresponding absence of standardised, semantic context. Each and every service has its own flavour, for want of an agreed turn of phrase, leaving it useful only in the company of other services with similar provenance.

Jansen and Bosch (2005) make a related observation:

… almost all the knowledge and information about the design decisions the architecture is based on are implicitly embedded in the architecture.

They argue for making architectural design decisions an explicit part of a software architecture.

We appear to have arrived at a situation then in which one can have architecture with traditional efficiency and predictability but limited adaptability, or an architecture designed for agility albeit with some considerable and never-ending risk of securing the behaviours required of the whole. And so McKinsey’s greenfield IT strategy (Forrest and Sprague, 2013) and Gartner’s bimodal model the following year (“What is Bimodal IT?,” n.d.) simply propose running the two in parallel, marrying the qualities of each to the corresponding organisational requirements. For example, a retail bank’s mission critical back-end is subject to the safer architectural approach, while its foray into smartphone banking benefits from the more agile architectural approach.

I advocate this split while feeling at the same time that it must be a temporary make-do. I’m not the only one. Morgenthal (2015) aspires to a single-mode world represented by the combination of the continuous delivery and lean IT methodologies. The first is dedicated to operational flow, and the second to the elimination of waste in the software architecture’s ability to respond to stakeholders’ needs.

A new analogy is now emerging and guiding developments. More accurately, it encompasses multiple related analogues — biomimetics. A word coined by Otto Schmitt in the 1950s, biomimetics is the systematic approach to design inspired by biological processes and nature more broadly (Bhushan, 2009).

The most advanced manifestation of EA for example is known as enterprise ecological adaptation. Similarly, Martin Fowler describes evolutionary architecture in terms of its never-ending responsiveness to the environment in which the system operates (foreword to Ford et al., 2017).

Societies, cities, organisations, any and all collectivities, are situated unavoidably in ever-changing contexts, inter-nesting and interpenetrating. These sociotechnical organisms enact deliberate (intended) and emergent strategy, and are dedicated in part to upgrading their adaptive facilities. Darwin would recognise the picture.

It is not the strongest of the species, not the most intelligent that survives. It is the one that is most adaptable to change. (Leon C. Megginson as quoted in Garson, 2014)

Currently, the informational content within human ‘architecture’ — our non-digital and digital communication and interactions — are bridged by the architectural processes described here informing software design processes that in turn govern data flows and combination in pursuit of information (patterned data made intelligible and useful) and knowledge (patterned information). It’s been noted in the organisational context that “strategy is a pattern in a stream of decisions or actions” (Mintzberg and McHugh 1985) and “the pattern that connects”, the obsession of Gregory Bateson, is both the consequence of and input to this ceaseless dynamic.

The processes of architecture and the processes that evolve to resign the “architecture” label to history will likely be informed by a variety of theories and models relating to sense-making, meaning-making, designing, and decision-making in the context of complexity. These may include complex adaptive systems (CAS), Cynefin, high reliability organisations (HRO), systems thinking including the viable systems model (VSM) and systems dynamics, and open systems theory (OST) (James Lapalme et al. 2016). In the context of what Lapalme et al refer to as new realities, they also reference the human-tech ladder, critical systems heuristics, and biocomplexity. Their application in approaching the sociotechnology of identity will be explored in future research.

An evolutionary architecture consists of incremental change, fitness functions, and appropriate coupling. Fitness functions are borrowed from the field of evolutionary computing — “Architects define a fitness function to explain what better is and to help measure when the goal is met.” This makes the difference between a merely reactive architecture and one that also responds to direction; both emergent and deliberate.

The zenith of software architecture may then be its very demise. Nowak (2006) observes cooperation manifest in pre-life chemistry as well as multi-cellular organisms and human society. Mutation, selection, and cooperation can be considered as three fundamental principles of evolution (Taylor and Nowak 2009). Might the last software architect instantiate this in data?

References

1. Bass, L., Clements, P., Kazman, R., 2003. Software Architecture in Practice. Addison-Wesley Professional.
2. Bhushan, B., 2009. Biomimetics: lessons from nature–an overview. Philos. Trans. R. Soc. Math. Phys. Eng. Sci. 367, 1445–1486. https://doi.org/10.1098/rsta.2009.0011
3. Butler, S., 1912. The Note-Books of Samuel Butler. A. C. Fifield, London.
4. Conway, M.E., 1968. How Do Committees Invent? Datamation.
5. Ford, N., Parsons, R., Kua, P., 2017. Building Evolutionary Architectures: Support Constant Change. O’Reilly Media, Inc.
6. Forrest, W., Sprague, K., 2013. Invest in technology for innovation, not productivity | McKinsey [WWW Document]. URL https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/invest-in-technology-for-innovation-not-productivity (accessed 6.14.19).
7. Fowler, M., Lewis, J., 2014. Microservices [WWW Document]. martinfowler.com. URL https://martinfowler.com/articles/microservices.html (accessed 6.14.19).
8. Garson, 2014. It Is Not the Strongest of the Species that Survives But the Most Adaptable [WWW Document]. Quote Investig. URL https://quoteinvestigator.com/2014/05/04/adapt/ (accessed 6.14.19).
9. Jansen, A., Bosch, J., 2005. Software Architecture as a Set of Architectural Design Decisions, in: 5th Working IEEE/IFIP Conference on Software Architecture (WICSA’05). Presented at the 5th Working IEEE/IFIP Conference on Software Architecture (WICSA’05), IEEE, Pittsburgh, PA, USA, pp. 109–120. https://doi.org/10.1109/WICSA.2005.61
10. Lapalme, J., 2012. Three Schools of Thought on Enterprise Architecture. IT Prof. 14, 37–43. https://doi.org/10.1109/MITP.2011.109
11. Mintzberg, H., McHugh, A., 1985. Strategy Formation in an Adhocracy. Adm. Sci. Q. 30, 160. https://doi.org/10.2307/2393104
12. Morgenthal, J., 2015. DevOps and Bi-Modal IT [WWW Document]. DevOps.com. URL https://devops.com/devops-bi-modal/ (accessed 6.14.19).
13. Morgenthal, J., 2009. The Reason SOA Isn’t Delivering Sustainable Software. Tech Evang. URL http://jpmorgenthal.com/2009/06/19/the-reason-soa-isnt-delivering-sustainable-software/ (accessed 6.14.19).
14. Nowak, M.A., 2006. Five Rules for the Evolution of Cooperation. Science 314, 1560–1563. https://doi.org/10.1126/science.1133755
15. Perry, D.E., Wolf, A.L., 1992. Foundations for the study of software architecture. ACM SIGSOFT Softw. Eng. Notes 17, 40–52. https://doi.org/10.1145/141874.141884
16. Service Oriented Architecture (SOA) [WWW Document], 2017. . Microsoft Dev. Netw. URL http://web.archive.org/web/20170707052149/https://msdn.microsoft.com/en-us/library/bb833022.aspx (accessed 6.14.19).
17. Sheldrake, P., 2017. Value flows when data flows meaningfully through sociotechnical networks — in search of the ideal data architecture. Philip Sheldrake. URL https://www.philipsheldrake.com/2017/09/value-flows-data-flows-meaningfully-sociotechnical-networks-search-ideal-data-architecture/ (accessed 6.14.19).
18. SOA Manifesto [WWW Document], 2009. . SOA Manif. URL http://www.soa-manifesto.org/ (accessed 6.14.19).
19. Software Architecture [WWW Document], 2017. . Softw. Eng. Inst. URL https://www.sei.cmu.edu/research-capabilities/all-work/display.cfm?customel_datapageid_4050=21328 (accessed 6.13.19).
20. What is Bimodal IT? [WWW Document], n.d. . Gartner. URL https://www.gartner.com/it-glossary/bimodal (accessed 6.14.19).
21. What Is Your Definition of Software Architecture?, 2017. https://www.sei.cmu.edu/research-capabilities/all-work/display.cfm?customel_datapageid_4050=21328

Image by Zhen Hu.