The Rise of Modern Software Architects

“If you think good architecture is expensive, try bad architecture” - Brian Foote & Joseph Yoder

In the world of software development, Forbes’s question of whether software architecture still matters has a resounding answer: absolutely yes.

Defining an architecture early, though subject to change during development, helps your team understand the impact of business and technical decisions. Serving as a clear reference, the architecture highlights the system's priorities and the constraints on application development. It offers a shared vocabulary and frame of reference, aiding teams in developing a common understanding of the application's workings and the ramifications of their decisions.

Traditionally, architects were pivotal in crafting this foundational architectural blueprint.

However, the role and significance of the Software Architect is undergoing dramatic changes, influenced by factors such as the:

• Rise of the Agile methodology and DevOps
• Prevalence of distributed systems
• Increasing reliance on SaaS, open-source software, and a plethora of new technologies that support “assembly-like” software

Today's fast-paced development cycles, characterized by high-frequency releases and quick-turnaround MVPs, demand that Software Architects remain actively involved throughout the entire lifecycle of applications. This evolution raises important questions about the nature of the role: Is it a full-time job for an individual, or one of many part-time roles played by senior engineers?

In this article, we delve into the evolving landscape of the Software Architect role, addressing common misconceptions and shedding light on its changing dynamics.

The Many Flavors of the Software Architect Role

Software architecture is such a crucial step in every phase of building a software system, that it is reflected in the many specializations that the architect role can take. Even by cursorily creating a mind map (see the below diagram), we immediately see how architects span from business-oriented, to operations and developer-oriented roles.

We can generally group them into three main categories:

(1) Enterprise Architect

“[…] Software is only one concern of the enterprise architecture. How the software is used by humans to perform business processes and the standards that determine the computational environment are two other common concerns addressed by enterprise architecture.” - Software Architecture in Practice

The scope of the Enterprise Architect (EA) extends to business and management. EAs work with top-level management to understand the organization’s business goals and determine the business, information, and technology principles governing the company’s products and services.

Enterprise Architects typically perform tasks such as:

• Mapping out existing technology solutions in the organization.
• Identifying systems for consolidation or optimization.
• Enforcing technology standards, choices, and frameworks across projects.
• Conducting market research on IT solutions and trends.
• Producing architectural artifacts (e.g., business capability models, target state architectures).

As we’ll see, the necessity of this role is the most debated, and has yielded many articles claiming that “enterprise architecture is dead”.

(2) Solutions Architect

This role is often associated with Architects who provide vendor-oriented or program-specific solutions — it’s most commonly found within software companies providing bespoke deployment strategies for their products to address complex business challenges.

Predominantly a customer-facing role, it demands not only a deep understanding of both business analysis and technical architecture but also a heightened focus on stakeholder interaction.

The Solutions Architect must be capable of advising stakeholders, closely collaborating with clients' technical teams (working within the framework laid down by the Enterprise Architects), and translating intricate business needs into solutions that achieve desired outcomes.

Often, their responsibilities include estimating the costs of a solution, coding prototypes, and monitoring proposed solutions along with accompanying issues (e.g., risks, time constraints, scope, resources).

(3) Domain Architect

Domain Architects are specialists with in-depth knowledge within a particular domain. They possess a set of skills required for a ‘niche’ area, allowing them to identify technologies and frameworks best suited to meet technical objectives in a way that aligns with business goals.

Under this definition, System Architect and Software Architect might be considered “domain architects” given that they respectively specialize in system design and software design. However, the term “Software Architect” is often used as an umbrella term for any software engineer responsible for high-level design choices related to overall system structure and behavior.

In practice, Domain Architect roles are often covered by senior / skilled developers or Subject Matter Experts (SME), who have the skills to design the technical solution related to that specific domain.

Regardless of the specific category and specialization, every architect is responsible for:

• Identifying and Understanding Business Requirements: They focus on producing and continuously evaluating the architecture for alignment with an organization’s business and mission goals. This ensures that the technological solutions directly contribute to achieving business objectives.
• Performing Trade-off Assessments: Architects must weigh the pros and cons of architectural decisions, such as choosing a specific technology or balancing flexibility and complexity. This evaluation helps in optimizing the architecture for both current and future needs.
• Setting the Technology Vision: Architects are tasked with choosing the IT solutions and setting the technology vision for the project. This includes imbuing 'evolvability' into the project's make-up, allowing them to anticipate and integrate emerging technologies effectively.
• Producing Architectural Documentation: This involves creating detailed documentation that outlines the architectural plans, decisions, and standards. This documentation serves as a reference point for the development team and stakeholders, ensuring consistency and clarity in the project's technical direction.

When performed correctly, these practices enable predictable product quality, fewer downstream problems, time and cost savings in integration and testing, and cost-effective system evolution.

Anti-Patterns in Software Architect Roles

Software architecture, while integral to the successful development and maintenance of software systems, is not immune to pitfalls and misinterpretations. Here are five challenges and misconceptions that can hinder effective software architecture:

(1) Associating the activity exclusively with a specific role

“Everyone on the team benefits from you becoming a better software architect - improvements to yourself help your team members become a stronger and more cohesive group and they help improve your designs on your projects.” - Keegan Williams, SEI, Carnegie Mellon University

This anti-pattern stems from the misconception that 'Software Architect' and 'Software Engineer' are separate roles with distinct activities. In truth, software architecture is a core engineering skill, and only those with substantial engineering experience can become effective software architects.

Software architecture crucially shapes how software is designed, developed, and maintained, profoundly impacting its functionality, quality, and longevity. It entails critical decision-making about system structure and component interactions, essential for achieving scalable and high-performing systems, especially in the realm of distributed software (which is most software nowadays).

In essence, possessing system design experience and considering architectural requirements when coding makes you a better engineer and results in a better product.

Most agile teams, particularly in startups and scale-ups, “instinctively” engage in system design and software architecture in their day-to-day activities. When faced with complex problems offering multiple solutions, they research options, discuss them with the team, and then write a proposal to gather feedback and identify potential issues. If this process of creating and circulating a "design document" sounds familiar, it's because it is often a responsibility attributed to Architects.

A dedicated full-time software architect becomes necessary only when a project attains a certain level of complexity and scale, requiring the coordination of multiple agile teams on a larger system.

Conversely, years of engineering experience are essential to becoming a competent software architect. Architects need to be able to contribute to the code - this experience provides a deep understanding of software development principles, practices, and challenges. It's akin to why successful engineering managers periodically engage in hands-on work: to maintain their technical skills and credibility gained from "time in the trenches.”

(2) Unbundling decision-making authority from the responsibility for results

“My core principle here is simple: only the people responsible for building software systems get to make decisions about how those systems get built. I can opine all I want on your architecture or ours, but if I’m not carrying a pager for you, you should probably just smile politely and move along.” - Charity Majors, Co-founder & CTO at Honeycomb

When architectural thinking is seamlessly integrated into the engineering process, the entire team collectively shoulders the responsibility for the final output. This encompasses all phases: designing, developing, maintaining, and operating a service. Technical decisions are made collaboratively, considering not only the optimal solutions to meet functional and non-functional requirements but also the team's existing technical knowledge, skills, and capacity.

However, when those who make technical decisions are not accountable for their implementation, a crucial disconnect emerges — they have "no skin in the game.” The lack of ownership in the final outcome (and its maintenance) often leads to “design by buzzword”, where choices are made based on trendiness rather than suitability. It can result in the selection of technologies unfamiliar to the team or the endorsement of sub-optimal solutions that reflect a superficial understanding of the problem, or worse, a disregard for modern engineering practices and the practicalities of implementation.

Furthermore, this separation of decision-making authority from responsibility also disempowers engineering teams. It places them in a position where they are compelled to follow directives they may not agree with or fully comprehend, stifling innovation and team morale.

(3) Perception of the Architect as an unquestionable authority

“If the software architect is perceived as this high-up oracle that you can never question, then you can’t get the team work that you need to be able to build systems that are going to be meet the business needs” - “What Makes a Good Software Architect” - SEI, Carnegie Mellon University

Perceiving the software architect as a distant, authoritative figure immune to questioning or challenges leads to diminished collaboration and limited team input in architectural decisions.

This issue is often exacerbated by significant disparities in pay and the perceived prestigious roles that many organizations maintain. Architects are often seen as high-level thinkers, too busy and important for the hands-on tasks of coding, debugging, and reviewing pull requests (PRs).

However, this approach is ultimately detrimental to both the architects and the development teams. It creates a figure lacking credibility, expertise, or moral authority to make informed decisions for the team. Simultaneously, it fosters a team environment where members may feel uncomfortable or ill-equipped to implement and maintain the final architecture effectively.

(4) Overemphasis on standardization in large organizations

Every company inevitably requires a degree of standardization and coordination. In large organizations, "architecture" tasks are often consolidated within specific roles, with the Enterprise Architecture group taking a leading position.

This approach is logical, considering the time needed to coordinate multiple stakeholders, interface with the business, and advocate for large-scale projects (such as refactors or replacements of legacy code) in enterprises. It's also apparent when considering the high overhead and complexity involved in documenting an enterprise system architecture, maintaining up-to-date static diagrams and documentation, and recording all architectural decisions.

The benefits of these activities are clear: businesses recognize the immense value in having a shared understanding of the system architecture and a clear vision of how to anticipate and integrate emerging technologies effectively.

However, over time, the Enterprise Architecture practice has devolved into a gatekeeper of innovation, hampering organizations’ flexibility and responsiveness to change. The focus on stardardization of processes, enforcement of technological rules, heavy upfront planning and the bureaucracy of excessive or overly detailed architectural documentation creates a disconnect that does not align with the reality of continuous transformation.

These processes often persist long after their usefulness has waned, in a self-reinforcing cycle, because specific jobs and roles depend on their existence.

(5) Neglecting architecture processes in agile environments

When the Agile Manifesto was introduced in 2001, it stated that one of the 12 principles of agile software is that “the best architectures, requirements and designs emerge from self-organizing teams”. This principle led to some confusion among teams, with a few interpreting agility as an excuse to entirely bypass architectural planning. This misconception has often resulted in the accumulation of technical debt and unsustainable development practices.

Today, most teams acknowledge the critical importance of software architecture and its significant impact on the final quality of the product. However, the execution of architectural practices is frequently haphazard, marked by inconsistent communication among team members, ad-hoc inputs to design decisions, and inadequate documentation of design assumptions and alternatives.

Commonly, design decisions and system architecture information are:

• Casually scribbled on whiteboards or in notebooks, lacking a structured format.
• Discussed in meetings, with insights limited to those in attendance.
• Communicated through emails or Slack messages, which are prone to getting lost or overlooked over time.

This scattered approach is partly due to the plethora of tools and resources that developers need to manage when working on distributed systems.

Multiplayer aims to address this issue by offering a unified platform where all system architecture actions and decisions can be made collaboratively. Our platform seeks to eliminate the need for context and tool-switching and the resulting loss of time, streamlining the system design process in agile environments.

The Future for Software Architects

With the advent of agile methodologies, the role of Software Architects has significantly evolved. Moving away from an authoritarian, decision-making stance that dictates architecture, they are now embracing a more consultative and engaged approach.

Today's effective Software Architects function as consultants, educators, and supporters, collaborating closely with the engineering team to co-create architectural decisions. In this new paradigm, architects no longer unilaterally make and enforce technical decisions for engineers to implement.

Instead, the engineering team maintains the authority to make final decisions, as they are the ones who will directly experience the consequences of these choices.

For developers, this shift means recognizing that software architecture is a core engineering requirement, significantly enhancing their ability to perform effectively.

This shift towards a collaborative and supportive dynamic signals a promising future for both software architects and engineers, fostering an environment where shared expertise and influence collectively elevate the entire development process.

---

Multiplayer is a collaborative developer platform for system design and architecture documentation. Visualize, design, develop, and manage your distributed software from a 10,000 foot view down to the individual components and dependencies. Sign up for free to see for yourself!