Software development is risky. But 25 years ago building a program was like gambling at a casino.

In 1996, 31 percent of US development projects failed entirely;

53 percent were over budget, schedule, or missing functionality;

and only 16 percent met initial expectations.

The problem was in the way IT projects were planned.

In the 90s, software development usually looked like this.

First, the team of user representatives gathered requirements,

development was scheduled and budgeted,

then thousands of pages of documentation was handed off to a tech team to begin design and development.

Eventually the system was integrated into existing processes and tested.

This was called the waterfall model as steps were discrete and strictly followed each other.

Today the drawbacks of this approach seem obvious. Planning and documentation could take months.

But what if you’ve made a mistake and the software solves non-existent problems?

This would then be revealed only at the end of the cycle.

And estimating new features that need deep research was essentially guesswork.

An additional headache became evident in enterprises and complex projects.

How to translate business requirements to technicians?

Project managers and analysts that worked on analysis stages spoke a language completely different from the one engineers spoke.

Programmers like solving tech puzzles, not business problems.

This made the gap between planning and coding even deeper.

There was a need for someone who’d take the technology leadership,

have a high-level view on the problem,

and suggest a solution expressed in a language accessible to technicians.

In 1999, engineers from Rational Software led by Philippe Kruchten

suggested building software in short iterations,

reevaluating assumptions and results at the end of each development phase.

The technique was called the Rational Unified Process.

This partly solved the first problem related to long planning and a feedback cycle.

However, today most development teams use other iterative and agile methods like

Scrum, Lean, and Kanban.

But Rational, by bridging the gaps between requirements and technology,

helped address the second problem - communication difference.

Later, this practice was dubbed solution architecture, and the role of the person leading it - solution architect.

The term architecture was borrowed from the construction industry to highlight the complexity of problems.

These were enterprise systems after all.

The term also depicts a vantage point: Architects oversee the big picture, without diving into low-level technical details.

Today, we know three main roles related to architecture: Enterprise, Solution and Software architects

First, there were enterprise architects and software architects.

Solution architects weren’t initially recognized either in enterprise architecture frameworks like TOGAF or in software development ones like Rational.

But sometime in the mid-2000s,

the role emerged as a middle ground between a wide enterprise perspective and a narrow tech perspective.

Ultimately, the roles differ in how high their vantage points are.

Enterprise architects consider the big picture and oversee the entire corporate tech ecosystem from the business perspective.

Solution architects work with specific products and define how they answer the business questions with technology.

Software architects specify the structure and behavior of a system described by solution architecture from a technical perspective only.

Any software development starts with gathering requirements from a diverse group of people:

stakeholders, user experience specialists, or even end users.

Business analysts are in charge. They document requirements in user stories,

short descriptions of product behavior from a customer perspective. These are functional requirements.

There are also non-functional ones -

the general qualities of software, its speed, maintainability, security, and some others.

Besides requirements, any project has its own constraints:

a budget, manpower, time, licensing, and risks.

A business analyst provides a solution architect with all these inputs.

The architect is responsible for converting them into a high-level technical guide.

But there’s an additional level of complexity. Usually, a product never exists alone.

It must fit into the current enterprise architecture, other systems a company has been using before.

For instance, if you use an old legacy monster, your shiny new admin panel must be able to tame it and communicate with it.

A solution architect considers existing enterprise architecture to make sure that a new product fits well in the ecosystem.

If you look at the Amazon Web Services certification Exam Guide for solution architects,

you’ll get the idea of the problem domains that this person faces:

Designing resilient architectures Defining performant architectures

Specifying secure architectures, and so on

Too vague, right?

For example, an architect building an AWS-based system

may need to decide which Amazon cloud service to use and how

if the client’s salespeople upload their sales figures daily.

They need a durable storage solution for these documents

that also protects against users accidentally deleting important documents.

The selection of specific technologies isn’t limited to cloud services only.

An architect decides: Technology stack, Databases, Architectural patterns, Standards and more

But where’s the architecture? What is that exactly?

Construction architects have concepts and plans. Solution architects have diagrams.

Remember Rational Software? In 1995, they suggested Unified Modeling Language, or UML.

UML is a language for visualizing how software works.

This is Hello World! in Java programming language. And this is Hello World! in UML.

UML suggested three main building blocks:

Things - boxes, circles, and other shapes to show functional objects

Relationships - lines and arrows to connect objects and explain how they interact

And diagrams - combinations of these to capture the big picture

Using this simple logic, you can describe very complex systems with modules, services, databases, flows...

Or even movies.

Here’s a recent Netflix Bandersnatch.

In 1997, Object Management Group standardized UML as a general-purpose modeling language.

It also became ISO standard in 2005. Now the language is widely used to describe architectures.

UML isn’t the only way to describe architectures.

With the growth of cloud services, their providers also suggest their individual visualization methods.

And that’s how business needs are first translated into users stories

and then into the format that engineers can work with.

“The ability to win the hearts and minds of our customers

while explaining abstract technical concepts as they relate to business value, is a must.”

This is a quote from the job posting for a solution architect in one of the tech companies.

Solution architects do lots of talking.

Diagrams look static, but making them, reiterating, and tweaking

entails a lot of communication with different parties:

Stakeholders and project managers - to capture the business problem and pitch a solution

Business analysts - to work over requirements

Enterprise architects - to fit a solution into the corporate ecosystem

Engineering team and software architects - to break down tech details

many people, besides that one guy that’s always at the coffee machine.

But who’s qualified for this kind of job?

Usually, solution architects have engineering backgrounds and a pedigree of complex projects.

Framing intricate requirements into an architecture takes top-to-bottom understanding of a product,

from the client side - through languages, operating systems, and virtual machines - to an infrastructure.

Most likely, a solution architect would understand a domain.

There are architects who know the ins and outs of financial systems, or healthcare software, or travel, you name it.

They may also have tech domains.

Cloud providers like Amazon Web Services and Microsoft Azure

offer architect certifications to demonstrate their proficiency with these ecosystems.

Look for that if you plan to hire one.

And obviously, you want the soft skills that keep all that communication humming.

But does it work?

In 2009, experts from Capgemini and the University of Amsterdam

surveyed people from 49 software projects to understand the business value of solution architecture as a practice.

They found that the use of solution architecture helps decrease budget overrun by about 19 percent,

improve customer satisfaction, and find a better technical fit for project results.

This doesn’t mean, however, that all software projects need solution architects.

If you build landing pages, integrate small modules, or make minor iterative improvements, most likely you won’t need dedicated solution architecture.

As one of the building blocks of enterprise planning, solution architecture remains in the realm of complex projects.