AI will Transform System Architecture Design and Management

I first become passionate about neural networks after reading David E. Rumelhart and James L. Mcclelland’s book “Parallel Distributed Processing.” I was obsessed with the idea that you can get computers to learn to do interesting things, not just perform pre-programmed tasks. However, computing power, memory, and storage weren’t really sufficient to solve many interesting problems in the late ‘80s.

Over the next several years I dove into designing, building, and managing large distributed systems but I couldn’t stay away from trying to find uses for neural nets - in fact, I even co-authored the paper ”Radial Basis Functions for Process Control,” where we explored using Gaussian mixture models rather than the sigmoidal activation functions in neural networks.

Skip ahead a few years and we finally have all the pieces in place for AI to revolutionize how we work. In particular, through the lens of my experience, I can clearly draw a line between the latest technological advances in AI and how we will evolve in designing and managing system architectures.

AI is a Powerful Tool

As with any major technological advancement in history, change is scary and it can be met with fear and doubt.

I find it completely understandable that there is so much hype and so many conflicted opinions about AI, especially after the March open letter urging a halt to the development of the next generation of AI technology and the May 22-word statement equating AI to the same threat levels presented by “pandemics and nuclear war”.

It also doesn’t help that you see articles with titles like “42% of CEOs say AI could destroy humanity in five to ten years”.

The truth is that AI is a very powerful tool, and as with any tool, it can be used for good or bad purposes.

While I do believe that it’s important to raise awareness of the near-term potential dangers due to bias, discrimination, hallucinations, disinformation, etc., and the potential negative consequences that the lack of regulation can bring (especially when AI is used by bad agents); I have several points of skepticism around the likelihood of the creation of a non-aligned hyper-intelligent computerized agent that will pose an existential threat to humankind.

In fact, I’m mostly excited and hopeful about these advancements in AI: beyond it being an accelerator for scientific discovery, it’s a tool that we can use in our everyday life to make us more productive and creative.

AI Assistants are Already Making Us Better Developers

It’s clear that AI will continue to radically change software engineering, but it’s an evolution, not an extinction. It’s not surprising that this distinction is apparent to many developers, who, as a group, are intrinsically predisposed to being adaptable, trying new technologies, and finding new ways to build more easily.

AI and automation have been a part of the developer workflow for a while now (e.g. from ML-powered security checks to CI/CD pipelines).

This SlashData May 2023 survey found that 63% of developers engaged in some aspect of AI-assisted development, and the latest Stack Overflow Developer Survey shows that 70% of all respondents are using - or are planning to use - AI tools in their development process this year.

The benefits are immediately visible: it lowers the barrier to entry for beginner programmers, makes experts more productive, and accelerates the low-code movement.

Thanks to the assistance of AI, developers will be able to automate tedious tasks like code reviews, testing, and debugging, and have more time to focus on more meaningful and innovative work.

Those who claim that eventually developers will be replaced with human-prompt engineers underestimate the role itself: a developer is a highly skilled knowledge worker, that engages in a variety of complex and creative tasks beyond just writing code (we previously discussed developer productivity).

That means that developers will be critical in reviewing the output that AI produces to avoid unnecessary complexity, technical debt, and the confusion that can result from a ballooning pile of code.

I believe we’ll come to value, even more, the unique abilities that only a highly skilled developer can bring to the table - similar to how the onset of the fast food industry didn’t put Michelin-star restaurants out of business, but only let us appreciate their chefs’ creations even more.

Problems with the Current System Architecture Design & Management

Even the most talented and experienced developers, come across these challenges when they are designing a system architecture or managing a large distributed system:

• Rapidly changing technologies - More and more projects are introduced every year to help you run the components involved in distributed software. It’s difficult to keep up, learn all these new technologies and best practices, and assess their effectiveness. The high pace of technological innovation is just one side of the coin, the continuous maintenance and upgrades needed are another.
• Identifying successful patterns - There are many ways to solve a problem, and while there is no single ‘right way’, there are patterns that have been proven to be more successful than others. Over time, developers accumulate this knowledge about the architecture patterns that work (or don’t work) for specific use cases. However, this requires years of experience, learning, trials, mentorship, and mistakes.
• Higher complexity of systems - Even the simplest application will likely require a distributed system and knowledge of multiple tools and technologies. Designing, deploying and managing an application as a single developer (or with a very small team), is becoming harder and harder.
• Getting early decisions right - There are hundreds of little decisions (e.g. authorization, authentication, database type, schema, caching, etc.) that you need to make, that will end up being important in the future (we previously discussed how to build on a solid foundation from the start).
• Creating architecture documentation - A well-documented architecture can be the difference between a project that succeeds and one that fails. However, it requires a significant and ongoing investment in time and effort.
• Learning how to leverage the services in a platform - There are more and more AI-powered chatbots to help you understand how to use a platform as a service and integrate with it, but there is still a need to manually go through the documentation, choose the correct approach, and customize it to your needs.
• Troubleshooting - Nowadays APM tools excel at providing graphs, log messages, charts, etc. that you can analyze to see what went wrong. However, you still need to learn where to look, what to monitor, and how to read this information.

Of course, the ‘painfulness’ of these points depends on your experience level and the breadth of your knowledge, but how much easier would your job be if AI could help?

How AI Will Help Solve System Architecture Pains

As we established, AI assistants are already making us better developers. So it stands to reason that AI will be able to significantly impact the pain points in designing and managing a system architecture.

Just imagine, for example, AI that:

• Recommends successful architecture patterns for specific use cases. You have an expert assistant, up to date on the latest technologies and best practices, that based on your use case (e.g. “I’m looking to develop a consumer app that allows mobile payments…”) can generate a full platform for you to review and tweak.
• Gets you up and running with intelligent automation. You no longer need to have extensive knowledge of multiple tools and technologies. For example, you can skip hours of studying YouTube videos of how to set up Kubernetes, and instead only need to click a few buttons.
• Automatically creates and updates all the documentation (architecture docs included) in a real-time, dynamic, accurate, contextually-aware, interactive, and accessible way.
• Provides a natural language interaction with your APIs. For example, you could ask a chatbot to “generate a sample JS file to integrate search into my application”.
• Allows for self-diagnosing and healing systems: Rather than going through charts and log messages, you could just ask “Why is my platform slow today?”. AI could distill the knowledge across all APM providers, create a holistic view of the situation and give you the response in a natural language interaction. Taking this one step further, it could automatically detect and diagnose system faults or performance bottlenecks and then initiate corrective actions, such as self-reconfiguration or adaptive resource allocation.

There are many more uses I can think of for AI in the context of designing and managing distributed systems (e.g. intelligent resource optimization, cognitive security, etc.) and the key will be how well they resolve the real-world pain points developers experience today.

---

Multiplayer is working on those exact solutions - to enable teams to design, develop and manage distributed software effortlessly. If that sounds good to you too, sign up for free now!