The strange case of AWS Proton: Conway's Law-as-a-Service?

Is that a good thing? It's ... different, anyhow

AWS Proton launched in preview last week to great fanfare and even greater confusion.

"Automated management for container and serverless deployments," says AWS marketing. So this is a tool specifically for deploying serverless apps, like Stackery or Amplify? Nope! I think the reason they're advertising Proton as a solution for "containers and serverless" is that the two provided sample template bundles are for a Lambda CRUD app and a load-balanced Fargate service. But while building your own templates appears to be frustrating, there's no technical limit on the sorts of apps Proton could help you deploy.

Proton itself is "serverless" only in the way CloudFormation is serverless: it's a managed service. So is Proton a new and improved CloudFormation? No, it uses CloudFormation templates all marked up with Jinja, but it operates at a higher level than that.

It took a while to click for me, but Proton is not just another half-baked AWS developer tool. It's doing something I don't believe AWS has ever attempted before.

Proton's innovation is that it builds explicit assumptions about your organization into its technical design.

Proton is Conway's Law-as-a-Service.

Interested in upscaling or beginning your journey with DevOps? A Cloud Guru's AWS DevOps Learning Paths offers custom courses fit for beginners and advanced gurus!

I've spoken a lot over the years about the phenomenon (and, increasingly, the pathologies) of the central cloud team: the Terraform nerds at the center of a cloud-first org who create tools and standards for everybody else, particularly product developers. In your company, this team might be called the "platform team", the "infrastructure team", or just the "cloud team". It's a remarkably consistent pattern across larger orgs, even if it's ultimately sub-optimal.

In fact, the "one platform team supporting many product dev teams" pattern is so prevalent that AWS, in true customer-obsession mode, has now reasoned backwards from it to design Proton. Proton is a prescriptive way for platform teams to package up environments and infrastructure services, then toss them over the wall to development teams who will consume them. Proton draws the line between automation of infrastructure and development of application code exactly where your average enterprise does.

This means that if you do NOT have a central platform team and multiple dev teams -- say, if you are a 10-person startup -- Proton is not for you, at least not now, even if you have many Containers and Serverless Apps to deploy. Not for the reason many AWS services will not be for you -- because they solve a technical problem you don't have at a cost you can't afford -- but because Proton solves the app deployment problem in a way that is incompatible with your organizational structure.

I really can't think of another AWS service that is built this way. Even services like CodePipeline that ship AWS's own organizational philosophy -- I'm thinking of its reluctance to support branch-based workflows -- do so implicitly and with workarounds. Proton is made for enterprise-shaped orgs. Period.

And that's not a criticism! AWS absolutely should be releasing services designed with specific customer workflows in mind. Heck, I've personally implemented at least three bespoke versions of AWS Proton over the years, and so have many, many other people who've worked on central cloud teams. The hard part was always the separation of duties between what the central team should do and what the development teams should do, and how to get buy-in on that line of separation, wherever you end up drawing it. I even wrote up a janky Service Catalog-y solution in this 2019 piece, which reads in retrospect like a cry for help.

Proton is prescriptive in the sense that it gives you a pre-baked pattern for coordinating application delivery, but descriptive in the sense that this pattern wasn't dreamed up as an arbitrary "best practice" - it's simply a canonization of what AWS sees customers already doing. In this way, it reminds me more of the reference architectures we often see open-sourced by AWS solutions architects and partners, rather than a typical top-level service.

Proton's been released in preview and is not fully baked yet, so I won't offer specific technical nitpicks here, but I'd like to address a criticism that I've already heard quite a bit from central cloud teams: "Proton doesn't simplify the process of creating deployment infrastructure nearly enough to be helpful!"

I justified my "pragmatic CI/CD design" last year by pointing out that, while it involved complex templating, that complexity was distributed mostly where you'd want it: on the shoulders of the platform team. (Their job is YAML! They can deal with it!) Proton has made the same calculation, and drawn the lines of separation mostly in the same places. They are giving raw, somewhat labor-intensive controls to platform teams in order to present a streamlined deployment workflow to app dev teams, with all that nasty infrastructure abstracted away. (Maybe that's a second reason Proton could be called "serverless?" No server automation for dev teams?)

Far from being too complicated, I worry that Proton may in fact be oversimplified. Because at the enterprise level, it's well-nigh impossible to create deployment templates that are really general enough to work out of the box for a diverse portfolio of app teams. Sure, you'll have a set of "happy path" teams that your templates support quite nicely. But then there are the edge cases, the teams that need services you don't support or workflows you can't accommodate. How do you keep them from dropping off into shadow IT ... or worse, ballooning every template into an unusable mass of one million parameters?

These are teams who maybe should be building some of their own infrastructure, who should have less centralized dictation and more decentralized enablement. That's the "Umbrella" pattern of org -> CI/CD expression I called for earlier this year, in a blog that reads like lost minutes from a Proton design session. It's the pattern I'm seeing more and more forward-thinking organizations adopt - those who've run with the central cloud team long enough to realize its limits.

Central cloud teams are bottlenecks, even if 80% of organizations haven't matured enough to figure this out. And that leads to the most curious question of all: Will teams complex enough to need Proton eventually become too sophisticated to use Proton?

That remains to be seen, but I do hope that as Proton evolves, it will lead its customers toward sustainable best practices, instead of productizing the organizational mistakes of the past. Customer obsession is great, but as Andy Jassy said in last week's keynote -- sometimes you have to give them not what they ask for, but what they need.