Broadening Our Horizon

The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination…The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.

Fred Brooks; The Mythical Man Month

We work in a unique industry. While there are real restrictions on what is computationally possible (P probably doesn’t equal NP, after all), the most complicated systems we build rarely come close to those limits. And while both memory and CPU efficiency are important, the availability and cost of cloud computing resources constantly pushes the boundaries of what was possible. More than anything else on earth, computing approaches the mythical “post-scarcity” world.

Even in that world, though, the work we do is still difficult. It turns out that building internally consistent castles out of the air is hard, precisely because they are so ephemeral. Our most important job is to manage complexity, but our systems are so complex that we can spend years working on them and still not understand everything. We are faced with a problem that no poet ever had to tackle - our creations have to work.

This conflict is why learning is so important for the software field. Things change so fast, and systems are so complex, that it is impossible to reach a state where you actually know everything. That’s why, at Pluralsight, we care more about how you learn than what you know. Anything you know is something that was useful at one point, but that doesn’t mean it is useful today. In fact, context is so important that it’s likely that you will never again work on a system that’s close enough to the one you worked on yesterday that the knowledge you have today is sufficient. Fortunately, we live in the information age, and the tools to learn are at our fingertips. In fact, the major challenge we face is that of focus, because there is far more information available than any one person can hope to peruse, let alone understand.

Given the wealth of information available to us, it’s not surprising that most people choose to specialize. After all, programming is a relatively new endeavor, with constraints that are very different from other industries. The technical challenges we face change so rapidly that even attempting to keep up can often feel overwhelming. In the face of that technical complexity, though, we sometimes forget that the human aspect of software development can be even more challenging. Fortunately, we can gain a lot of insight into how other disciplines have overcome similar challenges. In fact, those insights have lead to several of the more revolutionary ideas in our field.

Those insights are also why I advocate for every developer spending some time learning things that seem, at first glance, to not relate to the software field at all. In order to convince you of the importance of that kind of learning, I’ll first remind you of the origins of some of our most important techniques. Then, to prove that this kind of learning isn’t just something for ivory-tower types, I’ll discuss what I learned while watching a documentary about designing the Ford Mustang. It requires effort, but I’m convinced that the only way to see farther is for each of us to broaden our horizons.

Architecture (the building kind)

In short, no pattern is an isolated entity. Each pattern can exist in the world only to the extent that it is supported by other patterns: the larger patterns in which it is embedded, the patterns of the same size that surround it, and the smaller patterns which are embedded in it.

Christopher Alexander; A Pattern Language

Back in the 1970’s, a small group of architects noticed that they saw similar features in a variety of different communities. They put in the effort to understand the problems that these features were solving, and came up with a list of 253 “patterns”: combining a simple statement of the problem to be solved with a simplified solution that was designed to be adapted to local conditions. The book they published, A Pattern Language, is still one of the best-selling books on architecture, four decades after it’s release.

Twenty years later, a group of software developers who had read Alexander’s book noticed similar repetition in the code they wrote. They decided that cataloging these software patterns would provide a way to discuss these features, as well as helping developers recognize when they were dealing with a common problem. Soon, they wrote a book (Design Patterns) that would become one of the most widely read and discussed books in object-oriented programming.

Interestingly, somewhere along the way software developers forgot one of the most important lessons of “A Pattern Language”—that each community is different, and while patterns provide a useful language to discuss problems and solutions, things work best when the community is allowed to create a unique solution. Far too often, implementing the patterns in exactly the way specified in the Gang of Four book became a mark of “well architected” software. The resulting rigidity and formalism was something that Christopher Alexander was specifically working against, because he had seen the problems that it caused. If more software developers had learned from him, we might have fewer needless “singletons”.

Manufacturing

The most dangerous kind of waste is the waste we do not recognize.

Shigeo Shingo

In the 1970s and 80s, Japan sparked a revolution in manufacturing. Traditional wisdom held that manufacturing was a “fast, cheap, or good; pick two” process. Suddenly, however, Japanese goods were competing on all three fronts. Leading the charge was Toyota, whose management system, the Toyota Production System (TPS) became world famous for its positive impact on the company. Established manufacturers were left scrambling to try and keep up, and the Lean manufacturing movement began.

At first glance, there are few similarities between manufacturing, where the goal is to produce exactly the same thing over and over, and software development, where (ideally) you never write the same code twice. However, when Mary Poppendieck moved from manufacturing to software management, she found that many of the problems that had existed in manufacturing before Lean also appeared in software projects. Software was being developed without feedback loops, managers assumed that they could sacrifice quality to gain speed, and a focus on local performance rather than on value delivered through the system were nearly identical to problems that Mary had first dealt with in the manufacturing world.

Here at Pluralsight, we have also used Lean as a way to help us deliver value faster. One of the first things I did after starting here was watch Competing on the Basis of Speed. It’s a fantastic introduction to many of the whys behind the techniques that have enabled us to move fast. I highly recommend taking the hour to watch the whole thing.

Car Design

Designing a beautiful car is easy. Designing a car that the company can afford, the engineers can build…that’s damn difficult

Gale Halderman; A Faster Horse

Recently, I discovered a documentary on Netflix about the 2015 redesign of the Ford Mustang. It included interviews both with the team designing the 2015 Mustang, and some of the surviving members of the team behind the original 1965 Mustang. While watching, I was struck by how so many of the problems they described seemed to be mirrored in my own experience working at Pluralsight. True, I’ve never had to worry about how harmonic resonance from an exhaust can create a rattle in a dashboard, or how much a 10 cent increase in the bill of materials will impact the bottom line of my company. And yet, as we worked to build a distributed system, it quickly became clear that we’d have to actually build the whole system in order to observe how the pieces would interact with each other, and more than once I’ve found that seemingly small changes can have a large impact when compounded by enough traffic.

Even more interesting to me were the people problems that both teams described. The story of the 1965 Mustang team, as told here, is a story of a small team using their constraints to enhance their creativity. Coming on the heels of the disastrous Edsel, Ford was understandably concerned that another big flop would sink the company. This meant that when Henry Ford II green-lit the program, he gave it only half the usual budget. In addition, because the target market for the Mustang was the young baby boomers, price was a major concern. In order to get around these constraints, the team decided to build a car based on the existing Ford Fairlane, utilizing existing supply chains and expertise. The result was a runaway success, surpassing its one year sales goal in only three months, and selling a record-breaking 318,000 in its first model year.

I’ve often seen similar results come from having constraints at Pluralsight. The year I joined Pluralsight we were offered the opportunity to greatly increase our partnership with Microsoft, offering more courses to MSDN subscribers than ever before. By the time the deal was signed, we had a matter of weeks before the deal would be available, and our entire department at the time had fewer than 20 developers. The clarity that our constraints provided us helped us design tools in a way that let us launch on time, virtually without a hitch.

I always like the creative tension between the design studio and the engineering team. If you’re not frustrated with the design community, you probably have a boring design.

Dave Perizack; A Faster Horse

The importance of creative tensions has been one of the hardest lessons I’ve had to learn. It’s so easy to forget that there is real value from opposing viewpoints. Dave Perizack was referring to the challenge of developing a car that both looks beautiful and performs well, but similar tensions are everywhere in our industry. As I continue to learn, when either side in these important tensions stops fighting, everything gets worse. And so, I’ve found that I have two important jobs when I represent development in a discussion with other groups with conflicting priorities. The first is to advocate as best as I possibly can for the technical excellence and innovation that I bring to the table. The second is to make sure that everyone else in the room is advocating for their priorities as strongly as I am advocating for mine. Doing both at the same time is one of my most difficult responsibilities, but the results speak for themselves.

Conclusion

I hope the importance of continuing to learn is obvious to anyone who is even remotely interested in the software industry. Technical change and improvement happen at a remarkable pace these days, and without dedication to constantly learning new technologies and techniques, we will fail both ourselves and our customers. In many cases, our technical challenges are unprecedented, and so introspective learning is both understandable and important. Please don’t take this as an excuse to stop learning technical things.

And yet, some of the most complicated and difficult to solve problems that we encounter have more to do with interpersonal interactions than 1s and 0s. When facing these challenges, we can find inspiration and guidance in the experience of people in fields that bear little in common with our technical experience. I firmly believe that many of the important advances that we’ll see in the coming years will be influenced by people learning from things that, at first glance, have nothing to do with computer programming. I hope you’ll join me in spending at least some of your time trying to find them.