When you work for a startup, the job of writing code is clearer. Get shit out the door before you run out of money. There are no expectations on scalability or readability. Code doesn’t have to be art. It just has to do be good enough to get the job done. Maybe there will be a day in the future you can make this better. But, that’s not now. Ship it.
I love this clarity. It is brutally honest about the job of code and software.
Job is a little more relaxed when you work for a small-to-medium sized company. Dev team isn’t huge. 1-2 dozen hand-picked developers, who has grown up with the codebase, can juggle a few interesting framework choices. When Netflix blogs about another crazy idea, everyone can come together to make a decision on whether to adopt. Company is stable enough to spend a few sprints on updating frameworks to fit a new idea. Everyone inherently understands what is going on because the developers only hired others who knew loved Dropwizard instead of Springboot. They are all dog people. Most of them play D&D.
It is probably the best team you will ever work for.
Step into a large, multi-national enterprise… everything is on fire. It is absolute chaos. Every codebase is different. There is 4 versions of jQuery or 2 versions of React being loaded on a single webpage (unfortunate to say, true stories). You have to know about 3 programming languages to create a feature. There are at least half a dozen team backlogs to get your work lined up to release anything. Whole thing is a waste of everyone’s time.
Why is that?
Issues seem to be rooted in scale — scale of business, number of developers, and number of products and diversity of customer base.
Products that drive the business in an enterprise are complex due to their scale. They often have high amount of traffic and lots of variations to target different customer segments, geographical laws, merchandising strategies etc. Mature, scaled products have a lot of nuanced, well tested choices baked in. This creates a moat for the business. It is not possible to intimately understand all aspects of these products.
In order to support this scale of business, there are tonnes of people in various roles. I spoke about that in my other post Companies Grow. Development teams tend to focus on a specific problem that slices through one or many products. Sometimes, the problems are tiny. I still remember being completely blown away when my lunch time buddy for Google interviews said “I work on the +1 button”. +1 button was on millions of sites and required focused engineering.
Problems arise when individuals cannot solve problems without comprehending others’ problems. Having to make connections with teams that have different motivations slows things down. At code level, this is no different. It is difficult to make progress when a feature implementation comes attached to shared code.
Often, systems are designed deliberately this way by people who hoped to bring consistency and simpler decision making. A noble idea at smaller scale; terrible idea for large scale. It is challenging to get 100s or 1000s of humans to agree on how to solve unique problems in a consistent way.
Few years ago, a travel company I worked for went to war on duplication. It has taken me a long time to realise that was wrong. At scale, duplication can be good. We looked across 1000s of developers work and tried to draw clean lines over domain ideas. Then someone squinted at different pages and said “all these look the same, let’s create one object to represent it”. Millions were spent on trying to create a consistent way to describe a “travel offer” for anything. It isn’t trivial to describe a seat on a flight the same way a hotel room or car rental is described. But that wasn’t the biggest problem. With so much traffic flowing in, product managers and developers work tirelessly to optimise every aspect of the product. Everything is experimented and tweaked, working against the idea of consistency. Result is that big consolidation and deduplication efforts fall apart very quickly. Code ages faster than ever.
My learning was that we should have aimed for relevance, not consistency.
Read my book summary of Seeing like a State by James C Scott for some relevant commentary on designing neatly arranged systems.
Consistency is important. Ideas like design systems allow consistency to happen across large scale websites. Even these can’t target every future solution. Localised knowledge and understanding of domain problems cannot be ignored. That is a key ingredient for successful products. System designers should understand the entropy law (second law of thermodynamics). It is a much better idea to make it easier to do right things instead.
The entropy (disorder) of an isolated system always increases over time or remains constant in ideal reversible processes.
What is the solution then?
Create loosely coupled systems that allow for retention of autonomy locally. Let pieces grow at their own pace, in their own way. Provide mechanisms to go between pieces. If there is a winning idea, adoption occurs naturally.
Some examples:
- If you are working on a large frontend application, it doesn’t hurt to copy paste something that looks similar but hasn’t been introduced to the design system. Ask your designer to talk to design system designers about emerging patterns. Keep code separate and choose not to build your own component library.
- When you create service endpoints for experience applications, make them feature based instead of model based. That way, developers don’t have to talk to other feature owners to make changes to output.
There are opportunities to create burden within systems. These should be done deliberately for a higher purpose. Security tends to be one of these. You may want everyone to adhere to standards. Likewise, it is meaningful to enforce the use of the design system as a minimum for frontend applications.
I always liked the way Reddit organised its social network. Base set of rules setup minimum expectations for everyone to follow. There is a clear mechanism connecting different pieces (subreddits) that allow someone to consistently browse its posts. However, each subreddit sets its own rule extensions and grow at different rates. Subreddits have a level of autonomy.
Lastly, I’d remind everyone to keep system designs and implementations super simple. When there are too many people, concepts get lost. Junior developers may not understand streaming systems (and it isn’t their fault). Simple systems are easy to scale and change in the future.
Simplicity might be the hardest thing of all. Why?
In conclusion, if you are working in a large enterprise and have the opportunity to think about system design, or maybe you are looking to architect a simple feature, think about the following:
- What is the simplest solution that can be evolved in the future?
- Don’t avoid duplication as a hard principle. Avoid cross-contamination instead.
- Think about how many people someone has to talk to before changing something… in 2 years’ time.
- What is the mechanism for the product to come together?
- Is it easy to do the right thing?
- Where do you enforce your basics?
And, perhaps, most importantly… can you deliver a feature in a day?
Leave a Reply