Dropbox's former top engineer argues that designing for simplicity, validation, and understandability is more valuable long-term than creating intellectually complex systems. A simple system is more maintainable and its failure modes are easier to grasp, which is crucial for reliability.
Before big tech, systems research valued novel ideas. Then, companies like Google began publishing papers on their massive internal systems. This shifted the expectation towards demonstrating impact at a scale impossible for academic labs, overshadowing purely intellectual contributions.
Rushing into management without first achieving technical mastery is a career-limiting move. These managers may be good at coordination but lack the depth to guide technical strategy, evaluate their team's work, or push for excellence. This ultimately caps their influence and leadership potential.
While generally not recommended, Dropbox successfully migrated off S3 by building a storage system tailored to its unique file access patterns. This hyper-specialization, combined with using cutting-edge hardware, allowed them to achieve cost efficiencies that a general-purpose service like S3 couldn't match.
In large organizations, engineers are often incentivized to create complex systems because simplicity is mistaken for a lack of technical depth during performance reviews. This organizational flaw works directly against the principles of good, maintainable system design.
When a team's identity is tied to a system they built (e.g., "The Magic Pocket Team"), they become defenders of that system. Reorienting the team around the problem they solve (e.g., "The Storage Team") empowers them to make objective decisions, even if it means deprecating their own creation.
Go's garbage collector led to unpredictable memory usage. In Dropbox's storage system, a node OOMing would trigger a massive re-replication workload, which could cause other nodes to OOM, leading to a system-wide "congestion collapse". Rust's memory management provided the predictability needed to prevent these catastrophic failures.
After a certain point, becoming a more senior engineer isn't about writing better code. In fact, coding skill may decline. The key differentiator is the ability to zoom out and apply technical judgment to increasingly larger scopes—from a team, to a department, to the entire company's long-term strategy.
Instead of overprovisioning their physical hardware for worst-case scenarios, Dropbox built a system to temporarily offload writes to S3 when capacity got tight. This "escape hatch" allowed them to operate their own infrastructure with much higher utilization, providing the benefits of on-prem cost savings with cloud-like elasticity.
Keeping up with every new AI model or industry drama feels productive but is a form of entertainment, not education. Cowling argues it's like reading about celebrities. Real engineering skill is built by doing, not by passively consuming information about the tools or the people building them.
Simply modeling desired behavior is insufficient for leadership. A tech lead who always jumps on pages first doesn't teach ownership; they just do the work. True leadership involves explicitly transitioning from oversight to accountability, having direct conversations with team members about their responsibilities.
A PhD forces you to a point where you are the world expert on a niche problem, with no books or advisors to provide the answer. This emotionally challenging process of navigating total uncertainty teaches you to think for yourself, a critical skill that is hard to acquire otherwise.
While multi-region availability seems like a best practice, the synchronous writes required introduce significant latency (~60ms in the US) that is unacceptable for most applications. Cowling argues it's better for most companies to accept downtime when a cloud region fails than to bear the cost and complexity of active-active systems.