001: Avoiding the "Hard Problem" Trap

True innovation isn’t just about complexity; it's about relevance. A few months ago, our team had to choose to halt a project that consumed significant time and resources. The benefits grew murky, and the path forward became unclear, making us reconsider our investment.

After giving ourselves a moment to step back, we convened to reflect. The question on everyone's mind was simple: “Really, what the hell happened here?”

We were facing a challenge post a re-org: one engineering team was split into two. This brought up a core decision about a particular module - it was large, shared, and complex.

Let's unpack the decision-making that followed.

First Instinct: Divide the Module

Our gut instinct? Let’s split the module in half. Two teams, two modules, right? It seemed logical. But then the reality of months of re-architecting, testing, and refactoring loomed large. And beyond the labor, there were concerns of maintaining consistency and increasing dependencies.

What's the Actual Problem?

Instead of diving straight into the "hard" solution, it was essential to zoom out. I wish I'd asked the team to think more about the "whys" before the "how". What would happen if we didn’t split the module? Some possible challenges:

1. Alerts: Would errors or notifications get routed to the wrong team? Misdirected alerts can delay fixes and create confusion.
2. Code Reviews: With both teams contributing to a shared module, would we end up with a barrage of misrouted code review requests? This could hamper productivity and blur team boundaries.
3. Module Navigation: Could the combined code become a navigational challenge, causing delays and potential oversights?

Reframing the Problem

The crux wasn't just about splitting a piece of code. It was about how to make two teams work efficiently, with minimal cross-over disruptions. By looking at the "right" problems, like refining the alert routing and clarifying the code review processes, we could streamline collaboration without undergoing the monumental task of splitting the module.

The truth is that every engineering team confronts its unique set of "module-like" challenges. And sometimes, pausing, reflecting over a cup of tea, and diving deeper into the "whys" can reveal simpler, more efficient solutions.

In software engineering, strategic decision-making is as important as coding. It's good to differentiate between reacting to complexities and genuinely addressing root problems.

The Pitfall: It's easy to become engrossed in intricate details, thereby missing the broader objective. I’ve been there, too, lost in the details, missing the larger picture. It's tempting to get pulled into complex problems, sidelining the real challenges.

Teams, especially those eager to grow, can mistakenly equate complexity with learning. However, the primary focus should always be on addressing the correct challenges, even as we navigate the complexities that come with growth.

I believe, engineering leaders have a role in shifting this mindset. Continuously diving into the "why" helps illuminate the "how". If we can create a culture where we question, reflect, and challenge – we push for innovation rooted in understanding.

The Disconnect

So. Why does this happen? My hypothesis is twofold.

First, there's an innate human tendency to boast about tackling "tough" challenges. It feels good. It sounds impressive.

Second, the tech industry often emphasizes solutions, sidelining the problems. We discuss code, features, and algorithms but rarely prioritize the root challenges. This solution-focused discourse clouds genuine issues, making decision-making challenging.

Think about the number of times someone has suggested implementing some "cutting-edge" tech stack without assessing if they're even needed for a project. Or, the times we’ve seen teams creating a micro-service without understanding the challenges they aim to address.

As Seth says in their post:

As leaders, it's not just about guiding teams through projects but creating a culture of introspection. Here are a few actionable steps:

• Cultivate Curiosity: Encourage team members to ask 'why' before jumping into solutions. Maybe it’s through regular brainstorming sessions or “spike-sprints” that start with problem exploration.
• Reward Problem Identification: Often, we reward solutions. How about recognizing those who identify and articulate problems clearly? Make it a part of your team's reward system.
• Continuous Learning: Promote resources, talks, and courses that emphasize problem understanding as much as problem-solving.

Bottom Line

Let's solve right, not just hard.

True innovation isn’t just about complexity; it's about relevance.

Challenge for you

In the coming week, before diving into any task or project, spend an extra 10 minutes understanding the 'why' behind it. Share your findings with your team and observe the difference in your approach. It’s a small step, but it could be the start of a journey in how you think and act as an engineer or a leader.