I haven't written much for the past few months- for a couple reasons.

First, I've become utterly addicted to AI coding tools, and I haven't been able to pull myself away from watching my AI agents absolutely tear through my feature backlog. Suddenly nothing can escape my reach, from long-postponed major refactoring, to minor annoyances, to features that were tricky enough I wasn't sure they'd ever make sense to tackle.

Even my old nemesis, CSS, is no match for me on a dopamine-fueled, manic, AI coding rampage.

Secondly, while I'm learning an insane amount with this stuff, it feels like all my mind-blowing insights have a shelf-life of about 5 minutes. Everyone else in the world seems to be coming to all the same conclusions. The next morning, everything I write just seems painfully obvious.

I've been thinking a lot lately about the significant discrepancy I've observed in how effective different people are at using AI for problem solving. I'm certainly not the only one; there seems to be a strong consensus building in my field (software engineering) that AI tools disproportionally augment the effectiveness of experienced people (e.g. those who developed their craft before AI tools became omnipresent), and that providing AI to people early in their career can actually slow them down- and impede their growth.

This certainly fits the observations I've made personally, but raises a lot of questions about what experienced people are doing so differently with AI. I've spent some time intermittently reflecting on my own use of AI, and also paying attention to how some of my younger colleagues are using it, and I think I have a working theory.

A couple years ago, my daughter, Sadie, began a gradual transformation that eventually led to her being able to honestly describe herself as "bird obsessed". Everywhere we go, I notice her staring up at trees, rooftops, and sometimes vaguely up and into the distance. When she was younger, she used to get lost in daydreams, and I thought this was just a remnant of that, but I've come to understand this is actually a product of intense focus and fascination- with our feathered neighbors.

I'm not sure exactly where this all started, but it was most likely rooted in her interest in photography. I'm told this is common: photographers become birders, and birders become photographers- it's a pretty natural progression.

I chat with a lot of younger engineers about their future, career goals, and what kind of work makes them happy. By far, the most frequent conversation I have is about engineering management- what it involves, how to get into it, and whether it's a good fit for them.

I've also worked with people who've been thrown into management without any real guidance or preparation. This can be a really hard situation, because you absolutely need to be able to ask for help, but it can be terrifying as a new manager to admit that you're struggling.

I thought I'd spend some time summarizing the conversations I often have with prospective managers of what I think life as an effective engineering manager looks like. I'll start with the high level purpose, and then we'll get into the details of what day-to-day activities look like.

Since I've been primarily working in platform and infrastructure recently, a lot the "code" I've been writing is domain-specific configuration languages (e.g. terraform, helm charts, OTel collector config, Kyverno policies, etc). Despite so many glowing testimonies of massive wins with generative AI dev tools, the results from my first few attempts to use them were a bit underwhelming. My guess is that config code usually doesn't get open sourced, so the models just don't get a lot of good examples to train on.

This has left me hankering to do something more ambitious, and probably in a general purpose language, where I can get a better sense of what the tools can do (beyond fairly mundane auto-completion).

Then recently, I had a few days where a lot of my colleagues were off on vacation (school vacation in Massachusetts), and it occurred to me I'd been quietly stewing on a problem that might be a great candidate for my own little hackathon.

I spend quite a bit of time helping developers figure out how to make sense of telemetry data they've collected. Often this is showing them to navigate our observability tooling, how to think about visualizing data, or just tossing some saucy query tricks into the mix.

Sometimes though, I find that their telemetry is... ill-conceived. I occasionally find metrics being written like they were a point-in-time event, missing context in logs because someone was worried about adding too much cardinality, or log lines with variable and static context needlessly mashed together. Alas.

This kind of thing makes me wish I'd had a chance catch them earlier, when they were just beginning to consider instrumentation choices.

Given how often this happens, I've built up some insights on some of the most common misconceptions about the different observability signals, what makes them different from one another, and why you'd choose a specific one for a given situation.

Now that OpenTelemetry has gained such significant traction, it's starting to attract a lot of attention beyond the hardcore observability community. While most of what I read about OTel is pretty positive, it also draws its fair share of shade.

Honestly, I get it. As a big user of OTel, I've spent plenty of hours rage debugging OTTL filters in OTel collectors, desperately searching for SDK examples that actually work, or pulling my hair out trying to figure out which version of a protobuf schema changed and broke telemetry from my Swift clients. And like a lot of the haters, I also get frustrated that the overall level of complexity in the specifications leaks into the implementation details of every part of the project.

All that said, I'm incredibly thankful that OTel exists. While it's still in its awkward teenage years, it's already changing the industry dramatically. Despite the challenges that currently exist, I think OpenTelemetry is the only reasonable path forward for observability.

At SimpliSafe, we manage a pretty large system of microservices. Because we're entrusted by our customers to protect their homes and families, we take reliability of our systems pretty seriously, so observability is pretty important to us.

Like most companies today, our observability strategy is built around the "three pillars" approach: metrics, logs, and traces. Of the three, we're currently the most dissatisfied with our logging tooling, and have been working on finding a better product.

We're already a Honeycomb customer, and in our conversation with them, they ended up making a pretty interesting case that we should consider a new approach: ditch the three pillars and make traces the center of our strategy. We had up to this point been thinking very incrementally, and here was Honeycomb, coming in hot with a bold and revolutionary vision: Observability 2.0.

AWS's Graviton is a 64 bit ARM-based CPU available on EC2. Why, you ask, would one want to use Graviton-based instances when trusty old x86 instances have served us so well in the past?

Well, for one, you'ds see a roughly 30% improvement in price/performance versus instances with x86 chips. This performance difference is even more pronounced at higher utilization, because unlike x86 chips, a Graviton vCPU is an actual CPU core, NOT a hyperthread you're sharing on a core with some rando. This should allow you to scale down, increase CPU utilization more than would be safe with hyperthreads, and still handle the same load.

At my work, we've been struggling a bit over the past few years with decisions made (almost 10 years ago now) about our AWS network design. While we have a full class A private network (16,777,216 IPv4 addresses), we've managed to paint ourselves into the very sad corner of looming IP address exhaustion.

There's a few reasons:

• Our integration with cell network carriers (to support our home security systems) requires a huge chunk of our IP space
• Our decision to use a multi-account architecture in AWS, and that we chose to use a flat IP space across our accounts. This means our IP space is fragmented across accounts, regions, and availability zones, making a lot of that address space effectively unusable.

Even with all of this, we might have been fine... until we went big on Kubernetes.