Why agent observability becomes critical when running at scale
Once you start running agents at scale, secondary capabilities like observability become really critical. It's not just about whether you can get one agent to do one thing — it's about how you oversee and run an entire business with many different agents and ensure consistent quality. The complete loop matters: it's not just a great day-one agent experience, and it's not just about curating and deploying agents. You need a full observability layer and orchestration story where you can look at all your agents running all the time and see how they're doing.
Go deeper with Greg Isenberg
Ask anything about this, and Greg Isenberg answers from everything they've shared.
More to learn
See everything in Learn
Agent skills should be evergreen — continuously improved, not one-and-done
The first version of a skill is never the final version. A dry run might produce an okay first effort — maybe 50% of the way there — but the whole point is that skills should be evergreen. Every time a skill is used, it can improve: either
Managing agents is like management 101 — you can't review everything
Running an agent-first business is management 101, just applied to agents. If you're a CEO, you literally don't have time to look at every single thing every person in the company has done — and the same is true for agents. As you scale up,
MembersLLM-as-judge: automated quality scoring for agent outputs
Instead of relying on yourself as the sole judge of every agent output, you can use a separate LLM as a judge that fires off and scores outputs along the dimensions you care about. You can then see the trend line of how an agent is scoring