Vollständiges Transkript anzeigen (3.407 Wörter)
How fickle you all are. A few weeks ago, what everyone wanted to hear about was Harness Engineering. That was the expression of the moment. But now, just a few weeks later, the buzzword has changed. You want to hear about the Agentic Development Lifecycle, the ADLC, the future of the SDLC with coding agents on the scene. And let's face it, it's an extraordinary time to be in the business of writing software right now. We're facing so much change, we're facing so much hype, and a wave after wave of LLM slop. And a lot of us feel like we're turning from being professional software engineers to being professional code reviewers for a living. So, how to handle all of this change, and how to be a software engineer in this new world, and how to still have fun doing what we do? Well, I don't have all the answers, but I do have some of the answers, and I'm going to explain to you in the next few minutes what we know, and what we don't know, and where we are with the ADLC. But first, I want to show you a product that we'll be working with.
We're looking here at a product called Bench, which is short for Workbench. It's actually a suite of applications, and you may recognize some of these applications because we've built them together in the last few weeks in different videos. There is, for example, CRM. This is something which is like a sales CRM, a nod to kind of PipeDrive or Salesforce. It's been built out a bit. It looks really nice, really fancy. This was built by a combination of Opus and Fable, and you can see that it's got all the functionality that you may remember, like a drag-and-drop Kanban board style pipeline view. And there's also a personal workspace here, which is like inspired by Notion or Obsidian, and has all the usual kind of stuff you'd expect, including the ability to have a little Kanban board here, too. Rolodex, which is your personal list of things like your circles, your inner circle, close friends, wider network, distant network, with a calendar to help you organize when people's birthdays are coming up, and who you need to reach out to. And then finally, Groove is the groove box. This is Fable's version. This is something that can entertain you and plays a bunch of different tunes. And this has all been brought together into one repo that I'll show you in a second, this suite. And one of the things it can do is we can switch it all to be on light mode, in which case everything changes. We've got a CRM in light mode, the Notion clone in light mode, the Rolodex in light mode, and also the groove box looks fabulous in light mode, too. So you can go through and experiment with this. The instructions will be in the description below.
And I'm now going to put some structure around the ADLC, and then we'll put that into practice with the repo, the product that you just saw. We're going to make a change that covers that 40,000 line of code repo without knowing how it works, and see how we can do it in a disciplined way that gets good results. A manager of mine used to say that major transformations often come under three pillars: the three P's, platform changes, process changes, and people changes, in terms of roles, responsibilities, and skills. The three P's. And I actually think that applies really nicely here. But I would say that when it comes to the ADLC, the three P's actually becomes the three H's. Those are the Harness, the Handoffs, and the Humans. Harness, handoffs, and humans is how I would group the major changes associated with coding agents. And let me explain more. The Harness is referring to the agent, and in particular the scaffolding around the agent that allows it to achieve a goal. The humans is referring to you and me, to the people here that are going to be held accountable for changes that get delivered. And the handoffs is referring to the interactions between both sides. It's how the agent and the human collaborate to achieve the outcomes we need. And here's an interesting thing: when it comes to the harness, we're starting to see convergence across the industry. A lot of the bigger issues are settled. There's a common understanding of how you go about sharing things like skills and instructions across the team. But when it comes to the process, it's still at least partly up in the air. Last year, SDD was all the rage. This year, it seems to be more about specking the intent and using the repo as the memory for the project. So there's open items here. Some ideas are coming together. But when it comes to the humans, it's still largely open. There are big question marks about the roles of the humans versus the agents. It's clear that humans always need to be accountable for the work that gets delivered, but how that happens and whether you need to do a code review of every single line of code is still something that is being fiercely debated.
In just a minute, you are going to go and make a change to the entire repo, and it's going to be super easy. First, I'm just going to spend a minute on each of the three pillars, on Harness, Handoffs, and Humans. In a recent video, I define a harness and harness engineering. I now want to explain how you apply that to software engineering. So six different practices. One of them is about agreeing the shared capabilities that you will have as a team. What are the instructions, skills, the MCP servers, and put them in the repo and version them like code so that the whole team is consistent on the use of capabilities. Secondly, treat the repo like it's your living, breathing memory. Have markdown documents use progressive disclosure to make sure that you structure your information so that markdown documents refer to each other in the ways we've covered on videos, and have that be your snapshot of the state of the repo for your agents at any time. And thirdly, have standards that are enforced, that are baked into the repo. And what do I mean by that? I mean that every time you commit code, it will automatically run a suite of tests and checks that includes changing the code style and making sure that it conforms to your own style guidelines, doing type checking, linting, looking for code smells, like security holes, looking for unit test coverage, personal information, everything that matters to you and your team needs to be institutionalized in your repo so that the agent will have no choice but to run it if it wants to commit code. In addition to having your standards hardwired into your Git and CI/CD workflow, you'd also have it built into the agent's loop, into its process, so that it will always build and run checks with everything that it does, and it will keep iterating until all of the tests pass. And then security, this is a whole topic unto itself and I covered some in my videos, but this goes beyond just sandboxing, which is important, but additionally you need to look at every single API, every integration that your coding agent has, to make sure that the scope is limited and the allowed tool list is restricted so that you have a handle on what it could do. And then self-improvement and having discipline around your harness is the final piece. Every time an agent completes a project, it should then do a post-implementation review, look back at what happened, learn from what happened, and see if it can improve its own process so that it would do better next time. And similarly, we humans should do that as well. We should do a post-implementation review, see if anything went wrong during the process, and if so, look to improve the harness so that that problem or anything like that problem could never happen again. And as I mentioned in a prior video, there's a spooky way to take this a bit further, to have something called recursive self-improvement, when you also have an agent look to get better at getting better, look to improve the way it improves itself, and I'll show you that in a second too. So with this, let's go to the code and see what this looks like in practice.
So here we are looking at the code for this repo, Bench, and I'd really encourage you to clone the repo. The details are in the description, and take a look through yourself. And I'm not necessarily advocating for the specific tests that I run, but more for the way of hardwiring these tests into your harness so that they have to run, and so your agent is most positioned for success. Now take a look at Agents.MD. You can see this is a good example of what I've been talking about. It first of all lays out the structure of the repo, it gives some rules in terms of how the agent should check what it does, it gives commands that allows it to test its work, and then it gives structure of the documentation, which documents it should read to find more information. And I'd suggest it's really worth taking a quick detour to browse through these four documents to get a sense of how this is organized. Project.md, of course, lays out the purpose of the product, the architectural decisions, process.md explains the exact process the agent should follow in order to make a change and we'll look at that in a second together. And standards takes it through the coding standards it must adhere to, and controls tells it about the controls that will automatically run that will enforce those standards whether it likes it or not, so it might as well make sure it adheres to them. Okay, let's look at the process. So the process takes it through a step-by-step formula for how it should make changes according to the process that I would have agreed with the team that's working on this with me, that it needs to understand what it's doing, if it's given a spec, it needs to develop a plan, it needs to then build in increments, making sure that success criteria are met, and then test, and here's how the tests work, and then it needs to go in at the end to maintaining the test suite, to finishing up with the checks, and finally the self-improvement, and the self-improvement includes a little note that says, "Hey, you can also improve the self-improvement. Do some recursive self-improvement, too." And also take a look at the controls document. It's less important to see the exact tests that I'm running, what's more important is the way that it's baked into the harness. And also take a moment to see if you can find the skills and the hooks that are built into this project, and also the CI/CD workflow. All right, when you've done that, it's time to run some tests. And so within a terminal in VS Code, I'm going to type NPM run check to kick off the barrage of checks and tests that it's going to run. It's going to include checking the types, it's going to include a bunch of different types of static analysis, including looking for code smells, it's going to check the format, it's going to do a bunch of testing, check unit test coverage, and report back at the end. And I'll see you in a second when it completes for me, and hopefully it completes for you as well. And it ran, and I'm glad to say 482 tests pass out of 482, and the unit test code coverage was above 80%, which was the threshold, and all of the other checks passed. We have success, the test was successful, and that's no surprise because it's hardwired into the platform. It really had to be successful. And then NPM run end-to-end will now kick off a suite of end-to-end tests using Playwright that will all run now using a bunch of workers, and I will see you when that completes. We'll see if that's successful. Yes, indeed, 86 tests passed. Our harness enforcements seem to be working.
In just a minute, we're going to get a coding agent to change the whole repo to implement a change. First, let me tell you about the second H, Handoffs. There's still open debate about the best way to exchange information about business requirements with an agent. It seems that the prevailing wisdom now is that you should write a spec which gives the intent. It explains the goal of what you want to accomplish, you give constraints, you give success criteria with measures, and you say what is out of scope. And that forms an intent. That's what you put in a spec.MD document. And you can then ask your coding agent to create a plan from your spec, a plan with a series of phases where each phase has concrete success criteria and then an overall success criteria for the entire plan. You don't actually need to put your coding agent in plan mode anymore like we would have done a year ago. Nowadays, you can just say to Claude Code, "make me a plan" and that's what it will do. And you can then tell your coding agent to go ahead and execute against this plan phase by phase. It will go off and do that, and your harness will make sure that your standards are enforced at every step. And then when it completes, you raise the PR, you explain what happened, you show the evidence that the success criteria are being met, and that, of course, will only get merged in if all of the CI checks pass. And now for our change. I'm back in the repo, and check this out, I've got a file called spec.MD that I put in Docs, changes, and a new folder, Internationalization, and you'll find this in the repo. You can just use exactly what I've done here. Check this out. So I have documented my intentions in spec.MD, it's very simple. My goal is for it to add a selector to the UI that switches all the products between English and Spanish. That's what I want. I put it there, it's not a detailed business requirements, but I have included constraints, what's out of scope, and the success criteria. Let's give this to Claude. And in a terminal in VS Code, I'm going to start by creating a new branch called Add International Support, and now I'm going to go over to Claude Code. I got Claude Code running here and I'm just going to say, "follow your process with this spec document, starting with the plan." And here we go, I'm kicking that off, it's off and it's going to read all of its documentation, it's going to understand its process, it's then going to read the spec, it's going to work on a plan. It might ask me some questions, and then I'm going to tell it to go ahead and build everything. And I'm going to see you when all of that is done. And sure enough, Claude asked me a few questions, it asked about whether it should use an internationalization library or not, a couple of other things, it made a plan, it built it all, and first time all of the checks worked. It was enforced correctly, and you are looking now at a Bench, which is much like it used to be. We've got a CRM, we've got the space, the workspace, we've got the Rolodex, and we've got the groove box. Back to home. The thing about it is that there's also a new button over here. But you may notice over here is a little toggle, this is the results of about 40 minutes of Claude working, so let's press it, and when we press it the page translates itself to Spanish, let Google helpfully want to be to translate it back again. Translate to Spanish, this is the new home page in Spanish, this is the CRM with Spanish all over it, this is the workspace, the data itself is of course not changed because that's the user's data, but the navigation has been changed to Spanish. Rolodex in Spanish, the groove box largely unchanged because these tend to be general musical terms, and that is as specified in spec.MD, our fine job by Claude. And of course, my assignment to you is to try this yourself. Run this, let Claude or your agent of choice do it, check the code quality, make sure that it's kept the translations separate from the code, and make sure it's adhered to all of the standards of our repo, not that it has a choice. And finally, the punchline for this video is the third H, Humans. You and me. And the question out there is, okay, what does this mean for us? And are we basically becoming professional code reviewers? Well, there's no doubt that a human has to be held accountable for every PR. We have to take ownership for a PR that we deliver. You can't blame Claude for something that gets introduced in a PR. And that exposes an asymmetry, that it's so easy and quick to generate code with these coding agents, and it's so much more time consuming to review that code, and that is why there is this sense that we're all moving towards being basically professional code reviewers. In practice, there's a balance that needs to be struck between the level of autonomy that you're comfortable with, and the amount of review and control that needs to be in place. And that balance, where you set that dial, depends on the project. And to state the obvious, if you're working on a prototype or a personal project like Bench, like the project we've been looking at, then you can have a high level of autonomy. You would spot check, you would confirm the success criteria, you would look at the results. That would be good. But on the other extreme, if you're working on say a payments infrastructure platform, or on a product that involves healthcare data, super confidential data, or on an operating system where lots of other products will sit on top of the work you're doing, then typically you wouldn't have coding agents involved at all. You would have a human writing every line of code, or if you do have a coding agent, then you'd have a human review at every single step, every line of code gets eyes. And in practice, you're often somewhere in the middle. And so exactly where you set that dial, how much is written by coding agents and how much by humans, and how much of the code a human will scrutinize, that's something that's agreed and established by the project team and leadership set in advance, and it's something which may evolve as the project progresses. And I'm barely scratching the surface of everything involved in the ADLC, but you can come into the repo, it's linked in the description, and then you can look around at all the different aspects of the harness that you'll find there. And while you're there, you can also launch the product itself and then play around with the four applications that you've got, the CRM, the space, the Rolodex, and even the groove box. And then you can have your coding agent carry out the spec and convert the site into another language of your choosing. And I hope you enjoy doing that, and in fact I hope you enjoyed the whole video, and if so, please do like and subscribe, that's the way I know that you enjoyed these and want me to make more, and I will, and I hope to see you very soon for another video.
