On today's episode of Telemetry Now, my good friend Ryan Booth is joining us again to talk about vibe coding, the latest buzzword out there in the tech world. And, of course, we'll have plenty of talk about AI.
If you haven't heard that term yet or if you have and you're not exactly sure what it's all about, I think you're gonna like this episode. My name is Philip Gervasi, and this is Telemetry Now.
Hey, Ryan. Thanks so much for joining again. It's been too long. Last time you were on I don't remember.
We were talking about AI, I'm sure. But, it's been too long. So thanks for coming back on. Today's topic is AI-ish.
I mean, it's definitely related to what's been going on in the world of AI and specifically with regard to vibe coding.
I saw a yeah. Yeah. I saw a LinkedIn post of yours just the other day, and you were talking about it. It's top of mind for a lot of people, including myself.
And when I saw that, I'm like, yeah. He's got an angle. Like, you know, there's a pretext in those words there that he's putting out there for the public to see. Let's flesh that out a little bit. So, you know, thanks for coming on, and, I hope we, yeah, we're definitely gonna have a lot of fun today with this. Yeah.
So It's always a great time to to chat with you and and bounce ideas off each other. I I really appreciate you having me on.
Yeah. Yeah. Absolutely. So let's, sorry for using the management speak. Let's level set. What does the term vibe coding mean?
You know, you could start with what it means to you because I do have a definition here on my left screen, but I wanna hear from you first.
Alright. Yeah. So, the the term vibe coding pretty much derived I don't even know where it where it originally came from. If anybody knows the exact source, I'd I'd be interested to hear.
Well, I do have it here, but you you continue. Oh, yeah.
Yeah. Okay. So basically over the past six months to a year, models have been getting better and better at developing code, and more complex code and more code across various projects, instead of just, you know, a single script or or a single function or something.
And over time, it's kind of turned into what we consider vibe coding right now, where someone just sits down with any one of the agentic tools. There's, Replit was one of the very first ones. There's Cursor.
There is Cline for VS Code. There's a number of them out there, and one of the more recent recent one that I think helped really get a lot of popularity recently is is the Claude, Claude Code.
And they they allow you to sit down and write software in a conversational manner. So you're not actually creating files and, well, you're you're not taking and copying and pasting from, like, Chat GTP or any of the other models out there and just pasting it into files and doing everything. You're actually telling the the agent to build something, and it goes out and creates all the right files. It does all the right stuff. It it creates the environment. And so essentially, you are just kind of in this conversational prompting with the agent to to write whatever you're trying to write.
And that's that's a lot of what it is with Vibe Coding. Now I I kinda sense that the word vibe comes from the notion that there's not really much planning.
There's not much structure. You're just kinda stepping through the conversation and letting it flow how it goes.
So you ask it to do something to create this API endpoint to host up blah blah blah blah blah. It goes out there and creates it, and you're like, well, this did okay. This didn't do okay. And then you turn around and you're like, hey.
I need you to chain this. Change this. I don't want you to use a body. I want you to use the the argument parameters to pass in data yada yada, whatever it is.
And then you kinda just step through this whole process until you kinda get it to a point where you're satisfied with it.
And to me, that's that's I've I've seen that and I've experienced it with everything that I've been doing over the past six months in in AI development.
There's just never been a good term in my mind, for that just kind of process of flowing like that. So that's that's to me where vibe coding came from.
Yeah. I'm with you because I've been, you know, utilizing some of those same workflows perhaps to a lesser extent for months now. But the actual term is credited to, Andrej, Karpathy, the cofounder of OpenAI, a former AI leader at Tesla as well. And that was just last month, I believe, on Twitter.
There was some, you know, tweet that he put out there in February of of just last month. Right? Using that term and, you know, going on to say things like that, it's just it's not really coding. I just see things, say things, run things, and copy paste things, and it mostly works.
Mostly being a key indicator that there's some limitations there, and, you know, you need to, you know, work with it. It's not that you're just putting some stuff in a prompt like you said, copy and paste it into your, you know, Jupyter notebook, and then boom, everything is good to go. There is a process there. I I have personally been using tools like like Cursor, you mentioned.
Certainly, the, the foundational LLMs as well. Like, you know, I do I do like chat GPT, and I've been using o one a lot, for planning.
And then for the actual, like, development of code, I'll use I'll use a different model. And what's interesting to me is that I find it beneficial to actually use different models for, like, the design phase versus, like, generating code and even evaluation phase. But, the you know, it is interesting that that, Karpathy said that straight out, that he said it's not really coding, in in the classic sense. So would you agree with that then?
Because, you know, you really are leaning on the pseudo code generation and the planning and the workflow and then letting the tool develop it.
So it it it calls out, a a big differentiator, to be aware of in the software industry. So, you know, there there is a very clear difference between software development and software engineering.
Okay. You know, development is basically taking the task at hand and writing code and outputting exactly, you know, what you're supposed to output and handing it off. Software engineering is exactly what you're doing in VibeCoding. You're you're stepping through and architecting a solution for whatever you want, and then something else is doing the software development.
And a lot of organizations and a lot of people out there, you know, both roles are mixed together, and that's fine.
You know, a lot of scenarios, that's perfectly acceptable. But, yeah, it's it's a clear difference between the two.
Okay. So then do you believe that software engineering is currently undergoing a massive shift in what that even means in the industry, in the software engineering industry as a whole?
Yeah.
I don't think it's in the middle of going through it. I think we're at the very beginning.
We are just now starting to see the larger software industry as a whole even understand what the k or even know about the capabilities and what you can and can't do right now.
Even up until two to three months ago, I was seeing where a lot of software engineers that I talked to, in the circles that I'm in, just had no idea how far you could get with with Vibe Coding, as you would say right now. But over the past four to six weeks, a lot more people understand that now. And so I think we're right at the start to see a transition. And you're what you're gonna see is a lot of the positions are this is at least what I'm thinking. You're gonna see a lot of collapse in positions.
So you don't necessarily need four layers of software engineering and management to to get from product down into developing features. You you just need one. And so, having a a clean handoff and then dropping it into a software developer who can take on the rest, or a software engineer who knows what they need to put together, why it matters to the customer, what it means to to build it in the way they need to and then ship it. And I I think that's where we're gonna we're gonna see a lot of collapse there. So the engineers that really just focus on nothing but writing code, you might wanna start expanding your horizons a little more.
Yeah. You know, I have heard for years and years and years throughout my entire career, in the networking space, but in IT in general, in the software space, this desire to focus on outcomes, aligning with the business and business outcomes. And I really feel like, for the most part, we've always just paid lip service to that across technical industries.
But I wonder, and maybe you agree here, that this it really is the focus now, isn't it? I mean, that's what you're doing through this process of quote, unquote vibe coding Yeah. And going through the outcome based, you know, workflow and then letting the tool develop the actual code, which really is just the nuts and bolts that we don't really care about as long as we get to the outcome that we want. And again, we've been we've been paying lip service of that for years and years and years, but we we get lost in the weeds.
And I really feel like that's that's different now.
Yeah. You make a really good point there.
I don't necessarily disagree with that. I I think you're right. And we've we've had to organize in in tech teams, around, you know, putting this whole workflow together in this chain from the customer all the way to shipping a product.
And we've had to put the right people in the right positions with the right skill set. And that's always kinda where it's architected around in my opinion.
Product managers really are great at seeing the overall vision, seeing what the industry wants, understanding what their customers want and translating that into workable features.
But they're always limited because they don't have the ability to sit down and write code.
Vice versa, networking or engineers in general have the ability to create and build whatever you're asking them to build, but they don't always necessarily have the full scope of what the customer wants or what the industry wants. And now one of the things that AI is doing shaking all this up is it's giving all the different people the skill set or at least a good enough skill set in the other areas to be able to press through and do what they need to do without having to involve the rest of the groups with it. Now, will that still be needed? Will will those types of people still need to be in place? Sure. Especially as you get into larger organizations. Absolutely.
Smaller shops though? Nah, probably not.
And there's there's even the the discussion going around that that gets me a little bit excited myself is the notion that, in the next coming years we're gonna see Deca corn companies, Deca corn startups come out that are just a single person.
Yeah.
I've heard of that.
And that is where I see, you know, the power of the technology coming into play as you take one person and they have their skill sets and they have their weaknesses, but this tool or these tools are gonna allow that person to run an entire business very successfully because of their help. Mhmm.
So one thing that I I believe it was Karpathy who said maybe with somebody else that, you know, vibe coding is really for, like, a weekend throwaway project. And for me personally, though I didn't call it vibe coding for many months now, more than months, I've been doing that for stuff that I'm building at home in the lab, because I'm not a developer for my day job, nor am I an engineer anymore, in that in that sense, infrastructure engineer.
And so for me, they they're they really are throwaway projects in the sense that they're experiments.
I'm forcing myself to learn a new thing.
I'm trying to see how I can, you know, do a thing with certain types of data formats, stuff like that. And, and then when I'm done, I sort of move on. Maybe I put a blog post out there, whatever.
Do you think that that is not the case? You did say that we are in the initial stages. So are we moving toward the this, this idea that Vibe Coding is not really not necessarily for throwaway projects for the weekend project, but can be for, you know, sophisticated development projects and for, enterprise deployment?
Yeah. Yes and no. So I think it has its place. That's kind of what what the whole reasoning behind that post that I gave was was to highlight this point.
You know, yes, Vibe Coding can get some some very good results working pretty quickly. And more important, it can give someone who normally couldn't build those the ability to build them. The problem is is after you build it, when you want to augment or change or add features, that lack of control on what you can and can't do really starts to show through. And you start seeing a ton of wasted time and a ton of, lost control the further along you go. And so some some good examples that I've had is, in in some of the projects that I've worked on doing nothing but Vibe coding, I can sit down and I can say, okay, go go build this feature, make the API endpoints look like this on the back end, make make the front end look like this and it produces this output.
And we step through all of that and it looks great. I like how it works but it's buggy. It has some stuff that I need to stabilize.
And so I'm I'm happy with where we're at. It's a good checkpoint, and then we move forward to fix the bugs, and the agent just blows everything up, destroys it all. Like, whatever it does, whatever made it go that way, things just go south. And it's like, okay.
This is the not this is not the right approach. Let's back off. Let's go back to a previous state. Let's fix our changes and then move forward.
The problem is is as they fix stuff, they start it starts changing Oh, yeah. How the outcome is. And you're like, woah. Woah.
Woah. I like the original outcome.
Mhmm.
And Vibe coding's like tough shit. This is what I'm giving you now. And so you lose that control.
Pet projects, proof of concepts, you know, side stuff, that's not that big of a deal because it's it's one or two people you're probably developing for, maybe probably yourself. So, you know, you can adjust to stuff like that. But when you have customers that are like, I need to see features like this. I wanna see things like this. Or you have an industry that's like, you're only gonna succeed if you apply software in this manner or do this, that, and the other. That type of control takes a different level of AI interaction than just vibe coding. And that is where the separation line is for me.
Yeah. I've been there. I've been there where, I wanted to add another data format or another I wanted to get returned in this way. So I'm I'm, you know, interacting with with the model, and and it it kinda reformats a whole section and and, you know, I try to keep whatever code modular.
And, it's not the way it spits it out generally. And so I have struggled with that where, like, alright, I wanna make this change and I got a new thing to work with and then something else doesn't work now. So I have noticed that. And for me personally, because I'm not a, a very, very experienced high level developer, you know, it's hard for me to then go through the code line by line and really understand exactly what's going on. Of course, I do. And it takes me time. And I in the prompt, I'll add things like please include, you know, context, include, comments, all these other things, and and it does.
But it does take me time to go back and see, alright, we're what is it doing here? Whereas when I write it, though it might take me significantly longer, like you said, total control. I know exactly what's going on every line and why and why I did this. And I can do it in a modular fashion as well where I know Mhmm.
Later on and that's that's one of the things that, you know, people need to think about when you're doing any kind of AI ML product in in your data engineering pipeline is, you know, what kind of data do I have and where can I be going with this data in the future? And am I hamstringing myself today with my data pipeline and engineering processes? Same thing with your code, with vibe coding. Are you creating this product where, you know, later on you can't develop, in the way that you want to?
Mhmm. So, yeah, you're you're kinda generating bottlenecks, but you are also removing some. Like for me, you know, it it makes it a lot easier because I'm not an expert developer. So it does make it a lot easier and I'm like, holy cow, I never thought of that before?
Oh, is that how you do it? But you know one thing that's interesting, and this could be just for me. So, you know, maybe this isn't the situation you've encountered. Sure.
Bugs, things like that.
The ability to understand, you know, the code because I wrote it, all that stuff. But, does it make us sidestep best practices? You know what I mean? From a software development, like a craft perspective.
Absolutely. If you want to.
You know, it's that's that's kind of the the the, double edged sword you get when you're a software engineer and you're you're writing code is, you know, you can go the other way that's easier. You can go the other way that's faster.
And essentially what you're doing is you're adding technical debt to your project. And the more technical debt you get on, the lower your velocity and the the lower, you know, of time to getting a feature deployed, or adding in stability, adding it where developers have to have a much larger skill set to be able to work with your code base.
All these types of things come into play.
Kind of the nice thing that I see with all of this is the software industry has kind of already been through this motion through its maturity.
You know, we started off within the early days where, you know, everybody would just share files and then drop them into a file share and everybody has their own version and their own folder and then at some point in time, they spend two to three weeks merging everybody's stuff together or stuff gets accidentally overwritten. All those problems developed us into using Git, and, you know, now everybody's using GitHub or Bitbucket or whatever to to store projects and that that allows us to streamline how we manage multiple people writing code. And then we get into a thing where, you know, we got scope scope creep, you got bloat, you got unwasted time and features. And so we add in the product management element of it. We we add in developing PRDs and, designs and engineering docs so we we we control the flow of it.
Adding in testing and adding in, you know, CICD pipelines to sanitize and validate the code. All of these things we have put in place to address the human elements of writing code and to control that.
We just now need to apply the same thing to AI agents.
AI agents do the same thing. They make errors, they have issues, they go sideways, whatever it is. Sometimes they throw code in that just looks like garbage.
Working it through a pipeline, going through each one of these steps, each one of these check boxes, that's that's how you transition away from vibe coding into, more of a term that I like to call of, AI driven development.
Yeah. Yeah. I'd like to talk more about, specifically the role of the agent in AI driven development at some point. I mean, I've been experimenting with some very low code, almost no code drag and drop, platforms recently that are you know, there's a part of me that's like, well, I wanna be in the code and, you know, touch all the the knobs, the nerd knobs and the stuff. But I'm like, man, this is just so easy, though. And then you can open up the pane where you see the code and you can adjust things if you like. But it it just makes it so much easier and gets me to the goal much faster.
Mhmm. You know what's interesting is, when I when I do when I give it some sort of prompt, go through the workflow, go through the process, I noticed that often I'm just getting, code generated that looks like it's just from scratch and it does not go it doesn't use, like, certain libraries that I would normally use. And, you know, being in the observability space is gonna be some, like, data analysis libraries, scikit, things like that, Matlab. You know? And and and it doesn't use a lot of that stuff. And you see all of it done in this, like, long form manner, and that's an interesting difference. And I don't know if that's a good thing or a bad thing, but I wonder if we're gonna see if this continues to grow and vibe coding, which is kind of in in its infancy, or at the c CLI level of vibe coding, right, before we get into more advanced, mechanisms to do it.
You know, I wonder if we're gonna see a decrease in some library usage because of that, because it's just being written by by a tool that can do it. And and, frankly, these models are very effective at doing it. They're good at that. I mean, when I write or rather when I go through the process with, like, GBTO one to go through a plan and then maybe use a different model, whether it's like SONNET or you mentioned Claude as well, which I find to be a little bit better for for pure code generation.
It's it's almost there. And then I admit I admitting to you and to our audience, I will take that code and it's not working right. I will copy the section that's troublesome blindly. I'll think about it for a minute and be like, should I, like let me, like, start looking at it, see if I can fix it.
And I'm like, nah. Copy that right into the prompt and say, what's going on here? And it fixes it, you know, generally. And yeah.
So there's a little massaging, but my goodness, I'm so much of the way there for a novice, coder. So, I you know, we're talking about those concerns. We didn't talk about security concerns, but that's another one. We talked about the bugs.
We talked about the, you know, the the the changes that we're that we're seeing and going to see, but I I can't deny the benefits, in my day to day. Oh, yeah. I can't deny it.
Yeah.
Another one you didn't bring up, and I I haven't really heard anybody talk about this one, but I've noticed it myself through various projects.
And so for all the product people out there, pay attention real quick.
So there are certain packages and there are certain products out there that have a really crap API or they have really crap documentation.
Versioning is horrible so documentation and product usage is all over the place.
Humans struggle with that but they can figure it out. AI struggle with it ten times worse. And I've had some projects that have gone and wasted ten times the amount of time they should because of the product or because the package that I'm trying to work with has crap documentation, has crap APIs, whatever.
I'm gonna start avoiding those products moving forward because I don't want my AI wasting time on them. And so it's another element. You know, you're you're gonna see certain packages get preferred over the others, and that's one of another really good reasons.
Yeah. Yeah. I agree. That makes a lot of sense, and I hadn't thought about it in those terms. And that, of course, does presuppose that this is kind of the path moving forward. And I I think it is important to sort of, recognize that the we are in the infancy of all of this stuff, and so folks are kind of experimenting.
We're gonna see maybe we're gonna see a very GUI driven, you know, I don't know, mechanism to drive a lot of, AI software development. Who knows?
And and making it that much easier since we're abstracting it more and more. I don't I don't know. But, certainly, I do believe that we're in the infancy sort of fleshing this stuff out. Now Absolutely.
Several times now, you mentioned the use of your interaction with agents to carry out specific actions. So can we dive into that a little bit? What we're talking yeah. What we're talking about here is not necessarily, like those initial vibe coding things that we did a year ago where you're just throwing, you know, some prompt into the GPT window or into the, you know, the llama window, and it generates the code.
And then you just continually go back and forth, adjusting it with that same prompt window and blah blah blah. What you're talking about is much more sophisticated where you have agents involved carrying out particular tasks. I assume, is sort of like a subject matter expert agent as well, carrying out specific tasks with specific tools and abilities to call on certain tools.
Can you flesh that out a little bit? How how does that look as far as a landscape?
Okay. So to to to start and kinda get everybody a clear picture of of what we're getting into here, you think about chat GTP two and a half years ago compared to chat GTP now or or any of them. Perplexity, Claude, whichever model or chat application you wanna use. So you jump in there and you ask it a question, like how do you blah blah blah blah blah blah.
And then what happens is you start seeing the the application work. You start seeing the LLM process and go through different tasks. It actually searches the Internet, it refines some of its data. And sometimes it hides these steps, sometimes it shows them to you partially.
All of these are basically separate agents that are running through and pulling data together to then form a stronger, a stronger decision and this is kinda getting into RAG as well.
And so what's happening or what happened is as this became more of a need, a couple years ago, all of the bigger models started building and supporting the use of calling tools. And so tools was basically allowing an LLM to to use a piece of static code or use an API that can pull in information.
And so tools became pretty popular and then it got to a point where people were like, well, dang, tools don't just have to be a simple script. We can write full on applications that sit next to an LLM and the LLM calls it as a tool to get information out of it. So it could be as simple as like, I I need to go do some sort of Ansible configuration that's gonna automate this, but I need to get some switch details.
So you tell the LLM, go look at this specific site for all the specs that you need, pull it together and write the code. And so we would use a tool to reach out to the Internet and do that. Well, it got to a point where all the different models essentially had their different ways of doing tools.
It it wasn't as robust as was needed because it was an early stage type thing or a phase one. And so Claude Anthropic, Anthropic, sorry not Claude. Anthropic came out with what's called MCP or model control, model context protocol. And it essentially is a communication protocol that allows a model to talk with third party software to collect data in a format that it's it's it's working with.
And MCPs are written with normal software, most of them are using TypeScript and and React, and you launch them as just a separate service or instance on your local machine and you point your models to it that support MCP or you point your tools to it. And now you have this additional functionality to to help you get details that the model itself or the agent itself is not aware of.
And so you you start adding in value of so I I walk through all of these things that, you know, software engineers had to develop and figure out over the years, you know, get product managers, documentation, testing. All of these things can be built as MCPs. So one of the more powerful MCPs that I use almost every single time is is a GitHub or a Git MCP. And so essentially when I'm working on a problem, like the example I had earlier where, you know, we had a feature that worked great.
I loved how it was. There was a couple bugs. We lost all that work. Well, with the Git Git MCP, I can just sit there and say, hey, pull out out of this specific commit ID how everything worked.
Revert back to these changes, but let's fix the bugs.
I'm not having to go to Git, pull out all the information, or I'm not having to manually step through this. The agent itself reaches out through an MCP to Git, pulls in all the changes, understands them, and then steps forward with the process. And then that's repeatable for any type of tooling out there that you want to use, auditing, security checks, however you want to go. And so that's that's kind of where this is taking another step.
So for Vibe Coaters, it's gonna be one of those where you add power tools to to your tool belt. You're you're adding additional stuff that's really gonna push you a lot further. But when you're transitioning into a more robust workflow that a team's gonna need to work with or we're gonna push out into production, these are tools that's gonna help you make sure that you're developing consistently, that you're following best practices, that you're securely building your code. All the stuff that's needed for enterprise grade development, this is what's gonna bring it to you.
And it puts everything into an actual workflow at that point as opposed to using, you know, on my other screen, I have whatever model open, and I'm just throwing prompts and copying the code into somewhere else. Here, I'm also through the use of an MCP server and agents and tool calling, I'm also updating repos. I'm also pulling from here and doing that, and you really have a much more consistent workflow as a result.
And, yes and no.
Okay. Yes and no. I I feel like you do, but explain why you disagree.
Well, if if so you don't specifically have to tell any of these agents to reach out to an MCP server or to do something. Okay. Yeah. They have descriptions.
So you can sit there and say, hey hey, Claude Code. I need you to, update this file, validate the last git, check-in and see what the difference is. It'll recognize that it's trying to do git Mhmm. From your prompts and it's gonna see if there's any MCP servers available to go do that.
So you don't have to specifically say reach out to the MCP server and do blah blah blah blah. It it interprets it.
Mhmm. Right.
So the good of that is you can be very flexible and fluid with how you move forward and not understand what MCPs are needed.
But the flip side, you might not necessarily know what your workflow is.
So from a vibe coding standpoint, you just roll with it. But from an AI driven development standpoint, it's like I need to nail this down a little harder and I need to establish a workflow through my props.
Yeah. I see your point now.
So that's kinda why I say it could be either or.
Yeah. And, you know, my experience is that, you know, just integrating an MCP server, you feel like, oh, I'm adding a level of complexity, and it's kind of the opposite. It's really simplifying things.
Yes, you're adding another abstraction layer, but it does make things more simple.
Yeah.
Yeah. And, you know, if you're looking at it, another good way to look at it too, MCPs are just basically like importing modules or importing packages into Python or using PIP or in MPM for for React and front ends. You know, it's it's it's the same type of thing.
And I think I think moving forward, a lot of what folks do in the day to day as far as Vibe coding is gonna be through the use of integrated tools.
Whether it's something built into Versus code like you were talking about earlier, GitHub or in, you know, like, any program.
I think that's honestly, to go a little bit more macro, I think that's the way that folks are gonna be using any kind of AI tools, you know, in as far as their personal interaction with them moving forward.
And we're gonna see that more and more. I mean, just like you open up, like, Teams or whatever you're using, and there's an AI functionality that you use built into that. Why? Because it's easy, and you can access the data that's in your company's Microsoft stuff, you know.
And so it's all there, and it just makes life easier rather than building this other thing. So I think that's how we're gonna consume a lot of these, products, whether it's vibe coding or AI in general. Now not not to say that there aren't gonna be those AI ML initiatives that enterprises take to do some sort of predictive analytics or to, you know, do something sophisticated and, you know, detect fraud or figure out if, you know, prospects are gonna buy or not buy, things like that, click on a on a on an image or not click on an image. So we're gonna still have that.
But from, from just people, whether they be software developers, people working in IT operations, or anything like that, I think it's gonna be through the use of those integrated tools.
I agree. I agree.
I I think we're gonna also see, in the next few years, we're gonna start seeing a lot of pipelines and a lot of automation stem from this as well. But we're also gonna see some new workflows that weren't possible in the past.
One of them that I've kind of leaned in on, I've I've given a number of presentations around this one as well, is the the kind of process you get from the traditional process of taking a feature request from customers and the whole workflow to get that to a shippable product where it actually shows up in the customer's environment.
And, you know, it's it's it's it's a complex workflow and the large bulk of failed projects happen because this process is horrible.
And essentially what happens is we we hand off customer feedback to product managers.
Product managers have this complex job of pulling everything together, understanding what needs to be built, understanding requirements, what the customers are wanting, what the limitations are in engineering.
Is it actually gonna be workable in our product? Does it make sense to look like this? And then they have to gather feedback, they have to hand it off to engineering, engineering gives feedback and passes it back, things get finalized and then we can start writing code to see what it looks like.
But something like Vibe coding and being able to have throwaway POCs, you you can automate a process to where every iteration that the product manager goes through in developing a feature, they can have working code to play with hands on to validate their thoughts right away. And we don't have to pull software engineers away from the engineering department and the features they're working on to build this for the product manager. The product manager can do it themselves.
And then at every step along the way where decisions are made, there's the possibility to introduce, you know, a throwaway demo of what they just asked for.
That's an interesting And so we we can we can we can really nail down exactly what we're after before it's handed off to engineering.
And then even better for the engineers, when they have a working POC to build off of, they don't have to use it. They can throw all of the code away and rewrite it. They know exactly what they're going after and they know exactly what they're building because the POC shows it and the code is right there and it's static and they can work with it. Same thing for engineers. They can have something built out right away to hand off to POCs at every single one of the checkpoints without having to do that themselves.
So these automated workflows where it allows people to extend beyond their normal abilities is really gonna open up how we do a lot of business.
Yeah. I I agree. And I like the example that you gave, because in my experience on the infrastructure side, on ML ops side, that kind of stuff, So many POCs just, like, they go nowhere. They they fail.
And so project ideas stay in the POC stage, perhaps because the resources aren't there to, like, keep running with it or because nobody had clear objectives, you know, what this is supposed to lead to. Was this a successful POC? Or it we learned it's just too expensive. This is too expensive and the return is not there.
Or last but not least, the POC itself is too much of a big lift. It's too expensive in and of itself to do, you know, which bars entry even to, you know, explore the idea in the first place. And so we're gonna, you know, fall back to something manual or something else. I have found that so many initiatives, fail at the POC stage.
Now to be fair, many of those initiatives fail at the POC stage, especially in the AIML space because they don't have their data under under, control. Their data pipelines are not there, and there's disparate databases. There's no transformations and cleaning and processing has been done, and it's just figuring all that stuff out. That's probably the bulk of the reason a lot of these things, fail.
But I love that idea. I love that idea of being able to create because it kinda goes along with Carpath with Karpathy said about, like, it's good for a throwaway project. Okay. Awesome.
So he says that with a negative connotation, but you're presenting it as a, like, no. That's an awesome way to, like, experiment and get something going. And then and then perhaps continue to leverage vibe coding or get into some more, traditional software development to to to develop and create your your final product. Really, really interesting.
I hadn't thought of it, that way.
Yep.
The other one that I kinda add on to that one as well is, you know, we waste a lot of time in the industry and in business in general where we'll sit around and have these big powwow sessions of, like, we're gonna spend two days designing this new feature and we're gonna build this or we're gonna build a whole new blah blah blah blah blah and we're all gonna sit in a room and spend x amount of time hashing it out. And that's great. We get in there, we do amazing work, everybody comes out excited, but no one comes out with the same viewpoint.
And the further along the project goes, the more drift everybody has in their understanding of what the actual solution is.
So if you can take AI and you can record the conversation, record the transcript of the meeting, record the notes, and then everybody leaves and goes to lunch or everybody flies home for the weekend, by the time they get to their desk the next day, is there a working product or a working POC that can be built automatically out of that? So everybody can sit down and play with what they just agreed to do and build and go towards in the meetings. So you don't lose all that context of what's needed.
Now, of course, there's multiple technologies. It's not just coding, but it's it's getting into voice recognition and in text to speech and blah blah blah blah blah, that stuff. But that's another good example that I have is we we kill those inefficiencies in in the that humans introduce.
So is it vibe coding if I have model generate my code for me, but then I go back and I check it and understand it line by line. Is that still vibe coding? Or is that just using a a an assistant? It it is.
You you you know where I'm going with this, Ryan? Like, is that just an assistant developing? Because it's just syntax at that point. I know what's going on.
I can read it and understand it.
You know what I mean?
Yeah. I I think so. You know, if if if you're you can have I I think if you have a deeper understanding of what's happening with the code and you can you can walk line by line through it, but you didn't necessarily choose to get into that state and keep it in that state, sure, you know. I mean, that kind of falls in line with what Vibe Coding is.
But I do think it's the introduction to the gray area between, you know, what you're doing with Vibe Coding and and what a a nailed down process would be with, like, AI driven development type stuff.
Well, I asked the question because I I I read online some people discussing that. I don't know if it was Reddit or not, not, whatever. But, you know, some folks were saying that Vibe coding, the way they saw it, was, like, not really knowing software development very well, and it's awesome. So you can use this, you know, method to just whatever.
And you can generate some code, continually copy and paste the error problems and and eventually fix it and have a a working, working code. And then somebody else said, well, you know, if you really understand the code and you're just using it as as that assistant to generate it and you're able to go in because you have the experience and the knowledge. It's not vibe coding anymore. Now it's just like an autocomplete.
It's just syntax. It's not really it was an interesting argument. And it, again, it was a back and forth argument, that probably devolved into, like, name calling as these things do on Reddit, and people claiming that you're not a real developer at that point and things like that. But, that's why I brought it up because it is an interesting thing where it does get into these gray zones of, like, well, it's an assistant.
It's it's a you know, like, agents are. Right?
Where we have this agent that does the tool calling to to fulfill some sort of task. I could have done manually.
So in the same way, you know, vibe coding is something that helps you along the process. And if you're, you know, an experienced, knowledgeable developer, that helps you, you know, along the way to get to that to get to that, conclusion much faster.
You know, me personally, I'm one of those. I I I don't really subscribe to the whole yak shaving of what does a term mean in our industry or not. You know, we throw these terms around all the time and they they get meaningless quick. But I do like to pay attention to those types of discussions and conversations because those are the conversations where people bring up their ideas. They bring up problems. They bring up possible solutions.
They bring up possible areas for a new product or, you know, a a new feature that can be developed. People are this is where people as they're debating back and forth if vibe coding or whatever it is is a valid term or valid to use, that's where you see opportunity to make it better.
So, you know, I wanna get closer to wrapping up here. But before we do, Ryan, how can folks get started with, you know, vibe coding? Is it is it really just, firing up chat GPT and throwing your request for what you wanted to make, or is it more than that?
Yeah. There's it it really is just just find find some products that work great for you, that you kind of feel you mesh with. Some you'll notice really quickly, they seem weird and awkward. Others will have a very natural feel for you like, oh my gosh. This is amazing.
Find those. It takes a little bit of trial and error. The one that I prefer is called Cline. It's an open source free product, so it's great to jump in and try. It's built into v or it's not built in, it's an extension that you can install into v s code.
It does great. It's amazing. I love it, but there's also cursor, there's also replit.
I know a lot of people have been crazy impressed with, Claude Code.
So I I just encourage you get in there and try them all, pick one, nav or, figure out which one works best for you, and then just start doing. Like, throw stuff at it. You don't have to build anything that works. You don't have to really do anything important.
Like, something that watches your toaster for toast to pop up or something. It doesn't matter or, you know, build a financial tracking system. Whatever. The failure is the learning part and so that's the important stuff. So you just jump in and start doing and figure out how far you can go.
Yeah. I would also like to add that don't get hung up on models. Model selection is it can be important for sure. And I do prefer, like, o one for planning and then, you know, then there are other models that are a little bit, more adept at code.
But honestly, honestly, they're all pretty good. They really are. Yeah. So I I wouldn't get hung up on that.
Just start. And then later on, sure, you can use different models for different purposes and your workflow can have, you know, an LLM router even and start to do those kind of interesting things. But I wouldn't get hung up on that. If you're reading through, like, some of the Reddit threads about that, there's gonna be some heated discussions on exactly what model to use when, and everybody's a jerk that doesn't use the model I say.
That's silly. Just just get started like Ryan said. I think that's the best way to do it.
Evaluating and understanding what models do what best where is one of those that can eat up all of your time. And you're right. It it has very little meaning to the end goal of what you're trying to do. It's kinda like picking what database you use. Who cares? Pick the one that works, that vibes with you, and move forward.
And you can waste so much time. I I usually just keep, you know, conversations rolling with the people that spend all their time watching models, and I gather their opinion or I kinda just hear what the industry is saying. Three five Sonnet's great. Work with it. Roll with it.
Yeah. Yeah. I mean, there are considerations. Once you get into very sophisticated and complex workflows and its production environment, you know, you wanna take things into consideration like latency and, you know, dealing with, you know, real time telemetry things like that. I mean, there's there are considerations.
But that's down the road.
That's Yeah. Exactly. And then For sure. Those types of things usually appear pretty, clearly. So when you need to address it or when you need to think about it, you'll probably know already that you need to already that you need to do it. And so it's it's not that big of a deal.
I agree with you there. Oh, thank goodness. Alright.
It's the first time. Well, Ryan, it's been it's been great having you back on the, the podcast. Gotta do it again sometime. I definitely wanna get into, agents and MCP at a much deeper level and really discuss what that looks like, especially for IT operations, moving into the the rest of this year, which I know is not necessarily your big focus, these days, but certainly, something I know is important to, to our audience and our listeners. So Yeah. Ryan, thanks so much again for joining.
Really appreciate it.
Anytime. Anytime. I appreciate you having me.
Oh, yeah. So if you have an idea for an episode, I'd love to hear from you. You can reach out to us at telemetrynow@kentik.com. So for now, thanks so much for listening. Bye bye.
On today's episode of Telemetry Now, my good friend Ryan Booth is joining us again to talk about vibe coding, the latest buzzword out there in the tech world. And, of course, we'll have plenty of talk about AI.
If you haven't heard that term yet or if you have and you're not exactly sure what it's all about, I think you're gonna like this episode. My name is Philip Gervasi, and this is Telemetry Now.
Hey, Ryan. Thanks so much for joining again. It's been too long. Last time you were on I don't remember.
We were talking about AI, I'm sure. But, it's been too long. So thanks for coming back on. Today's topic is AI-ish.
I mean, it's definitely related to what's been going on in the world of AI and specifically with regard to vibe coding.
I saw a yeah. Yeah. I saw a LinkedIn post of yours just the other day, and you were talking about it. It's top of mind for a lot of people, including myself.
And when I saw that, I'm like, yeah. He's got an angle. Like, you know, there's a pretext in those words there that he's putting out there for the public to see. Let's flesh that out a little bit. So, you know, thanks for coming on, and, I hope we, yeah, we're definitely gonna have a lot of fun today with this. Yeah.
So It's always a great time to to chat with you and and bounce ideas off each other. I I really appreciate you having me on.
Yeah. Yeah. Absolutely. So let's, sorry for using the management speak. Let's level set. What does the term vibe coding mean?
You know, you could start with what it means to you because I do have a definition here on my left screen, but I wanna hear from you first.
Alright. Yeah. So, the the term vibe coding pretty much derived I don't even know where it where it originally came from. If anybody knows the exact source, I'd I'd be interested to hear.
Well, I do have it here, but you you continue. Oh, yeah.
Yeah. Okay. So basically over the past six months to a year, models have been getting better and better at developing code, and more complex code and more code across various projects, instead of just, you know, a single script or or a single function or something.
And over time, it's kind of turned into what we consider vibe coding right now, where someone just sits down with any one of the agentic tools. There's, Replit was one of the very first ones. There's Cursor.
There is Cline for VS Code. There's a number of them out there, and one of the more recent recent one that I think helped really get a lot of popularity recently is is the Claude, Claude Code.
And they they allow you to sit down and write software in a conversational manner. So you're not actually creating files and, well, you're you're not taking and copying and pasting from, like, Chat GTP or any of the other models out there and just pasting it into files and doing everything. You're actually telling the the agent to build something, and it goes out and creates all the right files. It does all the right stuff. It it creates the environment. And so essentially, you are just kind of in this conversational prompting with the agent to to write whatever you're trying to write.
And that's that's a lot of what it is with Vibe Coding. Now I I kinda sense that the word vibe comes from the notion that there's not really much planning.
There's not much structure. You're just kinda stepping through the conversation and letting it flow how it goes.
So you ask it to do something to create this API endpoint to host up blah blah blah blah blah. It goes out there and creates it, and you're like, well, this did okay. This didn't do okay. And then you turn around and you're like, hey.
I need you to chain this. Change this. I don't want you to use a body. I want you to use the the argument parameters to pass in data yada yada, whatever it is.
And then you kinda just step through this whole process until you kinda get it to a point where you're satisfied with it.
And to me, that's that's I've I've seen that and I've experienced it with everything that I've been doing over the past six months in in AI development.
There's just never been a good term in my mind, for that just kind of process of flowing like that. So that's that's to me where vibe coding came from.
Yeah. I'm with you because I've been, you know, utilizing some of those same workflows perhaps to a lesser extent for months now. But the actual term is credited to, Andrej Karpathy, the cofounder of OpenAI, a former AI leader at Tesla as well. And that was just last month, I believe, on Twitter.
There was some, you know, tweet that he put out there in February of of just last month. Right? Using that term and, you know, going on to say things like that, it's just it's not really coding. I just see things, say things, run things, and copy paste things, and it mostly works.
Mostly being a key indicator that there's some limitations there, and, you know, you need to, you know, work with it. It's not that you're just putting some stuff in a prompt like you said, copy and paste it into your, you know, Jupyter notebook, and then boom, everything is good to go. There is a process there. I I have personally been using tools like like Cursor, you mentioned.
Certainly, the, the foundational LLMs as well. Like, you know, I do I do like chat GPT, and I've been using o one a lot, for planning.
And then for the actual, like, development of code, I'll use I'll use a different model. And what's interesting to me is that I find it beneficial to actually use different models for, like, the design phase versus, like, generating code and even evaluation phase. But, the you know, it is interesting that that, Karpathy said that straight out, that he said it's not really coding, in in the classic sense. So would you agree with that then?
Because, you know, you really are leaning on the pseudo code generation and the planning and the workflow and then letting the tool develop it.
So it it it calls out, a a big differentiator, to be aware of in the software industry. So, you know, there there is a very clear difference between software development and software engineering.
Okay. You know, development is basically taking the task at hand and writing code and outputting exactly, you know, what you're supposed to output and handing it off. Software engineering is exactly what you're doing in VibeCoding. You're you're stepping through and architecting a solution for whatever you want, and then something else is doing the software development.
And a lot of organizations and a lot of people out there, you know, both roles are mixed together, and that's fine.
You know, a lot of scenarios, that's perfectly acceptable. But, yeah, it's it's a clear difference between the two.
Okay. So then do you believe that software engineering is currently undergoing a massive shift in what that even means in the industry, in the software engineering industry as a whole?
Yeah.
I don't think it's in the middle of going through it. I think we're at the very beginning.
We are just now starting to see the larger software industry as a whole even understand what the k or even know about the capabilities and what you can and can't do right now.
Even up until two to three months ago, I was seeing where a lot of software engineers that I talked to, in the circles that I'm in, just had no idea how far you could get with with Vibe Coding, as you would say right now. But over the past four to six weeks, a lot more people understand that now. And so I think we're right at the start to see a transition. And you're what you're gonna see is a lot of the positions are this is at least what I'm thinking. You're gonna see a lot of collapse in positions.
So you don't necessarily need four layers of software engineering and management to to get from product down into developing features. You you just need one. And so, having a a clean handoff and then dropping it into a software developer who can take on the rest, or a software engineer who knows what they need to put together, why it matters to the customer, what it means to to build it in the way they need to and then ship it. And I I think that's where we're gonna we're gonna see a lot of collapse there. So the engineers that really just focus on nothing but writing code, you might wanna start expanding your horizons a little more.
Yeah. You know, I have heard for years and years and years throughout my entire career, in the networking space, but in IT in general, in the software space, this desire to focus on outcomes, aligning with the business and business outcomes. And I really feel like, for the most part, we've always just paid lip service to that across technical industries.
But I wonder, and maybe you agree here, that this it really is the focus now, isn't it? I mean, that's what you're doing through this process of quote, unquote vibe coding Yeah. And going through the outcome based, you know, workflow and then letting the tool develop the actual code, which really is just the nuts and bolts that we don't really care about as long as we get to the outcome that we want. And again, we've been we've been paying lip service of that for years and years and years, but we we get lost in the weeds.
And I really feel like that's that's different now.
Yeah. You make a really good point there.
I don't necessarily disagree with that. I I think you're right. And we've we've had to organize in in tech teams, around, you know, putting this whole workflow together in this chain from the customer all the way to shipping a product.
And we've had to put the right people in the right positions with the right skill set. And that's always kinda where it's architected around in my opinion.
Product managers really are great at seeing the overall vision, seeing what the industry wants, understanding what their customers want and translating that into workable features.
But they're always limited because they don't have the ability to sit down and write code.
Vice versa, networking or engineers in general have the ability to create and build whatever you're asking them to build, but they don't always necessarily have the full scope of what the customer wants or what the industry wants. And now one of the things that AI is doing shaking all this up is it's giving all the different people the skill set or at least a good enough skill set in the other areas to be able to press through and do what they need to do without having to involve the rest of the groups with it. Now, will that still be needed? Will will those types of people still need to be in place? Sure. Especially as you get into larger organizations. Absolutely.
Smaller shops though? Nah, probably not.
And there's there's even the the discussion going around that that gets me a little bit excited myself is the notion that, in the next coming years we're gonna see Deca corn companies, Deca corn startups come out that are just a single person.
Yeah.
I've heard of that.
And that is where I see, you know, the power of the technology coming into play as you take one person and they have their skill sets and they have their weaknesses, but this tool or these tools are gonna allow that person to run an entire business very successfully because of their help. Mhmm.
So one thing that I I believe it was Karpathy who said maybe with somebody else that, you know, vibe coding is really for, like, a weekend throwaway project. And for me personally, though I didn't call it vibe coding for many months now, more than months, I've been doing that for stuff that I'm building at home in the lab, because I'm not a developer for my day job, nor am I an engineer anymore, in that in that sense, infrastructure engineer.
And so for me, they they're they really are throwaway projects in the sense that they're experiments.
I'm forcing myself to learn a new thing.
I'm trying to see how I can, you know, do a thing with certain types of data formats, stuff like that. And, and then when I'm done, I sort of move on. Maybe I put a blog post out there, whatever.
Do you think that that is not the case? You did say that we are in the initial stages. So are we moving toward the this, this idea that Vibe Coding is not really not necessarily for throwaway projects for the weekend project, but can be for, you know, sophisticated development projects and for, enterprise deployment?
Yeah. Yes and no. So I think it has its place. That's kind of what what the whole reasoning behind that post that I gave was was to highlight this point.
You know, yes, Vibe Coding can get some some very good results working pretty quickly. And more important, it can give someone who normally couldn't build those the ability to build them. The problem is is after you build it, when you want to augment or change or add features, that lack of control on what you can and can't do really starts to show through. And you start seeing a ton of wasted time and a ton of, lost control the further along you go. And so some some good examples that I've had is, in in some of the projects that I've worked on doing nothing but Vibe coding, I can sit down and I can say, okay, go go build this feature, make the API endpoints look like this on the back end, make make the front end look like this and it produces this output.
And we step through all of that and it looks great. I like how it works but it's buggy. It has some stuff that I need to stabilize.
And so I'm I'm happy with where we're at. It's a good checkpoint, and then we move forward to fix the bugs, and the agent just blows everything up, destroys it all. Like, whatever it does, whatever made it go that way, things just go south. And it's like, okay.
This is the not this is not the right approach. Let's back off. Let's go back to a previous state. Let's fix our changes and then move forward.
The problem is is as they fix stuff, they start it starts changing Oh, yeah. How the outcome is. And you're like, woah. Woah.
Woah. I like the original outcome.
Mhmm.
And Vibe coding's like tough shit. This is what I'm giving you now. And so you lose that control.
Pet projects, proof of concepts, you know, side stuff, that's not that big of a deal because it's it's one or two people you're probably developing for, maybe probably yourself. So, you know, you can adjust to stuff like that. But when you have customers that are like, I need to see features like this. I wanna see things like this. Or you have an industry that's like, you're only gonna succeed if you apply software in this manner or do this, that, and the other. That type of control takes a different level of AI interaction than just vibe coding. And that is where the separation line is for me.
Yeah. I've been there. I've been there where, I wanted to add another data format or another I wanted to get returned in this way. So I'm I'm, you know, interacting with with the model, and and it it kinda reformats a whole section and and, you know, I try to keep whatever code modular.
And, it's not the way it spits it out generally. And so I have struggled with that where, like, alright, I wanna make this change and I got a new thing to work with and then something else doesn't work now. So I have noticed that. And for me personally, because I'm not a, a very, very experienced high level developer, you know, it's hard for me to then go through the code line by line and really understand exactly what's going on. Of course, I do. And it takes me time. And I in the prompt, I'll add things like please include, you know, context, include, comments, all these other things, and and it does.
But it does take me time to go back and see, alright, we're what is it doing here? Whereas when I write it, though it might take me significantly longer, like you said, total control. I know exactly what's going on every line and why and why I did this. And I can do it in a modular fashion as well where I know Mhmm.
Later on and that's that's one of the things that, you know, people need to think about when you're doing any kind of AI ML product in in your data engineering pipeline is, you know, what kind of data do I have and where can I be going with this data in the future? And am I hamstringing myself today with my data pipeline and engineering processes? Same thing with your code, with vibe coding. Are you creating this product where, you know, later on you can't develop, in the way that you want to?
Mhmm. So, yeah, you're you're kinda generating bottlenecks, but you are also removing some. Like for me, you know, it it makes it a lot easier because I'm not an expert developer. So it does make it a lot easier and I'm like, holy cow, I never thought of that before?
Oh, is that how you do it? But you know one thing that's interesting, and this could be just for me. So, you know, maybe this isn't the situation you've encountered. Sure.
Bugs, things like that.
The ability to understand, you know, the code because I wrote it, all that stuff. But, does it make us sidestep best practices? You know what I mean? From a software development, like a craft perspective.
Absolutely. If you want to.
You know, it's that's that's kind of the the the, double edged sword you get when you're a software engineer and you're you're writing code is, you know, you can go the other way that's easier. You can go the other way that's faster.
And essentially what you're doing is you're adding technical debt to your project. And the more technical debt you get on, the lower your velocity and the the lower, you know, of time to getting a feature deployed, or adding in stability, adding it where developers have to have a much larger skill set to be able to work with your code base.
All these types of things come into play.
Kind of the nice thing that I see with all of this is the software industry has kind of already been through this motion through its maturity.
You know, we started off within the early days where, you know, everybody would just share files and then drop them into a file share and everybody has their own version and their own folder and then at some point in time, they spend two to three weeks merging everybody's stuff together or stuff gets accidentally overwritten. All those problems developed us into using Git, and, you know, now everybody's using GitHub or Bitbucket or whatever to to store projects and that that allows us to streamline how we manage multiple people writing code. And then we get into a thing where, you know, we got scope scope creep, you got bloat, you got unwasted time and features. And so we add in the product management element of it. We we add in developing PRDs and, designs and engineering docs so we we we control the flow of it.
Adding in testing and adding in, you know, CICD pipelines to sanitize and validate the code. All of these things we have put in place to address the human elements of writing code and to control that.
We just now need to apply the same thing to AI agents.
AI agents do the same thing. They make errors, they have issues, they go sideways, whatever it is. Sometimes they throw code in that just looks like garbage.
Working it through a pipeline, going through each one of these steps, each one of these check boxes, that's that's how you transition away from vibe coding into, more of a term that I like to call of, AI driven development.
Yeah. Yeah. I'd like to talk more about, specifically the role of the agent in AI driven development at some point. I mean, I've been experimenting with some very low code, almost no code drag and drop, platforms recently that are you know, there's a part of me that's like, well, I wanna be in the code and, you know, touch all the the knobs, the nerd knobs and the stuff. But I'm like, man, this is just so easy, though. And then you can open up the pane where you see the code and you can adjust things if you like. But it it just makes it so much easier and gets me to the goal much faster.
Mhmm. You know what's interesting is, when I when I do when I give it some sort of prompt, go through the workflow, go through the process, I noticed that often I'm just getting, code generated that looks like it's just from scratch and it does not go it doesn't use, like, certain libraries that I would normally use. And, you know, being in the observability space is gonna be some, like, data analysis libraries, scikit, things like that, Matlab. You know? And and and it doesn't use a lot of that stuff. And you see all of it done in this, like, long form manner, and that's an interesting difference. And I don't know if that's a good thing or a bad thing, but I wonder if we're gonna see if this continues to grow and vibe coding, which is kind of in in its infancy, or at the c CLI level of vibe coding, right, before we get into more advanced, mechanisms to do it.
You know, I wonder if we're gonna see a decrease in some library usage because of that, because it's just being written by by a tool that can do it. And and, frankly, these models are very effective at doing it. They're good at that. I mean, when I write or rather when I go through the process with, like, GBTO one to go through a plan and then maybe use a different model, whether it's like SONNET or you mentioned Claude as well, which I find to be a little bit better for for pure code generation.
It's it's almost there. And then I admit I admitting to you and to our audience, I will take that code and it's not working right. I will copy the section that's troublesome blindly. I'll think about it for a minute and be like, should I, like let me, like, start looking at it, see if I can fix it.
And I'm like, nah. Copy that right into the prompt and say, what's going on here? And it fixes it, you know, generally. And yeah.
So there's a little massaging, but my goodness, I'm so much of the way there for a novice, coder. So, I you know, we're talking about those concerns. We didn't talk about security concerns, but that's another one. We talked about the bugs.
We talked about the, you know, the the the changes that we're that we're seeing and going to see, but I I can't deny the benefits, in my day to day. Oh, yeah. I can't deny it.
Yeah.
Another one you didn't bring up, and I I haven't really heard anybody talk about this one, but I've noticed it myself through various projects.
And so for all the product people out there, pay attention real quick.
So there are certain packages and there are certain products out there that have a really crap API or they have really crap documentation.
Versioning is horrible so documentation and product usage is all over the place.
Humans struggle with that but they can figure it out. AI struggle with it ten times worse. And I've had some projects that have gone and wasted ten times the amount of time they should because of the product or because the package that I'm trying to work with has crap documentation, has crap APIs, whatever.
I'm gonna start avoiding those products moving forward because I don't want my AI wasting time on them. And so it's another element. You know, you're you're gonna see certain packages get preferred over the others, and that's one of another really good reasons.
Yeah. Yeah. I agree. That makes a lot of sense, and I hadn't thought about it in those terms. And that, of course, does presuppose that this is kind of the path moving forward. And I I think it is important to sort of, recognize that the we are in the infancy of all of this stuff, and so folks are kind of experimenting.
We're gonna see maybe we're gonna see a very GUI driven, you know, I don't know, mechanism to drive a lot of, AI software development. Who knows?
And and making it that much easier since we're abstracting it more and more. I don't I don't know. But, certainly, I do believe that we're in the infancy sort of fleshing this stuff out. Now Absolutely.
Several times now, you mentioned the use of your interaction with agents to carry out specific actions. So can we dive into that a little bit? What we're talking yeah. What we're talking about here is not necessarily, like those initial vibe coding things that we did a year ago where you're just throwing, you know, some prompt into the GPT window or into the, you know, the llama window, and it generates the code.
And then you just continually go back and forth, adjusting it with that same prompt window and blah blah blah. What you're talking about is much more sophisticated where you have agents involved carrying out particular tasks. I assume, is sort of like a subject matter expert agent as well, carrying out specific tasks with specific tools and abilities to call on certain tools.
Can you flesh that out a little bit? How how does that look as far as a landscape?
Okay. So to to to start and kinda get everybody a clear picture of of what we're getting into here, you think about chat GTP two and a half years ago compared to chat GTP now or or any of them. Perplexity, Claude, whichever model or chat application you wanna use. So you jump in there and you ask it a question, like how do you blah blah blah blah blah blah.
And then what happens is you start seeing the the application work. You start seeing the LLM process and go through different tasks. It actually searches the Internet, it refines some of its data. And sometimes it hides these steps, sometimes it shows them to you partially.
All of these are basically separate agents that are running through and pulling data together to then form a stronger, a stronger decision and this is kinda getting into RAG as well.
And so what's happening or what happened is as this became more of a need, a couple years ago, all of the bigger models started building and supporting the use of calling tools. And so tools was basically allowing an LLM to to use a piece of static code or use an API that can pull in information.
And so tools became pretty popular and then it got to a point where people were like, well, dang, tools don't just have to be a simple script. We can write full on applications that sit next to an LLM and the LLM calls it as a tool to get information out of it. So it could be as simple as like, I I need to go do some sort of Ansible configuration that's gonna automate this, but I need to get some switch details.
So you tell the LLM, go look at this specific site for all the specs that you need, pull it together and write the code. And so we would use a tool to reach out to the Internet and do that. Well, it got to a point where all the different models essentially had their different ways of doing tools.
It it wasn't as robust as was needed because it was an early stage type thing or a phase one. And so Claude Anthropic, Anthropic, sorry not Claude. Anthropic came out with what's called MCP or model control, model context protocol. And it essentially is a communication protocol that allows a model to talk with third party software to collect data in a format that it's it's it's working with.
And MCPs are written with normal software, most of them are using TypeScript and and React, and you launch them as just a separate service or instance on your local machine and you point your models to it that support MCP or you point your tools to it. And now you have this additional functionality to to help you get details that the model itself or the agent itself is not aware of.
And so you you start adding in value of so I I walk through all of these things that, you know, software engineers had to develop and figure out over the years, you know, get product managers, documentation, testing. All of these things can be built as MCPs. So one of the more powerful MCPs that I use almost every single time is is a GitHub or a Git MCP. And so essentially when I'm working on a problem, like the example I had earlier where, you know, we had a feature that worked great.
I loved how it was. There was a couple bugs. We lost all that work. Well, with the Git Git MCP, I can just sit there and say, hey, pull out out of this specific commit ID how everything worked.
Revert back to these changes, but let's fix the bugs.
I'm not having to go to Git, pull out all the information, or I'm not having to manually step through this. The agent itself reaches out through an MCP to Git, pulls in all the changes, understands them, and then steps forward with the process. And then that's repeatable for any type of tooling out there that you want to use, auditing, security checks, however you want to go. And so that's that's kind of where this is taking another step.
So for Vibe Coaters, it's gonna be one of those where you add power tools to to your tool belt. You're you're adding additional stuff that's really gonna push you a lot further. But when you're transitioning into a more robust workflow that a team's gonna need to work with or we're gonna push out into production, these are tools that's gonna help you make sure that you're developing consistently, that you're following best practices, that you're securely building your code. All the stuff that's needed for enterprise grade development, this is what's gonna bring it to you.
And it puts everything into an actual workflow at that point as opposed to using, you know, on my other screen, I have whatever model open, and I'm just throwing prompts and copying the code into somewhere else. Here, I'm also through the use of an MCP server and agents and tool calling, I'm also updating repos. I'm also pulling from here and doing that, and you really have a much more consistent workflow as a result.
And, yes and no.
Okay. Yes and no. I I feel like you do, but explain why you disagree.
Well, if if so you don't specifically have to tell any of these agents to reach out to an MCP server or to do something. Okay. Yeah. They have descriptions.
So you can sit there and say, hey hey, Claude Code. I need you to, update this file, validate the last git, check-in and see what the difference is. It'll recognize that it's trying to do git Mhmm. From your prompts and it's gonna see if there's any MCP servers available to go do that.
So you don't have to specifically say reach out to the MCP server and do blah blah blah blah. It it interprets it.
Mhmm. Right.
So the good of that is you can be very flexible and fluid with how you move forward and not understand what MCPs are needed.
But the flip side, you might not necessarily know what your workflow is.
So from a vibe coding standpoint, you just roll with it. But from an AI driven development standpoint, it's like I need to nail this down a little harder and I need to establish a workflow through my props.
Yeah. I see your point now.
So that's kinda why I say it could be either or.
Yeah. And, you know, my experience is that, you know, just integrating an MCP server, you feel like, oh, I'm adding a level of complexity, and it's kind of the opposite. It's really simplifying things.
Yes, you're adding another abstraction layer, but it does make things more simple.
Yeah.
Yeah. And, you know, if you're looking at it, another good way to look at it too, MCPs are just basically like importing modules or importing packages into Python or using PIP or in MPM for for React and front ends. You know, it's it's it's the same type of thing.
And I think I think moving forward, a lot of what folks do in the day to day as far as Vibe coding is gonna be through the use of integrated tools.
Whether it's something built into Versus code like you were talking about earlier, GitHub or in, you know, like, any program.
I think that's honestly, to go a little bit more macro, I think that's the way that folks are gonna be using any kind of AI tools, you know, in as far as their personal interaction with them moving forward.
And we're gonna see that more and more. I mean, just like you open up, like, Teams or whatever you're using, and there's an AI functionality that you use built into that. Why? Because it's easy, and you can access the data that's in your company's Microsoft stuff, you know.
And so it's all there, and it just makes life easier rather than building this other thing. So I think that's how we're gonna consume a lot of these, products, whether it's vibe coding or AI in general. Now not not to say that there aren't gonna be those AI ML initiatives that enterprises take to do some sort of predictive analytics or to, you know, do something sophisticated and, you know, detect fraud or figure out if, you know, prospects are gonna buy or not buy, things like that, click on a on a on an image or not click on an image. So we're gonna still have that.
But from, from just people, whether they be software developers, people working in IT operations, or anything like that, I think it's gonna be through the use of those integrated tools.
I agree. I agree.
I I think we're gonna also see, in the next few years, we're gonna start seeing a lot of pipelines and a lot of automation stem from this as well. But we're also gonna see some new workflows that weren't possible in the past.
One of them that I've kind of leaned in on, I've I've given a number of presentations around this one as well, is the the kind of process you get from the traditional process of taking a feature request from customers and the whole workflow to get that to a shippable product where it actually shows up in the customer's environment.
And, you know, it's it's it's it's a complex workflow and the large bulk of failed projects happen because this process is horrible.
And essentially what happens is we we hand off customer feedback to product managers.
Product managers have this complex job of pulling everything together, understanding what needs to be built, understanding requirements, what the customers are wanting, what the limitations are in engineering.
Is it actually gonna be workable in our product? Does it make sense to look like this? And then they have to gather feedback, they have to hand it off to engineering, engineering gives feedback and passes it back, things get finalized and then we can start writing code to see what it looks like.
But something like Vibe coding and being able to have throwaway POCs, you you can automate a process to where every iteration that the product manager goes through in developing a feature, they can have working code to play with hands on to validate their thoughts right away. And we don't have to pull software engineers away from the engineering department and the features they're working on to build this for the product manager. The product manager can do it themselves.
And then at every step along the way where decisions are made, there's the possibility to introduce, you know, a throwaway demo of what they just asked for.
That's an interesting And so we we can we can we can really nail down exactly what we're after before it's handed off to engineering.
And then even better for the engineers, when they have a working POC to build off of, they don't have to use it. They can throw all of the code away and rewrite it. They know exactly what they're going after and they know exactly what they're building because the POC shows it and the code is right there and it's static and they can work with it. Same thing for engineers. They can have something built out right away to hand off to POCs at every single one of the checkpoints without having to do that themselves.
So these automated workflows where it allows people to extend beyond their normal abilities is really gonna open up how we do a lot of business.
Yeah. I I agree. And I like the example that you gave, because in my experience on the infrastructure side, on ML ops side, that kind of stuff, So many POCs just, like, they go nowhere. They they fail.
And so project ideas stay in the POC stage, perhaps because the resources aren't there to, like, keep running with it or because nobody had clear objectives, you know, what this is supposed to lead to. Was this a successful POC? Or it we learned it's just too expensive. This is too expensive and the return is not there.
Or last but not least, the POC itself is too much of a big lift. It's too expensive in and of itself to do, you know, which bars entry even to, you know, explore the idea in the first place. And so we're gonna, you know, fall back to something manual or something else. I have found that so many initiatives, fail at the POC stage.
Now to be fair, many of those initiatives fail at the POC stage, especially in the AIML space because they don't have their data under under, control. Their data pipelines are not there, and there's disparate databases. There's no transformations and cleaning and processing has been done, and it's just figuring all that stuff out. That's probably the bulk of the reason a lot of these things, fail.
But I love that idea. I love that idea of being able to create because it kinda goes along with Carpath with Karpathy said about, like, it's good for a throwaway project. Okay. Awesome.
So he says that with a negative connotation, but you're presenting it as a, like, no. That's an awesome way to, like, experiment and get something going. And then and then perhaps continue to leverage vibe coding or get into some more, traditional software development to to to develop and create your your final product. Really, really interesting.
I hadn't thought of it, that way.
Yep.
The other one that I kinda add on to that one as well is, you know, we waste a lot of time in the industry and in business in general where we'll sit around and have these big powwow sessions of, like, we're gonna spend two days designing this new feature and we're gonna build this or we're gonna build a whole new blah blah blah blah blah and we're all gonna sit in a room and spend x amount of time hashing it out. And that's great. We get in there, we do amazing work, everybody comes out excited, but no one comes out with the same viewpoint.
And the further along the project goes, the more drift everybody has in their understanding of what the actual solution is.
So if you can take AI and you can record the conversation, record the transcript of the meeting, record the notes, and then everybody leaves and goes to lunch or everybody flies home for the weekend, by the time they get to their desk the next day, is there a working product or a working POC that can be built automatically out of that? So everybody can sit down and play with what they just agreed to do and build and go towards in the meetings. So you don't lose all that context of what's needed.
Now, of course, there's multiple technologies. It's not just coding, but it's it's getting into voice recognition and in text to speech and blah blah blah blah blah, that stuff. But that's another good example that I have is we we kill those inefficiencies in in the that humans introduce.
So is it vibe coding if I have model generate my code for me, but then I go back and I check it and understand it line by line. Is that still vibe coding? Or is that just using a a an assistant? It it is.
You you you know where I'm going with this, Ryan? Like, is that just an assistant developing? Because it's just syntax at that point. I know what's going on.
I can read it and understand it.
You know what I mean?
Yeah. I I think so. You know, if if if you're you can have I I think if you have a deeper understanding of what's happening with the code and you can you can walk line by line through it, but you didn't necessarily choose to get into that state and keep it in that state, sure, you know. I mean, that kind of falls in line with what Vibe Coding is.
But I do think it's the introduction to the gray area between, you know, what you're doing with Vibe Coding and and what a a nailed down process would be with, like, AI driven development type stuff.
Well, I asked the question because I I I read online some people discussing that. I don't know if it was Reddit or not, not, whatever. But, you know, some folks were saying that Vibe coding, the way they saw it, was, like, not really knowing software development very well, and it's awesome. So you can use this, you know, method to just whatever.
And you can generate some code, continually copy and paste the error problems and and eventually fix it and have a a working, working code. And then somebody else said, well, you know, if you really understand the code and you're just using it as as that assistant to generate it and you're able to go in because you have the experience and the knowledge. It's not vibe coding anymore. Now it's just like an autocomplete.
It's just syntax. It's not really it was an interesting argument. And it, again, it was a back and forth argument, that probably devolved into, like, name calling as these things do on Reddit, and people claiming that you're not a real developer at that point and things like that. But, that's why I brought it up because it is an interesting thing where it does get into these gray zones of, like, well, it's an assistant.
It's it's a you know, like, agents are. Right?
Where we have this agent that does the tool calling to to fulfill some sort of task. I could have done manually.
So in the same way, you know, vibe coding is something that helps you along the process. And if you're, you know, an experienced, knowledgeable developer, that helps you, you know, along the way to get to that to get to that, conclusion much faster.
You know, me personally, I'm one of those. I I I don't really subscribe to the whole yak shaving of what does a term mean in our industry or not. You know, we throw these terms around all the time and they they get meaningless quick. But I do like to pay attention to those types of discussions and conversations because those are the conversations where people bring up their ideas. They bring up problems. They bring up possible solutions.
They bring up possible areas for a new product or, you know, a a new feature that can be developed. People are this is where people as they're debating back and forth if vibe coding or whatever it is is a valid term or valid to use, that's where you see opportunity to make it better.
So, you know, I wanna get closer to wrapping up here. But before we do, Ryan, how can folks get started with, you know, vibe coding? Is it is it really just, firing up chat GPT and throwing your request for what you wanted to make, or is it more than that?
Yeah. There's it it really is just just find find some products that work great for you, that you kind of feel you mesh with. Some you'll notice really quickly, they seem weird and awkward. Others will have a very natural feel for you like, oh my gosh. This is amazing.
Find those. It takes a little bit of trial and error. The one that I prefer is called Cline. It's an open source free product, so it's great to jump in and try. It's built into v or it's not built in, it's an extension that you can install into v s code.
It does great. It's amazing. I love it, but there's also cursor, there's also replit.
I know a lot of people have been crazy impressed with, Claude Code.
So I I just encourage you get in there and try them all, pick one, nav or, figure out which one works best for you, and then just start doing. Like, throw stuff at it. You don't have to build anything that works. You don't have to really do anything important.
Like, something that watches your toaster for toast to pop up or something. It doesn't matter or, you know, build a financial tracking system. Whatever. The failure is the learning part and so that's the important stuff. So you just jump in and start doing and figure out how far you can go.
Yeah. I would also like to add that don't get hung up on models. Model selection is it can be important for sure. And I do prefer, like, o one for planning and then, you know, then there are other models that are a little bit, more adept at code.
But honestly, honestly, they're all pretty good. They really are. Yeah. So I I wouldn't get hung up on that.
Just start. And then later on, sure, you can use different models for different purposes and your workflow can have, you know, an LLM router even and start to do those kind of interesting things. But I wouldn't get hung up on that. If you're reading through, like, some of the Reddit threads about that, there's gonna be some heated discussions on exactly what model to use when, and everybody's a jerk that doesn't use the model I say.
That's silly. Just just get started like Ryan said. I think that's the best way to do it.
Evaluating and understanding what models do what best where is one of those that can eat up all of your time. And you're right. It it has very little meaning to the end goal of what you're trying to do. It's kinda like picking what database you use. Who cares? Pick the one that works, that vibes with you, and move forward.
And you can waste so much time. I I usually just keep, you know, conversations rolling with the people that spend all their time watching models, and I gather their opinion or I kinda just hear what the industry is saying. Three five Sonnet's great. Work with it. Roll with it.
Yeah. Yeah. I mean, there are considerations. Once you get into very sophisticated and complex workflows and its production environment, you know, you wanna take things into consideration like latency and, you know, dealing with, you know, real time telemetry things like that. I mean, there's there are considerations.
But that's down the road.
That's Yeah. Exactly. And then For sure. Those types of things usually appear pretty, clearly. So when you need to address it or when you need to think about it, you'll probably know already that you need to already that you need to do it. And so it's it's not that big of a deal.
I agree with you there. Oh, thank goodness. Alright.
It's the first time. Well, Ryan, it's been it's been great having you back on the, the podcast. Gotta do it again sometime. I definitely wanna get into, agents and MCP at a much deeper level and really discuss what that looks like, especially for IT operations, moving into the the rest of this year, which I know is not necessarily your big focus, these days, but certainly, something I know is important to, to our audience and our listeners. So Yeah. Ryan, thanks so much again for joining.
Really appreciate it.
Anytime. Anytime. I appreciate you having me.
Oh, yeah. So if you have an idea for an episode, I'd love to hear from you. You can reach out to us at telemetrynow@kentik.com. So for now, thanks so much for listening. Bye bye.