Shane Butler: Hey, everyone. Welcome, welcome, welcome. How’s it going?
Hai Guan: Hello.
Shane Butler: Let’s see.
Attendee: I haven’t gone.
Shane Butler: Hey, how’s it going?
Attendee: Good.
Shane Butler: Okay. Let’s, get the chat fired up a little bit. If you guys can hear me, can you just drop in the chat where you’re located? where you’re from. Maybe your role. Located in South Lake Tahoe, California. Southern California. Nice. We’re in Southern California. San Franc.
Attendee: Thousand Oaks, Ventura County.
Shane Butler: Oh, nice, nice.
Attendee: Yeah. Cool.
Shane Butler: My brother lives in Ventura.
Attendee: Nice.
Hai Guan: Director of Marketing. Also put in your role if you don’t mind. That’ll be… that’ll be cool. India and Croatia?
Shane Butler: Get the spread of the type of folks who are in here. San Diego, Boston, New York City. Okay, we’ll just give it one more minute as people roll in.
Hai Guan: Barcelona, UX researcher. Awesome. India. consultant based out of Toronto. Nice.
Shane Butler: Cool.
Hai Guan: Have a very wide variety of folks at this session.
Shane Butler: Yeah. Nice. Well… Austin, Germany. Sweet. Cool. Well, I know you’re all very busy people. Maybe we can just get started with some intros, hi, as people roll in. Get it going?
Hai Guan: Yeah, let’s do it. You want to go first?
Shane Butler: Sure, yeah. Hey everyone, I’m Shane, one of the co-founders of the AI Analy We… Primarily, I focus on research and education in the field of AI for analytics and data science. Been in the field of data science for a little over a decade, and in the past prior 2 years primarily focused on AI evals and Agentic analytics. Yeah, pass it over to Hai.
Hai Guan: Hey, everyone welcome. My name is Hai, one of the co-founders for AI Analyst Lab as well, and I’ve been in the data science and analytics space for roughly 20 years, and we’ve started doing this We have a bunch of free lessons, free courses, paid courses, all of that to help people stay at the forefront of leveraging AI in their day to day analytics workflows.
Shane Butler: Sravya.
Sravya Madipalli: Hello everyone, I’m Sravya Madipalli. I have around 14, 15 years of experience in data. Started off with Microsoft in Xbox as a hardware data scientist and moved on. to Ebay, then later next door. That’s where Shane High and I met, and most recently I was at Grammarly, which was renamed to superhuman. And yeah, so that’s about me very excited to have this set of folk looks like we have a varied set of audience today.
Shane Butler: Yeah, and if you guys have questions as we go along, drop them in the chat. Hive’s gonna be leading a lot of it today, but Stravi and I will be… Watching the chat or you can just raise your hand and. Come off mute and ask a question. If you turn your cameras on, it feels a little more like, We’re actually in a… In a lesson together, unless, like, we’re just, talking at a camera. But yeah. Hi, Attendee. Hey, Attendee, what’s up? We got another Tahoe person in the house, hell yeah.
Attendee: Super excited to be here. Yay.
Shane Butler: I just run into Attendee, on the road when I’m on my runs. I’ll just be like, that person looks familiar. Okay.
Hai Guan: That’s awesome.
Sravya Madipalli: Also, one quick thing, looks like we have the setup at 45 minutes, so we could stay over for any Q&A as well. Just wanted to let you all know so that you all have your hour planned.
Shane Butler: Yeah, if you got questions, we’ll have plenty of time for questions throughout and at the end, happy to stay over.
Hai Guan: Yep. Cool. Actually, Shane, do you want to give me permission to share?
Shane Butler: Yes. All right, you should be. Good to go.
Hai Guan: Alright. Can everyone see my screen here? Yep.
Shane Butler: Code 101.
Hai Guan: All right. One second. Let me walk back to the screen. Cool, alright. Well, welcome again, everyone. Thanks for joining us, and, today we are talking about Claude Code 101. We will go through some of the, building blocks of Claude Code in making workflows really efficient, and, these are And perhaps many of you might have heard of these, and we’ll do a poll in just a little bit around the components that we’re going to go through today, would be skills, agents, hooks. And then why they are different and how they beat sort of like just traditional chat windows if you’ve used, you know, probably more the more mainstream version of interface to AI.
And, so, roughly speaking, I think we’ll have, roughly, like, maybe, like, 10, 15 minutes on concepts, and, just a general general slides that we’re going to go through and then probably around 20, 25 minutes for some demos. And then hopefully we’ll go through those in that order and then the rest of it would be just fielding questions. So anything you have top of mind, feel free to drop them in the chat as well. Shane and Travia is gonna monitor them. And feel free to interrupt me as well if, if some questions could be relevant for the group here. Alright, cool. You move on. All right, cool. So just a quick poll before we get into the actual content.
So so that we have a sense of sort of like the polls in the room. Could you drop in the chat if sort of like like, where are you with clock code? So one, if you’ve never touched it, totally fine. Two, if you use the Claude desktop app, for example, and the code tab on that. Or three, if you’ve used it in Terminal.
Shane Butler: Nice. We got a good spread. Lots of ones and twos. A few threes.
Hai Guan: Okay. Nice. That’s awesome.
Shane Butler: You’re in the right place.
Hai Guan: Yep, you are definitely in the right place. Perfect. So, you know, like, I think today is built for, sort of, like, the foundational knowledge about, some of these components. We’ll demo it in… We’ll demo, sort of, like, how that actually works in the desktop app, so folks who are in it, who have used it, would probably look familiar, and then, for folks who have been using it in a terminal, or have joined us in prior sessions or courses and and sessions. Some of these might be might be a little basic for you, but otherwise, I think, still would be helpful to get a get a sense for what the how they actually relate to to each other. clicker doesn’t work. Okay, so let’s see.
So, I’m going to go through today’s content in the framing of a… of an actual use case, so that it feels a little bit more, sort of, like, practical. So imagine every Monday, in your, in your work, either you, yourself, or, someone you know, in the product team or in the data team, or whatever else needs to write a weekly update on metrics. So that’s probably pretty common right for folks here. Like, you might have come across, hey, I need to write up my weekly, status report around, around what I’m seeing last week on metrics and, some updates associated with that, commentary, and stuff like that.
Shane Butler: Anyone have to do that? And maybe to stay on chat, does anyone have to… anyone have some weekly… metric update they have to give. They really like , this is probably your favorite part of the week, given that.
Hai Guan: I’m looking for people who are going to say no here so that I can ask which company you work at. Cool, yeah, that’s, seems like, Seems like that’s a very common use case, right? So, you know, like one way to leverage AI for something like that is probably very common as well is use chatbots, right? Like, oh, I can chat with it. I can perhaps create some projects, create some custom GPTs, if you use ChatGPT, and but like, you know, every, every single week, you’re probably gonna have to, like, re-explain it to some extent to the AI about, hey, I want this in this, you know, same format. You might have to copy and paste prompts. You might have to copy and paste data in, things like that.
So, may not have the right connection to the different tools, and all that. So. you know, like, that’s still majority of workflows out there in terms of how people do, weekly updates, and certainly with some of these more advanced capabilities, like Claude code and, and things like that, through the different components that we’re gonna go through, you can… you can actually automate quite a bit of that. And, you know, I think… How they… how the different components relate to each other and, how you can actually use it in a efficient manner is, something that we’ll get into. But generally, the setup is weekly metrics review every Monday.
we don’t want people to kind of like having have to do that. from from scratch every time. Cool. Okay, so here’s a way to think about it, you know, in that same practical example of what a skill is, what an agent is, what a hook is. You can start to think of Claude Code, for example, as a new hire. So… For… for those who said 1, who are not familiar with what Claude Code is. Claude Code is, if you’re in a desktop app, it is a section where traditionally it’s when it first started, it was like for engineers to help them write code and stuff, but it can actually do way more and it can actually help you do a lot of the practical day to days, even if you write no code whatsoever.
So… So think of Claude Code as a new hire in that, you know. picture in your head with that exact same setup around weekly metrics update. Cloud code as a new hire. And so, what a skill is, is think of it as a playbook. So, how you actually want to do the task. They’re all in plain English everywhere, and you can define it once, and then it can keep it can keep the standard for you. So that’s kind of like the what a skill is. It’s the playbook. Think of it that way. Then the delegation piece, which is the second one. That’s the agent. So you can think of it as, like, your teammates.
Like, could be multiple teammates, could be teammates playing different, in different functions doing different tasks for you. But it’s something that you can think of as something, some someone that you can delegate to. And, you know, it will come back, you know, you can ask it to, hey, go do this, and then it’ll come back when it’s done. So that’s, roughly speaking, what an agent is. Somebody, like, almost like a worker. And then the third one, what is a hook? This is probably less talked about, at, kind of, like, outside. A hook, think of it as the rule. So, something that would always happen, and it is enforced deterministically.
So, things like, hey, I don’t want certain things to happen. Or I want something to happen, at some trigger condition. So, an example of that could be, hey, every time when I start Claude Code, I want, it to check my bank account, like, every time. And it will do that for you. That’s super extreme. But like you get, you get the, you get the idea. So, you know, like, think of it as, cloud code or large language models in, in and of themselves, could be, they could be smart. pretty capable new hires, but if you don’t equip it with playbook, no delegation, or no rules, then you can think of it as just a very confident intern.
So these components, the skills, agents, and hooks, effectively Help them become a really effective, you know, like helper or. I’m not going to use the word helper here. Think of them as like a very helpful companion. Let’s see. OK. So… Just going to go into a little bit more depth around what each of these are. So a skill is the playbook that I talked about. It’s a short file. It is typically written in plain English. There is no code. in our example that we’re gonna look through together, it would just say something like, hey, here’s how we do the weekly review, for example. Like, hey, here’s how I like it to be, here’s the, you know, different sections and stuff like that.
Here’s how you read the data, which data to read, how do you… compare? How do you format yourself? Things like that? And we’ll we’ll we’ll actually read the one that that we have prepared for you. in just a little bit. If there is not, you know, like, a skill here, then it really depends on how you prompt it, or, you know, right? Like, you have to… give it all those instructions in your prompt. If it doesn’t already have a playbook or a skill that it can already reference, then practically you have to, like, you know, do all of those heavy lifting in the prompt itself. So this makes it reusable and makes it consistent. The teammate here is the agent. So this is the the idea number 2.
teammate that you can delegate to, as I said just a little bit ago. And just like the skill, you also describe this in plain English. So something like, you know, hey, this is your job. This is how you should think. This is what good answers look like. And so, you know, like, almost like a different pair of hands helping you on whatever tasks or or goal that you’re trying to accomplish. And in the demo that we’re going to go through pretty quickly, pretty soon, the agent here is a VP. someone very senior, at a… at a company, so the agent that we’re gonna look at is a pretend, let’s not call it VP, but a pretend VP, but somebody who’s gonna be very critical about a weekly report.
so that we can sort of like, you know, like, not just prep the report, but also anticipate really hard questions off of those. Okay, cool. And, let’s see. So, the hook, which is the third concept, or the third component, this one is, as I said before, not as well-known, many people don’t use it. But effectively, a hook is a rule that fires automatically and and and and always. So it’s not kind of like a a note in your prompt. It’s it’s not something that you hope that it does. It checks every single time if you, if it hits certain trigger condition.
So, again, the example I gave earlier is, You can insert a hook when you start a session, and you know, and for me I would have a hook every session to see if there’s any update on the on the folder that I’m working on. So the repo that I’m working on, and have it automatically sync with whatever update that perhaps Shane and Sravya have been making on a repo and a repository that we’ve been working together on. So you can insert custom rules just like that through hook, and you can define it at different points in in in your workflow. And, becomes pretty flexible. So, you know, it is one guardrail that you can, you can leverage it as a guardrail, if you wanted to.
So in our case, what we’re going to see is that in the demo, we’ll have sort of like, you know, a hook. go through some of the basic data quality checks, such that, the, you know, like, it will always flag and always stop when, when the weekly report makes no sense, for example, if the underlying data is, is, is, is wrong.
Shane Butler: Yeah, one of the biggest failure modes of, say, like, a skill is that it doesn’t fire, it doesn’t trigger when it should. There’s some probability, right, with a non-deterministic system that It’s just not gonna fire. Especially if you have a pretty large system or, like, a, like, overly large, complicated Claude MD file or something. So, a hook is a really nice solution when you have some sort of task that that must happen, that if it doesn’t happen, there’s a big risk to the output.
That being said, because it is more deterministic, you kind of lose some of the, yeah, I don’t know, the reasoning logic of like the system itself trying to figure out when something should work and something doesn’t. So you don’t want to have everything in hooks because you don’t always need all this stuff firing.
Hai Guan: I see a question from Attendee about whether a hook is what people are calling a harness today. So harness, think of it as. referring to the whole package. So, like, all the skills, all the agents, all the hooks, all the helper functions, all the instructions in a given folder that you give to Claude, that’s… the harness. That’s almost like a system that instructs Claude to behave a certain way, and has certain standards, and does things different, in, you know, like this, this way or that way. That, that is, that is what they’re referring to. Like, the entire system is the harness. So we’ll we’ll we’ll have a harness here as well. It’s it’s a pretty basic one.
But the idea is but the idea carries over.
Sravya Madipalli: So there’s a question from Attendee around, “Can Hook be used as an agent governance function? How does it tie to eval.
Hai Guan: Can hook be used as agent governance function? How does it tie to eval? So You can think of a hook as sort of like one tool that you can use. in whatever goal you’re trying to accomplish. So, if you want the agent to be, like Shane said before, to behave certain ways, and you want additional guardrail, for example, you can leverage hooks, if that makes sense for your use case. So, for example, like. In the example that you’re gonna see, like.
It will not allow the agents, or… yeah, it will not allow the agent to write a report if… it senses that, like, when… when the agent wants to write a report, it will automatically go in and, like, check some of the data quality stuff, and refuse to let the agent write the report itself. If if if some of these don’t check out so you can, you can leverage that as like a like a tool in the toolbox kind of thing.
Shane Butler: I’ll say as it pertains to evals, you could create a hook that kicks off some sort of , eval score, you’re probably not going to be running to run, like, your evals in real time all the time when you’re going through your process. One of the ways I leverage it for is more around traceability. So rather than having to go through the past transcripts or ask Claude itself. I have, like, hooks that basically whenever there is a lot of the queries, I use Snowflake, so whenever there’s a tool call to Snowflake to Run a query that kicks off a hook to record. the query that was actually written, the timestamp, assign an ID to it, assign the context around the question I’m asking, and then later on.
when I do run my evals, I’m actually able to trace back to the exact SQL I knew that ran in code. I could get to the same thing if I went, like, into… into Snowflake and looked at my query history or something, but that’s… that’s one way I do use it for evals. It’s more about traceability.
Hai Guan: Cool. I know there’s a bunch of questions. I think those would be answered once you see the demo, so we’ll get right into it. Okay, so, cool. So here’s a… here’s the setup, think of this company called Brewly. This is a coffee subscription app. And, what we’ll see is that, it has daily signups, activations, revenue, and, you know, it’s Monday morning. Pretend today is Monday morning. Which means, somebody owes the team a weekly review. So, we’re gonna look through a few things, together. We’re gonna run… So it comes with a skill, agent, and hook. We’re gonna run the skill, we’re gonna change something within that skill, and then try to rerun it and see what happens.
We will have my pretend VP read the report, and then kind of like grill it. And we’ll we’ll fat finger some of the data, and then we’ll see what the hook does. So it should be… should be pretty fun. Let me switch over here. Okay. So, you should now see my Claude, or, yeah, Claude desktop. You guys see this? Blank screen, almost. Cool. Awesome. So, for those who are not familiar, you can download the app from Claude. You just type Claude in your app store, you should be able to find it, and then up top here, there is a tab for code. This is where Claude Code is. if you are on the desktop app.
There is terminal option as well, and that’s where we teach some of the more advanced stuff, but we’ll talk a little bit about that after we see it in In the desktop app. So, let’s see. So, how does Claude Code work? Claude Code basically sits on top of a folder that you give it, so it has all the context within that folder. That folder is practically your harness, If you will, it’s, I’ll show you where this one is. So this is my folder for this. This this really kind of like company. It’s called CC. 101 demo. But this is the folder, and within it We can see that, I have You know, I have an agents folder, and this is sort of like the agent that we’re gonna take a look at.
This is the executive reviewer, kind of like my pretend VP. We have skills here, so it has gotten a folder on skill, which is the weekly review, skill, which will… would have instructions for how to actually write a weekly review report, and then it’s got some data, so… So, you know, it’s got a… it’s got some data, and I’ll show you what the… what that looks like. It’s got some data around, ad spend for Brewly, the… this, coffee subscription company, so pretty simple. Just show you this. you know, every single day. It’s got a channel for paid social, and how much spend was was put in. And you know, there’s this gap here. I’m just, you know… We’ll see Claude kind of flag that for us.
And and then we have another data set around some of the metrics that I alluded to, which is again pretty simple. It would just be, you know, by channel, how many signups, how many activations, what, how much revenue we’re getting from, from these on a daily, on a daily basis. So just very simple setup, pretty bare in terms of you know, like, the different, files and systems. But what we’re here to do is, just showcase how these things work in a very, in a very contained way. So, on clock code, then, what I do is… I 1st pointed at that folder specifically, so that it knows what we’re what we’re doing here. So remember the CC. 101 demo. This is basically where you point it to.
So local And then here is, where you can actually open a folder, and so, I have already pointed it at that folder that I showed you, CC101Demo, so it has access to all the… all the files within that folder, so it knows all the context there. Now, I’m just going to say I want a weekly review report. So what I can do to trigger a skill, I can do slash weekly review. Now. And then press Enter. So how does it know that it is looking out for the weekly review skill? Again, if I come back to the folder here, it is under skills.
I mean, it’s grayed out because it’s hidden files, and we’re working on desktop app, so there’s not a easy way to sort of navigate, but imagine This is the skills folder, a little hard to read, and then within that, there’s a weekly review skill here. And while it’s working, I’ll show you what that looks like. so I can open the text. So this is just the text editor. So these are skill.md, md means markdown. Think of it as just text files that AI can read pretty well. So… Hopefully you can see this. If you can’t, you can do command plus, or actually I can do it myself. So the skill itself is pretty. It’s pretty simple. So description generate release weekly metric review from this data sets.
And then the instructions are… You’re writing… Brulee’s Monday morning metrics update same format every week. So read this first, st and then do the you know, like, how do you compare in terms of which day to which day compared to last completed week. Write a report to this Folder here, which is… just right here under reports. Just write it here. That’s basically what it’s saying. And then, you know, this is the format or the standard that you’re giving, AI, to… to output, so TLDR, the number, what moved, one risk, one action, and then keep the whole report under 250 words, numbers only, no charts, and then round percentages to one decimal place. So that’s a that’s a skill.
And so, once this guy… so I just basically did nothing, but ran it, right? So I triggered it, and then… It went to analyze the data, the metrics.csv file that we talked, that we saw, and, it outputs this, report over here. And I can click that. and then on the right panel. This is the output. So you know. it’s got the Tldr. It’s got numbers. It’s got what moved and one risk and one action. So basically, what it’s flagging is, hey, there’s a double digit decrease in signups, activations, and revenue. But, versus the week before. But then the entire decline in here traces to 2 days when paid social recorded almost no activity. While organic and referral held steady.
So we sort of saw that like just manually. with our own eyes in the in the Csv file. but it’s able to like. That’s kind of like the the punchline that it gave from this skill right here. Oops. Okay, cool. So, I am going to… Let’s see… I’m going to change this a little bit, this skill a little bit, so you can see… What that… Looks like. So if I go back to this skill file here. And I’ll just open it up again. So instead of, for example, like, you know, point number 4, keep the whole report under 250 words. I can, let’s say, modify it a little bit. So, open the report with a status light, green, yellow, or red, and one sentence on why.
So, I’m just gonna save this, so I modify the skill directly myself. And I’m going to run the same report again. So I’ll clear context first. So what I just did is slash clear. What that means is, I just wipe out the the conversation that we had, so that Claude can start fresh again. Weekly review. Oh, and I need to delete the previous copy here. Okay. So I’m running again the the same trigger. So triggering the same skill. And then it’s gonna do the same thing. But this time it should adhere to the standard, an updated standard that we gave it. Still thinking, still thinking.
Shane Butler: You can do it. I believe in you.
Sravya Madipalli: Maybe we could answer, one question here before. So, how was the folder created, and where did everything in CC101 demo folder come from? I just said we’ll answer quickly, live, but…
Hai Guan: Cool. Yeah, so for this lesson, I worked with Claude to create something that we can demo together around this fictional company. So Claude created the, sort of, like, the entire setup around, the skill, the agent, and, and the dataset.
Shane Butler: Hey, hi, would you consider yourself a coder?
Hai Guan: Would I consider myself a coder? No, absolutely not.
Shane Butler: Then ask the question, if you need to know how to use Claude code. Oh, no, not all of them. No, you don’t. It helps to review code, for sure, when it’s writing. But no, you don’t.
Sravya Madipalli: Also, there’s multiple ways of reviewing things. Yes, if you know code, how to code, it’s easier for you to review code. But if you know how to check the output of the code. That also is a great way to review things. So that’s the reason why it’s a lot more easier to build with Claude Code now, because you don’t need to know how to code, but you need to know how to validate the output of the code and how fast it took or have good analytical questions to ask Claude Code to help you understand if the code’s doing the right job or not.
Shane Butler: Yeah, and you can validate or review that output manually, or you can build automated systems. You may have heard of the term AI evals to create actual metric scores around. their reliability and trustworthiness of the output as well.
Hai Guan: Yeah, totally. Okay, cool. So the same skill ran, I mean, the same skill with an updated instruction on point number 4 ran, right? So now it gave us the… status light. So it determines that this should be a yellow. The headline numbers are down double digits, but that’s because of this gap, and it’s totally explainable. So Claude thinks this should be just a yellow, not a red, or something to be alarmed by. So now what I’m going to do is, let’s see, I am going to ask it. So I’m just going to exit out of this. Have the exec reviewer… Agents read the latest review. the way my VP would. So I’m going to ask it to basically trigger the agent.
That’s also in the same folder here, so that I can prep for tough questions that could be coming from Vps reading this report. Now again, so it’s going to go do its thing. This agent takes on this persona here, which I’ll show you. Okay, so back to… while it works, we’re back to this folder here. You can always inspect your, all these, agent files, skill files, because they’re just English. So you can… so I’m gonna open again in text edit. And… make a bigger. So this is the agent for… this guy. So, this agent is called ExecReviewer, and the… what it does is it reads the report the way a busy executive would, and flags the questions you’ll get asked.
Use it before sending anything upward, so… This is sort of like instructions for, for Claude to behave like this. You are really tough. Executive reader, act like the VP of growth, no patience for fluff. Who here deals with folks like this at your company? Or are this at your company? Nice. Cool. So, pretty common. I saw a lot of hands up. Yeah, so it would, read the way she would, skim the TLDR, numbers table. find the biggest unexplained change, demand… demand Hawaii, ask what we’re doing about it, flag any number that invites follow-up nobody in the room can answer, and then report back with exactly this. So you can also give it some behavior. So, three questions you’ll get asked.
one thing to fix before you you you hit send. So that’s that’s the idea. So again, all English. So let’s go back to it. Alright. So our exec reviewer, our pretend Vp. Has done its thing. So verify the numbers caught a real flaw in my framing. So So the three , let me see. All 3… yeah, so, let’s see. The questions. Let’s get to the questions. Lost or just miscounted. So this is the question from the VP. Like, did we actually lose those signups? Did we actually spend, or did we not? why did organic hit weekly highs the same two days?
So in the… in the actual dataset itself, it saw that, Within those two days, when… paid social, or, paid channel was, was gone, that, organic hits, organic saw a little bit of a bump there. So, it’s basically giving kind of, like, questions around, some of the stuff that’s unexplained in the report itself. And Claude is almost like, you know, it gets pretty meta. It’s like, oh, this agent tells me that I did these things insufficiently. And so, you know, now it says, okay, based on what the agent told me, I recommend that I fix this report. this way before it goes out.
So, you know, from here on, I can always say like, oh, yeah, go answer these questions directly, or, you know, like, do some other things to sort of, like, help you, help me prepare for these questions ahead of time.
Shane Butler: Yeah, we have a bunch of questions in the chat. Or if you finish your thought and then I’ll, I’ll, I have some questions for you.
Hai Guan: No, go ahead. I’m going to go to hooks right now. So if there’s relevant questions around this.
Shane Butler: Cool, I think this, so, two questions that are kind of linked together. Attendee asks, is that an agent or a skill? And then Attendee asks, how do you decide whether to build a skill file or an agent file?
Hai Guan: Yeah, that’s a great question. So this exec reviewer, if you recall, is an agent. The way to think about it is, like. Again, agents could be your… like, think of it as, like, your workers, so things that accomplish very specific set of tasks. and can take on very specific personas. Like the teammate was the was the was the analogy. I think some examples, for and this gets this is pretty like the line could blur because an agent could use skills. A skill could call an agent. An agent could call multiple agents and stuff like that. the way to think about it is some common uses of skills, for example, would be, you know, for example, here is a VP, like a VP agent, so it’s got its own context.
It will figure out, kind of like, hey, based on the instructions, I’m gonna do these things this way. So it’s almost, like, independent, of… everything else that’s going on. Some other… very common agents that people spend a lot ourselves included would be, you know, maybe a reviewer, maybe a planner, maybe a writer, maybe a researcher, something like that. So like they take on those specific sort of like personas and and and go do it specifically just themselves. Anything you would add there, Shane.
Shane Butler: No, I don’t think so. I had a couple more questions from the chat around… the way you’re accessing data. So if your data is connected via Snowflake MCP, How would that live in the folder or be? pulled in. But… And then, yeah, maybe that’s the first one. I can answer that, actually, for the… for the Snowflake, MCP, yeah. Yeah, all that analysis, all the queries, and everything that’s gonna be ran is gonna be run in Snowflake’s hosted environment. It’s not, like, ran on your machine. And then the output of those, if you’re going to do further analysis on that, those can be pulled into your folder.
You could have the results About any level of those queries pulled into your folder and, like, saved in, like. CSV or JSON or something like that. But, for the most part, it’s just ran in Snowflake, and then whatever the kind of answer is, is then, Store it and say, like, you generate, like, a deck or a chart or something. Yeah.
Sravya Madipalli: There’s a question by Attendee and also a question in Q&A by Attendee. This is probably around the context. This is around how does Claude know how does a certain position would respond? And that’s Attendee and the detail there. And even Attendee’s question is, can it answer all questions of the VP without the organizational context? So both are related to how does it know the information about how a VP would respond and the details and context around it.
Hai Guan: Yeah, so quickly answering that is the more context the better. Obviously like it’s not gonna like write what we, what we read together on what the VP kind of like file agent file looks like is relatively generic, right? It’s just, hey, you’re the toughest grader or the toughest, what was it, critic. Of the business and stuff like that. And you know, as specific as… it only got to as specific as, oh, you’re the VP of Growth. But you can imagine, you can have it be… you know, hey, this VP is, very short, very short-tempered, very, you know, like, very sensitive to numbers, very, you know, this or that, cares about these things.
Like, those would be contexts that you can add, in this example here. You know it’s it stays pretty basic just so it doesn’t sort of like blow up the blow up the thing. Alright. So I’m gonna teach, or I’m gonna go through an example of a hook, and then we’ll take QA. cool. So I am going back to the data set like the Csv file. So I’m just gonna you know, pretend that I just fat fingered something right? So let’s call 7272. I’m gonna change this to, let’s say, like 900, something like that. And I’m saving, and I just saved this file. Or save this change. And then I will be here and let me delete the report as well. And then I’ll run it again and see what happens. So I’ll do clear context.
So all this conversation that we had does not pollute. what it’s gonna do. So I’m just gonna run weekly review again. And then this time it should not allow me to write the report directly because we fat fingered something and it makes no sense in the data. And so while it works, we can also look at the hook. So hooks lives under dot clog. hook, or hooks. It’s a Python file. But again, the first block of it, you don’t need to know Python to read it. Okay. So, this is… pretty much the description of this guy. So Brulee data integrity gates. This is called check data. This file. So before any report is written, validate no duplicate date channel rows.
Activation can never exceed signups, and we just fat fingered something where activation exceeded signup. So it should flag that no negative or malformed values. So that’s that’s that’s all. That’s all it does. And then it has these Python codes to make sure it’s it does that. And it… It is trying to compute and write the report. And then And then it should flag it. And… While it’s doing that, let’s check on time. Okay. Let me quickly go to… terminal here. So obviously what we’ve been. Oh, well, here we go. Okay, so now. it is, it says, fail to write this file that it’s trying to write the report. Because a data integrity hook blocked it from shipping.
And so this is correct, because it did the second check there, where activations cannot exceed sign-up, and then we just fat-fingered ourselves, to that. So, effectively, this is the guardrail that, that got triggered. And so that’s what I wanted to show here in terms of how that works in the whole system. it’s a pretty basic system, like, very simple, right? Only 3 things, 1 skill, 1 agent, 1 hook. You can imagine it sort of, like, fanning out, depending on whatever workflow you’re trying to automate. And for us, we do analysis, and we do a lot of, number crunching and, understanding of data.
So, what I’m gonna show you is sort of like, the, the… the open source system called AI analyst that that we open sourced. That does a lot more on just you know, making sure that Claude behaves in a way that that incorporates it, incorporates all the best best practices of doing analysis and and and all that kind of stuff. So I’ll do that quickly, and then we’ll… Finish off, and we’ll go into Q& Cool, okay, so for those who might have seen Terminal, this is Terminal, Claude Code, and Terminal, and this is the system, or the harness, that we have, that we open source, so feel free to, feel free to go, you know, grab it, take a look, if, if… if you’re interested.
And, it’s got a bunch of skills around… so what this is, is, it… it does analysis end-to-end, just like, how myself, how Shane, how Sravya would in different tech companies. So… So it’s got like a bunch of guardrails, a bunch of skills on how to do certain things, a bunch of agents to actually do the things. And generally speaking, what it does is it takes a business question in, and then it gives a finished output of the analysis itself back out, either in the form of a deck or a one pager or anything like that.
I’m not going to go super deep into it, but you can imagine, based on what we just shown in cloud code desktop like you can start kind of like making the thing more tailored to your use case, make it more sophisticated so that it tackles the things that that you care about. And so that’s that’s really the idea there. Okay. Cool. So… I’m gonna skip that. So, I think this slide is worth going through, like, you know, like, why, why do all these things matter, right? Like, why, why are we all here today? Why do we want to even learn about this? Can’t we just do it, you know, traditionally, in, in our, in our, in our own ways? The thing that we tell people now is that every analysis is a bet.
And the impact is how many bets you place, times how often they win, times how big the win is. So it’s almost, like, mathematical, right? Like, 3 components, and… being able to build… use AI, build with AI in the way that accomplishes, you know, what we just… kind of, like, the different components that we learned about, actually helps you with all three of these components. And so, you know, obviously, the first one is, like, you just get more shots on goal, because now things that used to take, let’s call, like, an hour to do weekly review now takes, you know, one click, or, like, you know, five minutes or something like that, because now you have a system to help you do that.
So just by definition, you have… you’re much faster, you still have to review the outputs and stuff. But in general it’s going to be much less than if you do it manually. So speed is a pretty obvious one. The second one is your hit rate hopefully should go up as well, which, because AI can now do a lot more of the sort of like the slicing and dicing. So the depth that you are able to do, like the depth and quality, should also go up because of the system that you are able to build with AI. And the third is sort of like the size of, of any given bet that you, that you decide to, you know, analyze or, or look into. Traditionally.
when… your execution capability is limited, or your team’s capability is limited, because, you know, because of efforts, because it takes long, it takes a while. Now, like. in that world, a lot of times, we bias towards something that’s more of a sure win. Like, we can see that, oh, this would probably yield pretty good results, so we’re just gonna default to that, and we don’t go for the venture bets and stuff. Now with AI, you can actually go look at the venture bets, because everything else is compressed, and you can now, you know, take a swing at the home runs. You can afford to do that. And so all three of these components would actually just multiply in and of themselves.
If you’re able to leverage AI and also build some of these, you know, for us, it’s an analysis. So like Agentic analytic systems. Cool. This is the last slide, which is… we have, we have an Agentic Analytics 101, boot camp coming up, next week. This is 2 hours a day, live, from Monday to Friday next week, where we teach how to actually build something that’s tailored to your, you know, like your… or getting… acquiring the skills to build something that you can tailor to your specific company needs. So, we’re gonna go through how to design an Agentic system around your own work.
we’re gonna go through how to actually write your own skills and agents, obviously in plain English, we didn’t see any code there, connected to your tools, so your, you know, like, different things that we’re gonna showcase would be things like Notion, Snowflake, stuff like that, and, by the end of it. You’re gonna be leaving with your working system that, on your laptop, and so it’s kind of like the skill sets that we teach to get people equipped to be able to sort of like build anything you want to really automate your own workflows, especially in the domain of analytics and data.
Shane Butler: Yeah, it’s really cool. I think one of the things like anyone like, a lot of people are starting to use cloud code now, a lot of people are pressured by their leadership as well to, like, oh, you gotta use cloud code, you gotta use AI or whatever. Meanwhile, those people have no freaking idea what to do with it. They’re like, I hope this, like, AI thing works, because spending hundreds of thousands of dollars each month on it.
So this is one of those things we have people come through our courses, and, they go from just, like, basically using it like a chat back and forth to, like, hey, I’ve actually built an Agentic system that is totally customized to the goals we’re trying to accomplish with our team. Yeah. And helping us make better decisions. So, like, anyone can use data to… to, like, empower their workflows, whether you’re a data scientist, or a product manager, or a banking, or… we had a chef come in and join us a couple of cohorts ago, even. I’m not sure what they’re using it for, but that was pretty cool. So yeah, it’s pretty good.
The other thing that we go into that a lot of our courses don’t go into is we go into validation and evals. So we teach a bit around not only how do you build an Agentic system that outputs analysis for you, that outputs answers for you, but how do you actually build systems that validate whether. that output is reliable or trustworthy, because it’s kind of pointless if the AI is outputting stuff, then you have to go, like, do it all by hand. Or not by hand, but manually, the old way anyways, to even see if it works or not. We do have a lot of questions. Should we , .
Hai Guan: My turn.
Sravya Madipalli: There’s a question about if this is like an advanced course versus when people are just starting out, would that still be okay?
Shane Butler: 101 is our non-advanced course. Yeah, we’ve had people go through , so we have a bunch of tiers of courses. 101’s our most introductory course. You don’t need to know how to code. You don’t need to have used Claude Code before. You do have to have a Claude Code subscription. That’s kind of basically it, honestly. And then we have a 201 course, which I think right now is. set up in August or September, which is a multi-week course, which goes a lot deeper into advanced topics. We usually recommend people take the 101 first before jumping straight into that one, but you don’t necessarily have to. Okay.
Sravya Madipalli: Attendee had a great question about creating skills agents to parse qualitative data, and they were asking if this is something that you know that would be taught in the course, or maybe. Only if you’re here, you could ask it live, too.
Hai Guan: Parse qualitative data, how different, similar to the demo that you shared today. We’ll teach you the foundations for, let’s say, like, like, how to, you know, like, again, as we talked about, skills is just a standard that we give it. So, for example, you know, like. So for quality data, qualitative data, you might want to you might want, you know, like, Claude to parse it in these certain sentiments, or these certain, qualitative dimensions, or something like that. So, you can give it that, example. And things like that.
So, you know, we go through how to build your first skill, we go through how to chain skills and agents together, we go through, like, you know, in the thinking of the entire system itself. So this sounds like it could be just… could be a part of, sort of, like, the bigger system design thinking. That, that you can make it much richer or much simpler if you want to.
Shane Butler: Yeah, it’s really good at working with qualitative data, to be honest. So, one of the things, I don’t know if other people have this, but one of the things I always hear Or I sort of, my work is like, hey, you gotta, like, you need to be on, like, the customer calls more, you need to be on these UX calls, and, I enjoy being on them, but honestly, I have a bunch of other shit to do.
And, one thing that I’ve found is, so we use… gong that records transcripts of all of our customer calls, all of our UX research calls, and I went through, and I had Claude going through parts, like, 1200 customer calls, and kind of just go on a loop on itself of, like, hey, start forming hypotheses around, this is kind of what I think people are thinking about this product, but go in and see if you can basically collaborate that, or, like, take an adversarial approach, and try and, disprove it, and then just start, like. Identifying, like, people who are, like. for or against whatever this hypothesis, and then source all that information for me.
I think the cool thing with the qualitative stuff is you can summarize it, but if you have hooks Set up for every time it does, like, a summary, You can also have Claude store like a really nice kind of research hub where it’s sourcing back to the original stuff as well for you to validate.
Hai Guan: And I see some questions around, you know, like, you know, do we recommend using Claude Code or other Claude products? The way to think about it is… chat, like the chatbot, if you go to claude.ai, just like the interface or ChatGPT, think of it as, like, a question-answer thing. So if you ask a question, it’ll answer you back. That’s, that’s sort of like the, the, the, the, the model, the mental model there. Cowork, which I think many people have referenced here, is If you want Claude to do something for you, so… you know, perhaps, oh, I need to clean this spreadsheet, or I need to organize these folders, you can use Cowork to do that for you.
Claude Code is where you actually build stuff, so you can build systems, you can build, the… the thing that we just, that we just looked at, like the, you know, CC101 demo, the entire thing. You can build it in… that’s where you… that’s where, you typically build things, that would then, you can use, and improve on, and, keep iterating, afterwards. So, think of, think of it as, if you know cloud code, and again, you don’t need to code, you don’t need to learn how to code, I don’t know how to code. You basically know everything. that predates that in in that whole in that whole chain.
Shane Butler: a very out-of-left-field example of not needing to know how to code to use this is my wife is a ceramics artist, and she uses Claude code to help her know at what, like, degrees Fahrenheit she should fire certain pots, and also to build templates to make different types of, like, mugs, and also to build herself, like, a boot camp to get better at, like, using the wheel. It’s like, it’s crazy, the stuff you can apply it to. Obviously, none of that’s what we’re going to be covering in our course. But for the ceramic artists here. She has no idea how to code, and she’s using it.
Hai Guan: We have both.
Shane Butler: Cheers now.
Hai Guan: On ceramics, actually. Don’t tell me.
Shane Butler: Hey, she made this mug. Thanks, Claude.
Hai Guan: Wow. Oh, man.
Shane Butler: No, this is pre-cloud code. This is pre-cloud code. This is all human.
Hai Guan: All right, cool. We are at the top of the hour. I think we’re 15 minutes over. Does anyone have any burning questions that you wanna ask? Maybe we’ll take one or two more before we close.
Attendee: Can I just ask real quick? Based on what you said, hi, and is that Claude Cowork would probably be better for me if I need to build, like, status reports or project plans, is that what I would use, or use Claude code to do that? And just try to figure out, I would try to figure out what the best way to do that would be.
Hai Guan: Status report. Yeah, so you can… You can do co-work. I think it can help you build some pretty simple stuff. If your, you know, like, if your status report involves, for example, chaining together a lot of tools, a lot of different agents, you want it to have some sort of, you know, steps in between that reviews Things for you that does, you know, like, delegation of, splitting, like, responsibilities, and, and, and each one of them can have different behaviors and stuff. Like the more involved the thing is, the less. Cowork, for example, is going to be able to Help you in getting those out.
Attendee: Okay. So, the more detail I want to code is better, the just more basic code work would work.
Hai Guan: Yeah, cowork is really good at if you already have something for it to just do, like, work on.
Attendee: Wom.
Hai Guan: Like, oh, if you already have a system, you know, imagine the demo that we did. If it’s like very built out, more involved, you can use Cowork on top of that. And it will be like, yeah, it’s pretty good, for example.
Attendee: Cool. Thank you. Appreciate it. Great. Great session, guys. I appreciate it.
Hai Guan: No, that’.
Shane Butler: All great questions. If people have other questions, in the meantime, we have a Slack channel. I’m gonna drop a link to to join. One sec. So… Yeah, you can join here. If you want to join our Slack, it’s public chat, our public channel. There’s about a thousand people in there, so people who are working with AI, from beginners to advanced, people who have taken our courses, people who haven’t. We drop a lot of material and learnings there as we come along. We also, it’s also a good way to keep up with any of the free workshops we’re hosting, but Do you have any questions there, too? We’re on it all day, so happy to answer there. Or you can DM us on LinkedIn. Or email us.
At hello at AI Analyst Lab as well. I think we actually had a couple. Sign-ups for our course in the past 20 minutes from here, so we’ll see some of you there.
Hai Guan: Cool. Awesome. Welcome. And then after this, I’ll send out both the, like, I’ll send out the slides and also the, the, the demo folder. I think that there’s a lot of, interest in looking over what’s in it, yourself, so I’ll send that out as well, and, and a recap and recording and all that kind of stuff.
Shane Butler: Cool. Thanks, everyone.
Hai Guan: Awesome. Thanks, everybody.