Check out the conversation on Apple, Spotify and YouTube.
Introduction (0:00)
Aakash: So what questions are you actually asked in the AI PM technical interview rounds at top companies like OpenAI, Anthropic, Google, and Meta? The fastest way to end a PM interview is to freeze on one question. Do you use an LLM or an ML model?
Meet Prasad Reddy, former CPO at L-Nutra, senior director at Danaher, and someone who has coached over 150 PMs over the last 12 months in the job search. Almost all PMs know how to use ChatGPT and Claude Code, but none of them understand how this works underneath. Understanding these technical concepts is the key to nailing your AI PM interviews.
Most people think an agent is just a prompt, because they’ve been trained by custom GPTs. Really it’s prompt plus tool plus skill.
I’ve been coaching one of our cohort members who landed a role as VP of Product at an AI startup. The questions she was asked were how does this work, how does that work. These are the key concepts in nailing AI PM interviews.
Where these five concepts came from (1:12)
Aakash: One of our cohort members interviewed at NVIDIA and Glean and said she was asked these types of questions. So today Prasad and I are not just talking about these questions or listing them out. We are going to teach you the fundamental concepts you need to understand, and teach you how to answer these questions.
So what questions are you actually asked in the AI PM technical interview rounds at top companies like OpenAI, Anthropic, Google, Meta, Sierra? The list goes on and on. We used the database we’ve collected from our cohort members to come up with some of the top questions you have to understand. How does an LLM actually work? How does function or tool calling work? How does LangChain work? How does routing work? How would you define agent skills? If you stay till the end of this video, we will teach you how to answer all five of these, starting with first principles.
The typical way people answer this question is the reality, which is, hey, I do use AI for that. The much better way to talk about it is, here’s the model, here’s the call, and here’s how I’d route it. You want to be as specific as possible. Show that you don’t have just a business PM understanding, but a technical PM understanding. You need to be able to speak to model choice. You need to show you were actually in the room. You don’t want the interviewer to write down that you were just adjacent to the AI decisions and not driving them. To do that, you have to understand how these things actually work at a fundamental level.
We’re going to break this video down into five key concepts, and we’re not only going to teach them, we’re going to demo them. We’ll start with LLMs, and help you understand what next token prediction is. We’ll go into tools and function calling. We’ll cover skills and agent capability. We’ll cover orchestration with LangChain. And we’ll cover routing, how you choose the right model for the right job.
Prasad: Looking at these concepts, they might be overwhelming and sound too technical, but trust us, we’re going to break them down and simplify them over the next 30 to 45 minutes.
How an LLM actually works (3:17)
Aakash: Let’s start with how an LLM actually works. Fundamentally, LLMs are predicting the next token. When you ask it the capital of France is, it’s not working like a dictionary or a Google search. It’s figuring out which of these, Paris, France, located, the, is the right thing. The model is sampling from this distribution. It looks at it and gives Paris a 0.92 rating. That’s the highest rating. That’s the next token.
Now, how does it decide amongst these? There’s a temperature setting. Temperature sets how risky the pick is. It’s not really retrieval, it’s just a probability. That’s the very first thing to understand about LLMs.
There are a couple of things I want you to understand beyond that. Temperature is one. Low temperature means it’s going to be safe and repeatable. High temperature means it’s going to be creative and varied. That’s essentially the sampling dial.
Then there’s context window. Fundamentally, we need to understand that LLMs go through a training process comprised of pre-training and post-training, and at that point the weights are frozen. Once you actually get to a user querying, there’s a certain context window an LLM has access to. A lot of modern models, take Opus 4.8, have a 1 million token context window. This was made really popular by Gemini 2, which released with a 1 million context window nearly a year and a half ago. So within a context window, that is fundamentally on top of what the training data has that an LLM has access to. It defines the boundaries of what it’s looking at.
The final concept to really understand is hallucination. LLMs are doing that next token prediction we just talked about. The capital of France is France even gets a 0.02 probability rating. In order to reduce it from choosing the capital of France as France, we add in things like retrieval and tool calling. These prevent it from making confident decisions that are actually wrong.
So that’s the theory. Let me actually demo this for you live.
Demo, temperature in Google AI Studio (5:24)
Aakash: Let’s get into Google AI Studio and see temperature in action. We’re going to ask it to name an animal with temperature zero. It tried capybara. So theoretically, under temperature zero, if this is true, we should get capybara again.
Now, under temperature one, we should be less likely to get the exact same animal, because capybara shouldn’t dominate the distribution. Let’s see. We’ll try it once. We got, at temperature one, capybara. Now if we do it again and again, theoretically under temperature one it should be more likely to change. And if we go even further beyond temperature one, all the way up to two, here we see it’s chosen axolotl.
So what’s happened? This is the intuition you need to understand. At temperature equals one, you sample from the model’s native distribution. At temperature less than one, you sharpen it. The top token’s lead grows, and we saw capybara repeat. At temperature greater than one, you flatten it. The gap between the favorite and the rest narrows. And so one time you actually get a non-deterministic response like axolotl.
Another way to prevent hallucination, like we talked about, is adding tool calling. A tool calling example might be grounding with Google Search. Even at a temperature of two, if we do something like the population of the USA, in less than five words, we’re going to see the population is approximately 349 million with four sources. And if we do it again, we get approximately 349 million people with six sources. So we’re getting fundamentally the same fact, but a slightly different answer. That’s where tool calling comes in to reduce hallucinations.
Context windows (7:05)
Aakash: Finally, let’s understand the power of context. Here we have an entire book, War and Peace by Leo Tolstoy. What happens when we paste this into Google AI Studio? Well, it’s less than a million tokens, and so we get a really good response. That is the power of your context window.
Now what happens when you overload your context window? I’m going to go grab Crime and Punishment, another huge book. So we’ve now given it more than a million tokens. As you can see, an internal error has occurred. That’s because we’ve overloaded the context window of the model.
Tools and function calling (7:36)
Aakash: So now you understand the three basic building blocks of LLMs. Now let’s move into tools and function calling.
The model doesn’t actually touch your systems. What it does is emit a JSON call. Your code runs the function. The result goes back to the model, and your model writes the answer. So let’s see this in practice. You’re up, Prasad.
Demo, building a get_weather tool live (7:59)
Prasad: Yes, thank you Aakash. Now we’re going to do tools and function calling. I am in the OpenAI platform, platform.openai.com. As you can see here, I have the ability to pick the model. For demo purposes I’m just going to pick a mini, it’s going to be faster.
So now we’re actually going to build it a function for get weather. Again, don’t be alarmed with all the code. Let’s actually use AI to generate the JSON function for this. I’m going to say, create a JSON function for get weather, use location. Boom. It actually generated a JSON function for us. Again, as I said, we don’t actually have to write any code. And I’m going to say add.
Now that we have created a function, I’m going to say, what is the weather in San Francisco? It’s actually going to return us the get weather by location, because it’s actually throwing an error, not calling the function. So what I’m going to do now is manually add a tool result, for example temperature 63 degrees Fahrenheit.
Now let’s see what it does. It should write an answer using the data that we provided. The location we gave earlier was San Francisco. Now we’re saying the temperature is 63 degrees, and it’s returning that San Francisco is 63 degrees and foggy.
You hand the model a list of tools, each with a name, a description, and a JSON schema for its arguments. The description is what the model uses to decide when to call. So vague descriptions cause wrong calls, as we showed earlier, where we gave a vague description saying what is the weather in San Francisco.
When the model decides a tool is needed, its output is a structured tool call, the function name plus arguments as JSON. It does not run anything. Your application code executes the function. Then you pass the results back into the conversation and the model uses it to finish the answer. As you can see here, San Francisco is 63 degrees Fahrenheit and foggy.
The model can get the arguments wrong or call the wrong tool. Your code validates and handles errors. The intelligence is in deciding to call and with what, not in the execution.
MCP, and why it exists (10:28)
Prasad: MCP is the Model Context Protocol, an open-source standard from Anthropic for connecting models to tools and data sources through one interface. Think of it like the APIs in the old days of SaaS. That’s what MCP is. You’re not hand-building a separate integration for every tool and every model. You write it once and use it for hundreds of tools available.
What actually makes an agent (12:03)
Aakash: Now let’s move into the third topic you need to understand, agent skills and capabilities.
The most important thing you need to do is define how your agent works. Most people think this is just a prompt, because they’ve been trained by custom GPTs, but really it’s prompt plus tool plus skill. This is what you need to understand to really define an agent.
So, what is the instruction or the system prompt? Not just the one-off instruction. What are the tools that it can act on, the APIs or the MCPs it can call? And then what are the skills? What is the packaged know-how it has for a whole job? Defining the agent’s skills is one of the highest alpha areas. So let me show you exactly how important agent skills are.
Demo, the same PRD with and without a skill (12:43)
Aakash: Here we are inside of my PM operating system within Claude Code. I’m a hypothetical PM working on Slack. And so I’ve defined all of that information for my PM OS inside my context library. Inside my context library, I further have all of these skills that I’ve written. One of the skills I’ve written is PRD draft.
The skill is a packaged set of instructions, and it’s optional, with scripts and resources. It has all of the routing, the context priority. It might go check the context health, what we know about a feature, when to use it. It dictates whether it should ask you clarifying questions, how to structure drafts, and so it goes on and on. This is really the alpha.
So let’s go ahead and see what happens if we don’t use this skill. We’re going to ask it to write a PRD for an AI feature that summarizes unread Slack channels. Now, normally it would go ahead and just use the skill, but here we’ve asked it to not use the skill. So we’ve actually put that into there. There’s going to be some questions that it asks you, which is a good thing. That is some of the intelligence that’s already built into a high quality model like Opus 4.8.
Now we’re going to try the same thing, but we’re going to ask it with the skill. You can just ask it to write a PRD draft, but the most reliable way to call a skill is actually to use the slash command.
We can see there’s already a difference between how the two chats have gone. The no-skill chat asks some questions. The yes-skill chat first read the context library, and it’s already found that I have an existing PRD. So we’re going to go ahead and evolve that PRD. It actually asks fewer questions with the skill, because it has better routing of my content. So having a skill in this case actually reduced the amount of human input I needed.
And now we’ve gotten an output. This output has the hypothesis, the problem, the strategic fit, the solution. So it’s created its own sort of outline. Now look at how differently the version with the skill is approaching it. It understands I’m two weeks from the June 18 GA. It understands we’re moving from solution review to launch readiness, and it’s creating a launch readiness version.
It’s worth understanding exactly how all this worked under the hood. So our skill version PRD is ready. Let’s take a look. We’ve got the right stage. We’ve got the TLDR for the go/no-go, what we’re shipping and is locked, the success metrics, the launch gate, the go/no-go checklist, the rollout plan, the rollback and kill criteria. If we were to compare the two PRDs that we got, it’s obvious that this one is closer to something you could actually send to your colleagues and it wouldn’t be considered AI slop. This one was missing a lot of that. And that is the power of having a proper skill.
A skill does not retrain the model or change its weights, but it is instruction and capability that’s loaded at runtime. So the prompt is the one-off instruction in the moment. The tool lets the agent act on the world, or fetch data, or do something in the world. And the skill is the reusable know-how for doing a whole class of task.
When you find yourself in an interview, make sure you touch on all of these components. And you might even touch upon how the tools are creating memory for your agent, which is another thing that generally is going to enhance your agent. And so when you’re thinking about answering interview questions, don’t define features. Instead, think about defining agent capabilities. And now you understand how agents work under the hood, and how important skills are.
If you want a PM operating system like the one I’ve shown here, check it out on my website at news.aakashg.com. You can either purchase it one time on Gumroad for $49, or you can become a founding plan member where you get the $150 version of my newsletter for a whole entire year, which includes my bundle with free access to tools like Bolt.new, Arize, Dovetail, and many more. Plus you get access to this PM OS, my Job Search OS, my PM prompt library, and a couple of upcoming products.
Orchestration and LangChain (16:38)
Prasad: Let’s move into orchestration and LangChain. It might be overwhelming for you to think about what orchestration and LangChain is. Let me actually break it down.
Think of an orchestra. You’ve got violins, cellos, a percussion section. Each one is capable on its own. But without a conductor, they don’t know when to come in, how loud to play, or in which order.
Think of LangChain as the conductor. The model is the orchestra. LangChain does not make the model smarter. It coordinates when each piece plays and in which order. The model is still doing all the thinking.
So with that, what I will do is step into the demo to simplify how orchestration and LangChain work.
Demo, building a chain in Colab (17:31)
Prasad: Okay, here we are in colab.research.google.com. Easy one, you can just say Colab Google and the first URL, welcome to Colab. This is where we are. What we’re going to be doing is actually start creating our own orchestration.
So when you go here, you can actually insert new cells here. Let me insert a code cell. I’ve got a bunch of them. Before I run this, I will tell you everything in terms of what you need to install. The only thing we need is to get to this Colab, and then I’m going to tell you what we need to do to install LangChain.
Here I am going to copy and paste some code. So we’re actually going to be installing a bunch of things. Again, the good news is you’re actually not physically installing anything on your machine. It’s this instance of Google Colab.
We’re going to install LangChain. We’re going to install the OpenAI wrapper and FAISS vector. We’re going to talk about what that is in a few seconds. All I’m going to do is hit run and let it do its magic here. Now that it’s almost done, again, ignore the dependency for right now. But if you’re actually doing it for the first time, you’ll actually see everything green here. For right now, I will ignore this.
And what we will do right now is actually go ahead and create one of the smallest chains possible. We introduced what orchestration is here. Let me actually copy and paste this and walk you through what we’re trying to do.
As I introduced orchestration, what we’re trying to do is actually three things here in the chain. The prompt, the model, the parser. The prompt here, as you can see, was saying tell me one interesting fact about, and the topic here is black holes. So the prompt is, tell me one interesting fact about black holes.
The second one is the model. Here we’re actually saying we’ll use ChatOpenAI, we’ll actually use GPT-4o mini. And then the parser is how do you actually want the output. Right now I’m actually saying the output is going to be a string, but you can say JSON or anything else. To simplify things I’m just going to say give me a string.
So the other thing to notice is, because we are actually saying we want to use an OpenAI model, we need an API key. Let me actually get my API key here, and then I’m going to replace my API key here. So now this is good.
What we’re asking is, again, tell me one interesting fact about black holes. And let me hit run. It should print us the output in a second.
First, as I said, a prompt template. It takes your input and formats it into a sentence. Second, the model. It receives the sentence and generates a response. And the third is the output parser. So we asked, give me one interesting fact about black holes. As you can see here, it actually went and brought us back something saying, one interesting fact about black holes is that they actually grow by absorbing. Again, I’m not going to read it, but you get the essence of it.
So this is exactly what orchestration is. The prompt, the model, the parser. This is a simplified version of it, but you get the gist in terms of how to go about it.
RAG, grounding answers in truth (20:50)
Prasad: Now what I will do is actually touch upon another one. If you have heard of something called RAG, what we will do is, what is RAG? We will actually add retrieval right now.
This is the pattern that shows up in almost every AI product, and the one PMs get asked about most in system design. It’s called RAG. It’s retrieval augmented generation. Instead of letting the model answer purely from its training data, what we’re doing is we are augmenting, or fetching, relevant text from a store and injecting it into the prompt.
A number of reasons why we want to use RAG. We can reduce hallucinations. Or think of something like, if the information changes very frequently, you don’t want to go and retrain the model every year or every X number of months. That’s where the RAG systems are going to be helpful.
In the introduction package, we talked about LangChain and all the installation done. I also touched upon FAISS here. We’re not going to have to go back and reinstall it. Think of it like a vector database, that is, a vector store. That’s where RAG really comes into picture.
So what I’m going to do is copy some code and demonstrate how this RAG system works. The good news is we’ve already given our API keys, right? So it should already do it. We’re still using OpenAI.
And what we are actually providing as part of the prompt, right now I’m actually providing some snippets here, but think of it as your enterprise knowledge. It could be PDF documents, your database, your customer support knowledge, whatever you can think of. That’s what these snippets are for. To simplify this, I’m just giving some text, but essentially you can sort of replace it with anything else.
And now I’m going to hit run. You see it actually responded saying, black holes form when massive stars collapse under their own gravity. Again, the good news about it is the response is grounded in truth, right? The truth is what we actually gave here, black holes form when massive stars collapse under their own gravity.
So the question is, how do black holes form? What this has done is actually read the snippets from the RAG system. Again, I said this is just a string, but you can use PDF documents and other data sources. And their response is actually grounded in truth here, and black holes form when massive stars collapse under their own gravity.
When a raw SDK call is enough (23:36)
Prasad: So with that I will move on to the next topic. Again, right now we’re using the RAG system. We can actually do the same output with less code, in a raw SDK, no LangChain, no framework. This is the tradeoff made concrete for a single call. This is completely fine.
Here is the code. I’m copying it. The question we’re asking is, tell me one interesting fact about black holes, and the response is, one interesting fact about black holes is that they actually evaporate over time. So in this case we are not giving it RAG or any grounded reality, but directly asking questions of OpenAI, tell me one interesting fact, and that’s what it is.
Again, this is one call using the SDK. A system with many moving parts, consider a framework. This is where you need to understand in terms of tradeoffs about making simple calls versus using RAG systems versus using orchestration with tools like LangChain or otherwise.
Aakash: One more thing I’d add here is that LangChain isn’t the only option, and Prasad talked about that. But what’s worth noting is that LangChain itself split in two. There’s LangChain, the core library, and then LangGraph, the agent layer, from the same team. So if someone says they’re building agents in LangChain today, they’re probably using LangGraph.
If your problem is mostly retrieval, chatting with documents, you might consider using LlamaIndex. That is retrieval first, and so the RAG part of it comes out in about half the lines. For multiple agents working together, let’s say a writer, a reviewer, all these agents that you put together, most people are using CrewAI or the OpenAI Agents SDK. And if you’re not doing coding at all, visual tools like n8n or Lindy are also options for this. It’s the same takeaway as the SDK point. Pick the lightest tool that solves your problem.
Routing, picking the right model (25:26)
Prasad: So the next topic we’re going to touch upon is routing. What is routing? This is one of the most practical things you can understand about how AI products are actually built, and I’m pretty confident almost no PM candidate can talk about it correctly.
Not every query deserves the same model. So let me explain what this means. Sending “what is 2 plus 2” to a frontier model is like hiring a surgeon to put on a band-aid. Sending a nuanced reasoning question to the cheapest model is a quality risk. And this is exactly where a router comes into picture. A router makes the decision automatically. If we are asking a question, what is 2 plus 2, it understands and says, oh, you know what, this does not require sending to the best model. It actually sort of routes to a cheaper model where it can be answered. That’s what routing is.
Two patterns that come up constantly. Model routing, cheap small models for easy work, large models for hard work. That’s what the intent of routing is. Classify what the user wants and send it down an entirely different path. One agent handles billing questions, for example. Another agent handles technical support. They never overlap. The classifier doesn’t need to be fancy. It can be a cheap LLM call, an embedding similarity match, or hardcoded lines.
Prasad: So I will actually explain this with code, and I have some code here, so let’s see how this works. What we’re asking is to answer this question here, and then we have the large model, we have a cheap model. As we talked about earlier, the prompt, we’re actually saying go down the path of the cheap model here, and if the classifier says you can use a cheap model, it actually routes to GPT-4o mini.
Now, as you can see, the output is, we’re asking it to say, hey, which model did it route to and what is the answer. Because the router figured out and said this is a simple question or a query to answer, it actually routed to GPT-4o mini. And then as you can see, we’re printing out and saying, now tell me which model it was routed to and what the answer is. The model is the 4o mini, because it’s a simple question that we’re asking, and then the answer is 2 plus 2 equals 4.
And now let’s actually go to the router function here. The router is a function. A label comes in, a decision goes out, the right chain runs. Simple gets the mini model, complex gets the large model. Two lines of conditional logic, and I’m going to show that right now with the demo here. Again I go back to my code cell. I have some code here again. The large model is GPT-4o, and then the answer prompt is answer this question.
So actually the next topic we’ll go on to is two queries, and then we’ll do a cost comparison. And now we make the cost gap visible with two real queries. So with that, we are actually asking the question here, what is 2 plus 2, and with the model GPT-4o mini it’s actually 14 tokens, and the approximate cost, as you can see, is pretty negligible.
The same question in terms of what is 2 plus 2, we’re actually going through GPT-4o, and then the token consumption is 574. Again the cost is still negligible, but think about it, compare it to the cost here, and then you add that up over let’s say hundreds and thousands of times, or millions and millions of times. The cost adds up.
As you can see in this clear example, routing to a cheaper model versus a more expensive model, it might seem the cost is negligible. Yes. But then when you do this millions of times, that’s where it really comes into the cost. As you can see, that’s exactly where the router is going to come into picture, in terms of figuring out and saying which model do I route to, to help answer this user query correctly, accurately, and more importantly, cost optimization.
The four layer stack (30:05)
Prasad: So we’ve covered four layers. Interaction, which is the app, chat, voice, API. Then orchestration, which routes between agents and manages state. This is where the router sits, of course, that we just demoed. Then the agent layer. These are the named agents with non-overlapping jobs. Then data and infrastructure, retrieval, vector store, model APIs, and memory.
How to answer these in the room (30:25)
Aakash: You guys just got a masterclass in the basics of the technical concepts you need to understand. Let’s pull this all together now and talk about how to answer these technical interview questions actually in the room.
The meta move is to answer a technical question by showing you actually understand the system. Naming the mechanism and choice, not just reacting with a definition.
A line that often does well is, we don’t need an LLM when an ML model will do. Any sort of tabular prediction like churn or fraud forecasting, these are usually better served by gradient boosting trees. Things like XGBoost, which are cheaper and faster. They’re also easier to interpret. You really want to use an LLM for language, reasoning, and generation. Hybrid systems are normal. And saying that actually is going to signal your real experience.
When you genuinely do not know, name what you would check and how you would find out. For example, which eval metric you would look at, like recall or F1. That reads far better than bluffing. And bluffing on a technical detail is what ends interviews.
Now, we’re going to do a couple of practice interview questions so you can see what a good answer looks like.
Mock interview (31:35)
Prasad: Aakash, that was a great session. So let’s actually do a mock right now. Let’s say you’re building a churn prediction. What would you use? Would you use an LLM or an ML model?
Aakash: That’s a great question. The answer is an ML model, but let me explain. Churn is tabular, so you have structured features with a clean yes or no label. I would put gradient boosted trees on it, XGBoost, because they’re cheaper, faster, and more interpretable. And the first question after I ship is, why didn’t an account get flagged? Trees answer that well. An LLM does not.
Where an LLM really is going to earn a seat is in the language layer. Turning messy support tickets into a feature, or drafting the save email once an account trips the threshold. So the real system that I would build is a hybrid system. So overall it lands hybrid. Trees predict, the LLM reads and writes.
Prasad: Cool, that’s fantastic. I love the fact that you actually broke down and also made a comparison between what an LLM is great at and why ML is actually a better way in terms of solving this problem, and then also said, hey, there are certain things where we’ll have to do a hybrid approach. Yes, sometimes folks, the question is not an either/or, but I love how you framed it, saying for the majority of things ML models actually work, but for the output you just use an LLM. Awesome. Fantastic response there.
So let’s actually get to the next one. Your AI feature calls a tool. Walk me through what actually happens under the hood.
Aakash: Okay. So as far as I understand, the model never runs the tool. That’s what most people get backwards. I hand it a list of tools, each with a name, a description, usually a JSON schema for its arguments. The description is what it uses to decide when to reach for that tool. So a vague description is a real bug, not a cosmetic one.
When it decides, its output isn’t an answer, it’s a structured request. The function name plus arguments. My code runs that, gets the result, and passes it back into the conversation. Then the model writes the final answer using my data.
That’s the key distinction. So the intelligence is in deciding to call and with what. The access and execution live in my code, which is also where I validate, because it can absolutely pass the wrong arguments.
Prasad: So great response. One follow-up question. Would you think the orchestration we talked about earlier comes into picture here?
Aakash: Not for this, no. A single get weather call, like we talked about earlier, that’s just request and response. The model asks, my code runs it, the result goes back. That lives in my app code.
Orchestration shows up when the loop stops being a single step. When we have multiple tools to sequence, or a failure to retry, maybe state to carry across turns, or of course more than one agent to route between. I think that coordination is really what the orchestration layer owns. So one tool, one call, you don’t need it. Many steps with state to manage, that’s where orchestration earns its place.
Prasad: Awesome. Great response. Yeah, you are absolutely right. In this context we’re only doing one tool call, but in the bigger picture, if there are other things this particular system is doing, that’s where orchestration comes into picture.
Let’s do a couple more short ones, Aakash. So how does LangChain work, and would you use it here?
Aakash: Okay, here’s how I describe it. LangChain is the glue around the model. Prompt templates, output parsers, memory, retrievers. It’s the way to pipe them all together. For a single call, as we said, we’d probably skip that. A simple SDK call is all you really need. It’s also simpler and easier to debug.
It really is going to earn its place once I’ve got many steps, swappable pieces, memory, and tool use to coordinate. I’d say that trade-off out loud, because the abstractions can actually hide what’s happening, and that makes debugging harder. For this feature, if it’s one call plus a tool, I’d start on a raw SDK and add a framework only when the orchestration gets real. I’d also flag that LangChain split into LangChain and LangGraph. So for building agents I’d use LangGraph when it gets real.
Prasad: Awesome. Fantastic, Aakash. So why do models hallucinate? And if you are a PM, what’s your move?
Aakash: It’s really the model working as designed. It’s not a glitch. The model predicts the next most plausible token. It isn’t looking anything up. So with nothing to ground it, it’ll produce something false with complete confidence.
I don’t treat that as a prompt problem. When the answer has to be right, I ground it. Either retrieval to pull in real content, or a tool to fetch real data. Then I measure whether the answer actually traces back to what I retrieved. A longer or sterner prompt doesn’t fix it. Grounding really does.
Prasad: Great answer, Aakash. One follow-up question. How would you measure grounding?
Aakash: Okay, that’s a good point. Honestly, I’d probably confirm the exact metric with the team before I lock on one. But my instinct is two separate checks. So first, faithfulness, which I kind of alluded to. Does the answer trace back to the retrieved text? Second, retrieval quality. Did we even pull the right snippet in the first place? Those are different failures, and I wouldn’t really want to chase one when it’s the other.
Prasad: Awesome. That’s great, Aakash.
Aakash: All right, so those are the mock interview responses. You can see my responses are probably like an eight and a half or a nine out of 10. You could go deeper. You could say more. And that all comes from actually using the systems. Go execute the demos we talked about in this video.
Your questions answered (37:07)
Aakash: Now you have some hot questions you’re probably asking after watching this video. So let’s answer them.
Prasad: These are some of the questions we actually get from our Land a PM Job cohort members. So let me fire a few of them for you, Aakash. Are PMs expected to know this, or is it just a filter?
Aakash: If we’re honest, it’s mostly a filter. Nobody needs you to derive how attention works. They’re checking one thing. Can you sit between two developers and not freeze when they go a level deeper? The freeze is really what’s going to fail you. A wrong detail you can recover from in the next sentence. I’ve coached PMs who found out that they made factual errors and passed, and other PMs who were really weak in their responses and they got dinged. So it’s about showing that confidence in your depth.
Prasad: Okay, fantastic. Let’s move to the next one. We talked about LangChain and all, but the question we actually get asked is, is this a real tool, or is it just like a buzzword on the resume?
Aakash: Honestly, it’s both. Real teams do ship on it, that’s important to know. And it’s also the thing people list to sound technical. The tell is pretty much one sentence. The name dropper is going to say, we used LangChain, and stop. The builder is going to say, we used it here, ripped it out there, because the abstractions were hiding what broke. So it’s the same word with the opposite signal depending on how you actually reference it.
Prasad: Awesome. I’m glad you actually said it’s both a tool and a resume bit. Yes, this is such a common thing we keep hearing. Hey, you just slap a couple of key AI tools on the resume.
Okay, so we have seen a lot of improvements in AI models, at least in the last 6 months, right? Every week, every month, exponentially improving in terms of accuracy and everything else. Why does this all matter if models keep improving?
Aakash: The capability moves, but the shape doesn’t. You’re still sending tokens in, hitting a context limit, still handing tools, still grounding the results. If anything, routing matters more, not less. More good models means more decisions about which one handles what. And someone has to own the decision. Usually, that’s the PM.
Prasad: So one last question before we wrap it up. What do you think is the one technical concept every PM, or at least the majority of them, gets wrong with AI?
Aakash: It has to be that the model knows things. People talk about it like a database they’re querying. Let’s be clear, it doesn’t look anything up. It predicts the next token. That’s the whole reason hallucination isn’t a bug you prompt your way out of. When you need it right, you ground it. Retrieval or a tool, not a sterner prompt. That reframe separates the people who get it and those who don’t.
Prasad: I like it. I like the way you actually said it’s not a database that has all the knowledge. All it’s doing is predicting the next best token. I love how you simplified it and actually explained it. Awesome.
Closing (40:02)
Aakash: Hopefully you guys enjoyed this deep dive. This is literally like 5% of what we teach, both in my newsletter at news.aakashg.com and more importantly in the cohort that Prasad and I run.
We are currently in cohort 3. We had 30 students in cohort 1, 50 in cohort 2, 70 in cohort 3. We are going to be starting cohort 4 soon in August. If you want to get this type of coaching and instruction, but even deeper, more foundations of AI, more mock interviews about how to succeed in these, join our cohort. We’re not just going to show you AI PM. We’re going to show you how to create a PM resume, how to create a PM LinkedIn, how to create a PM GitHub, how to create a PM portfolio. We’re going to give you reviews of all of those products.
If you put together everything we’re doing, $500 resume review, $500 LinkedIn review, $500 portfolio review, $500 GitHub review, plus interview coaching, plus AI PM coaching, plus PM fundamentals in the AI era, it is worth way, way more than the cost it will be for you. So go ahead and apply at landpmjob.com so you can work with Prasad and me, including several one-on-ones with Prasad that people really love.
And with that, we’ll see you in the next video.