Check out the conversation on Apple, Spotify, and YouTube.
Why OpenAI has been talking about Codex nonstop (03:28)
Aakash: If anybody has been observing OpenAI at all, they have seen that in the last month or two, Sam has been talking about Codex. Codex, Codex, Codex. It seems like you guys have invested a lot of effort into Codex. Six months ago I told PMs Codex is the best way to use ChatGPT for their use cases. So what I want to understand is, as a PM inside OpenAI, what has Codex unlocked for your PM work?
Abi: Taking a step back, when ChatGPT launched a few years ago it started as a chatbot. Last year, as we added different tools to it including connectors, it became a collaborator. And with Codex it has truly become an agent. What that means is I can give it parts of my job to do and it will do it end to end. The meaningful differences for me there are I’m saving hours of my day, especially on repetitive tasks that I’m doing again and again, but it is also enabling me to do things that I’ve never been able to do beforehand.
On the former, in a PM’s life cycle there are things you have to do every week or every month. You have to prepare for a review, write a weekly update, send a note following up on things. All of that is automated by Codex. In addition to that, I was never a great engineer myself. I studied computer science at Michigan, but haven’t really been coding since then. And I always felt intimidated, especially with the caliber of engineers that are here, as to how I could contribute. But with Codex, there are now many examples where I have been able to take a feature to 70 or 80%. Engineering does not have bandwidth, but I’m just going to build it, take it to 80%, and then let the engineers take it to the final mile. That has been super empowering both because I feel like I’ve turned from just a product manager to now also a builder, and because it allows me to go from just writing docs to giving my team functional prototypes about what we should be building.
Aakash: What would you say is the single highest leverage thing that Codex has enabled you to do that wasn’t possible before?
Abi: I’d love to actually show you this, but first, some context. My job leading international growth involves thinking about how ChatGPT is growing around the world. We are in many different countries, and as your audience would know, growth has many layers of the funnel. We think about top of funnel acquisition, then activation, then engagement, then retention, then resurrection. We also think about competition. All of this information is in different dashboards at OpenAI, so me and my team would have to spend minutes and hours loading seven or eight different Databricks dashboards trying to figure out how to synthesize this. We were just getting lost in the noise. A few weeks ago, as I was seeing other people starting to use Codex, I thought, what if I could build a dashboard, one single web app that combines all these sources and then synthesizes what the important elements are. I was actually able to do it with Codex.
A live look inside the international growth dashboard (06:32)
Abi: What you’re seeing on my screen is a version of this dashboard. As much as I’d love to let you see the internal data, I’ve modified it a little bit so you can see the structure without revealing any of the internal knowledge. What you see is basically the exact dashboard we use internally to think about international. At the top, I’ve made up seven or eight countries we can think through. All these are made up numbers, but you can see how these correspond to real life. I can flip between different countries and see what is happening, the topline metrics I care about, how weekly actives are doing, how penetration is, how it’s been growing.
Then I can go one level deeper and Codex has categorized for me what the strengths and the risks are. Here are the things going well, and relative to the rest of the world and other peer countries, which Codex has figured out the peer set for, here are the places we could be improving. That gives me and my team a snapshot of the areas we should be focusing on or what is not trending well. We can go even deeper and do deep dives on market share, new user growth, how we’re doing relative to other benchmarks. What is cool is that I can give someone an exact snapshot, but if they want to go deeper they see all the relevant stats and how it compares to competition and to our performance in other countries. The final element that is awesome is that this is updated every single day through an automation at 9 a.m. every morning. Codex runs it. I don’t have to do anything. This has been a game changer not only for me but for my entire team, because now so many more people can look at this data in one place and make better decisions about how we should be investing our time and what user problems we should be solving.
Aakash: I’ll act as a skeptic here. This already lived in Databricks. What is the unlock? How should PMs be thinking about, okay, I have some scattered dashboards. What’s the generalized principle people should be using Codex for?
Abi: The broader takeaway is that a lot of times, especially in growth, you’re making decisions by looking at multiple different sources of information. There’s a cognitive overload of trying to piece together how all these things fit together. If it was just one Databricks dashboard, I wouldn’t have to do this. But because it’s seven or eight different sources, all with different cadences, and some of them are different tools, there’s a Tableau dashboard for two of these things and a Databricks dash for six of these things, I was able to bring it together in one place and provide the TLDR summary of what is important, because that doesn’t exist in a dashboard. So it’s the combination of the synthesis and the takeaways that has become a game changer for my work and my team’s work.
How to build a dashboard like this in Codex (09:32)
Aakash: Now I understand the value prop here. We’re synthesizing data across multiple sources. We’re leveraging LLMs and the connectors. If I wanted to build something like this myself, can you walk us through the steps, the prompts, what it would look like in Codex to end up with this type of a work product?
Abi: I’d love to show you. I’m going to walk you through the journey it took me to build it. I basically asked Codex, hey, I want to create a web app which shows how I monitor growth across priority markets with ChatGPT. The audience is internal stakeholders within OpenAI. Then I identified what I wanted the output to be. Let me switch between different tabs and different countries. Show me headline stats. Highlight key strengths and weaknesses. Have a red and green thing of what’s going well and what’s not. I’m a visual learner. I want it to stand out.
Then I said, here are the inputs. I used a few different connectors. The competition dashboard with Tableau, the dashboard in Databricks, and so on. For this case, since I’m demoing to your audience, I said this is an external demo, so we’re not using internal data, but I didn’t use that when I was trying it out. The key thing was just to clarify to Codex what the inputs and outputs are, and then it runs with it.
From that point forward it started building it. You can see it was thinking a bunch. It created a synthetic demo, added country tabs, added the views for the four sources of data. It added a few notes, and then the cool part is it ran a smoke test to validate it was working itself without me even telling it to. Then I said, okay, we’ve got something cooking here, how do I make sure it’s working well? So I said, run it on terminal locally and show it to me. It spun it up on localhost, got a web preview. Then I opened it, but the background was not what I wanted. It was too dark and not on brand. So I said, make the background like openai.com. Let’s stick to our brand and test it with the fixes so I can see it’s working.
Aakash: If people don’t know, Playwright comes up a lot when you’re AI coding. It basically allows Codex to see what it actually looks like to users. It kind of takes a screenshot of it so it can see it.
Abi: Exactly. Codex opens up the web browser, takes a screenshot, sends that information back to Codex for it to self-identify the UI issues. Prior to being able to use tools like Playwright, I would have to go open the browser myself and say this, this, this. Now it’s much more seamless. What’s also cool is we’ve now created a browser within Codex, so I can just see this preview right here. I asked it to open the preview and I could see the brand colors look okay, the data looks okay. Without going anywhere else I can do an end-to-end workflow in Codex. This is a really important point because a lot of PMs’ first entry point into AI tools was around February last year when they were told use Bolt, use Lovable. At this point, Codex is now able to build previews and show you previews directly in the app. So if you want, you can actually use Codex for prototyping.
Why he moved away from writing PRDs to building prototypes (14:48)
Abi: Just this week I got asked to write a PRD about a new platform investment we wanted to make. After writing the doc for like 20 minutes, I was like, you know, this is boring. I just want to build the thing. So I built a prototype and then showed it to people. That created a better discussion because I think everyone’s a visual learner and wants to see what the end product looks like. So I’ve moved away from writing PRDs to just creating prototypes.
Aakash: We’ve got to pause there because that’s the hottest topic in product management. You’ve moved away from writing PRDs and just creating prototypes. Someone might say the PRD, you probably still need that to figure out your null hypothesis, your success metrics, your guard rails. Use it as a document for stakeholder alignment and make sure privacy, legal, and compliance, and for you guys probably the safety, integrity, red team concept, is all checked off. If you’re just on prototype, how do you take over those key tasks of a PRD?
Abi: The key point is that the end output isn’t the document, it’s the product you’re trying to build, and that is conveyed with the prototype. Along with the prototype I have what I’d call a companion doc that explains what’s happening, and it plays the role you’re talking about. It’s a quick spec, kind of like an FAQ. As you look at this prototype you may have these 10 questions, and these are the 10 questions and it covers that. So yes, I still have an accompanying document, but it is a companion. The main show is the product itself, and that’s what people are initially reacting to.
How the product development process changed from the pre-AI era (16:41)
Aakash: You’ve been a PM in the pre-AI era. How would you compare and contrast, let’s rewind back to when you were at Meta, versus now at OpenAI? Walk us through the product development process and life cycle. When does prototyping come in? What does it replace along the steps?
Abi: In the pre-AI era the usual process was you started with a hypothesis, collected some data, built some conviction, and then to convince everyone around you that this was worth doing, you wrote something like a spec or PRD, which covered all the key elements of what needed to happen. After that there was some internal alignment, then you convinced the designer to visualize this. You’d do a few revs. Then you’d go talk to engineers, can we build this. A lot of the same thinking still needs to happen, but we’re moving faster now because without needing to leverage valuable designer and engineering time, I am able to start with here’s what I think this could look like. Everyone knows I’m not a great engineer and definitely not a good designer, so everyone takes it with a grain of salt. I want you to react to something, and that shortcuts a lot of the steps I was taking beforehand, because I can show here’s what I think this need can look like without having to waste designer and engineering time. Then we’re just talking about how we can build it and if it makes sense.
Aakash: At OpenAI, like you mentioned, you’d take the code to 80%. Would your Codex instance be hooked up to the main GitHub and working off the main codebase so it could use the real components, and would you be shipping a pull request, or would you just have a local repo so you could show them the 80% done? What are the mechanics?
Abi: It depends on the purpose. For the demo website I just showed you, I was mostly working locally because it didn’t need to integrate with the ChatGPT infra. It was more for internal consumption. But let’s say there’s a new feature I want to ship for users in India and I want to figure out, in a very scrappy way, how to get people to understand what this might look like. In that case, I would use GitHub, pull from our broader repo with all the ChatGPT codebase, then point it to, and this is the key part, point it to something similar it should build upon. The question I often ask my engineers when I’m going down this rabbit hole is, what’s the most similar thing we have done to this? And they’ll say, oh yeah, just look at this GitHub. So I take that, point Codex to it, because then it knows the thing it needs to build on top of rather than spending a bunch of time navigating the codebase. The critical thing is to get a reference from an engineer of the most similar thing we’ve done. I do a few revs with Codex, get it to a point where I have a pull request out, and then I go to engineers and say, help me understand if this is the best way to do this. Maybe something’s failing in my merge. It accelerates that entire process.
The two buckets Codex falls into and its honest limitations (20:04)
Aakash: Prototyping and dashboards, those seem like two major use cases for Codex. Maybe you can enumerate the others into buckets. And what I’d also be interested in is what it’s still not good at. What are the honest limitations?
Abi: In work I put it into two buckets. There is the set of things I was already doing but that were repetitive tasks. Updates, dashboard reviews, synthesis, preparing slides for an external or internal presentation. All those things Codex is now doing end to end, and the repetitive ones it is automating. All I do is point it to the right things. Then there’s the net new stuff, which I’d call becoming a builder. Those are things I was not doing beforehand, such as creating prototypes or dashboards.
Aakash: Marinating on the work front a little bit more, what is the operating system for a PM at OpenAI? Is it Slack? And does Codex help you with that?
Abi: I want to show you my automations every single day. We live and breathe in Slack. We use Slack obsessively. I’ve not seen a company so addicted to Slack as we are. All forms of communication, including with some external partners, we brought them into Slack. Now, every PM wakes up in the morning, and especially with me working with people in different time zones, I’m just overwhelmed with the amount of Slack notifications and pings, and I inevitably miss something. You don’t want to be the guy who gets a response, hey, I sent you that message three days ago, what happened. So what I built is an automation, a daily Slack inbox triage. It looks at all the key channels. I’ve told it here are the important people, make sure if a co-worker sends me a message you tag that, and tell me the things I haven’t read that I should read and the things I haven’t responded to that I should respond to. I get this once a day in the morning and that’s how I start my day.
The other automation is the dashboard I showed you earlier, on a daily automation at 9:30 a.m. that pulls all the data. And then weekly updates. We write a weekly update to your stakeholder group talking about what’s going well and what’s not. This one also pulls from a lot of data sources, a lot of it in Slack, some in Google Drive, some in Notion, some from dashboards. I’ve got an automation that pulls all these things together, creates a weekly update, and posts it into Slack for me to review and send out.
Aakash: Is there an art to giving it the right context to navigate Slack correctly? Sometimes your decision to ship a particular international growth feature might live inside the 38th message on a Slack thread where the legal team finally says, okay, now it’s good to go. Are there tips or tricks around the Codex Slack integration to make that work well?
Abi: With anything when it comes to ChatGPT or Codex, context is king. And context does not only mean pointing it to these connectors, like use a Slack connector. It also doesn’t only mean pointing it to these Slack channels. It’s giving it information about the kinds of posts it should index that are important. For example, anything that talks about progress on evals or metrics is important. Anything that’s a net new learning, anything flagged as a blocker, is important. But to be honest, this is one place I don’t think we’re perfect yet. I still think we struggle with the separation of signal to noise. That’s why, instead of asking Codex to directly post the update to my stakeholders, I ask it to send a draft to me. Oftentimes these three things made a ton of sense, this one thing probably isn’t that important, it missed this thing I should add, and here’s slightly different framing I’m going to change because I know that in a hallway conversation my boss asked me to cover this. So that’s one limitation.
Aakash: Are there any other mistakes or things you tried to do in Codex that you’re not doing anymore that people could learn from to accelerate their learning curve?
Abi: Another failure mode is regarding data sources. As you might imagine, pretty big product. We have a B2C business, we have a B2B business, and we have some data tables that look very similar but are very, very different. You could talk about weekly active users for consumer, or weekly active users for business and enterprise. A failure mode is giving a very generic query to Codex saying tell me how weekly active user growth has changed. That could be interpreted in many different ways, which are all correct. So I’ve learned I need to be very precise about the kind of things I want and ideally point it to a specific dashboard. With an ambiguous prompt it won’t be effective at getting the right data.
The Codex harness, skills, and who should author them (26:02)
Aakash: Final question on the work Codex setup. Ryan Lopopolo wrote this amazing harness engineering piece that you guys had on your blog. Have you developed your own personal PM harness for Codex? There’s this connectors component. Have you developed skills or a really deep agents MD file? What are those things within the harness people should know about?
Abi: One of the takeaways for me over the last three months is that for the longest time, we as an industry talked all about the model. We also talked a little about the product, and both continue to be important, but the big unlock has really been the Codex harness, because that is what is powering a lot of what is being done today and enabling me to pull from all these data sources to build these prototypes. The harness has truly been a differentiator in how I’ve used it. I’ll give credit to some of my teammates because I think they’ve used it even better than I have.
On any growth team, you make decisions by running experiments. To make sure we’re doing right by our users, we have a pretty rigorous experiment review process. Before you run an experiment, you write a quick doc explaining the hypothesis and what you’re trying to go for. Then you set it up on stats, run it for a few days, and after that you write a postmortem of how it went, what you recommend the decision to be, and take it to a live meeting where we discuss the trade-offs. A few engineers on my team actually built a skill for growth engineers to do experiment reviews. All you need to do is point to the stat signal. It writes the hypothesis itself, monitors how the experiment is going, updates it, and whenever the engineer is ready to present, provides a summary, comes up with recommendations, and things we should watch out for. That’s been an amazing skill, a game changer for our team’s productivity.
Aakash: Who is the right person to author that skill? Is it a product analytics expert that owns it and creates it as a shared team resource?
Abi: The beautiful thing about Codex is that the person who cares the most is the one who makes the skill. It doesn’t matter if it’s an engineer, an analyst, or even a PM. I’ve made some skills as well. The person who feels like this would be a game changer in my workflow and could help others is the one who ships the skill.
Using Codex in his personal life, from WhatsApp triage to computer use (28:29)
Aakash: That’s the work side of Codex. I’m keen to hear how you’re using Codex in your personal life and what some mind-blowing use cases are.
Abi: As you might expect, doing international growth, a lot of the people I’m talking to outside the US use WhatsApp day and night. Communication happens on WhatsApp, especially in countries like India and Brazil. I have a large Indian family in a bunch of family WhatsApp groups. You wake up, you see 1,700 messages. You’re not sure what’s important, what’s funny, whose birthday you missed. I always get yelled at for not being good at those things. Recently with Codex we came out with computer use, the idea being to enable Codex to see what else is happening on your computer to give it the context to take actions. I’ve got the WhatsApp desktop app on my computer. I recently went to India and was bombarded by messages. So I said, Codex, I’m getting back from a day of travel, catch me up on WhatsApp by looking at the desktop app and share the most actionable things for me to look at. It’s taking the same mindset I use at work but applying it to personal life. In this example, it opened the app and realized there were two actionable things. There’s a client meeting, which is actually work-related, but then there’s a personal meeting, my friend Claire, who is in town and wants to meet but cannot do Saturday dinner, asking what times would work. That was awesome because I would have probably missed these two things.
Aakash: One of the things with AI is we all need to keep pushing the latest models and feature toolkit to see what’s possible. Many of us probably tried something like this six or seven months ago. The computer use was very lossy. It took five or six minutes. The most mind-blowing thing for me in that demo was that it just took a minute and 8 seconds. You guys have really sped up computer use.
Abi: I agree. Even for me working within OpenAI, it’s hard to keep track of everything that’s launching. Yesterday we launched the ability to use it seamlessly with Google Chrome. A key thing for PMs who are inundated with all the AI news is to set aside maybe 30 minutes in your week, and you could actually use Codex to help with this. Tell Codex, what’s gone on in the world of AI that might be relevant to me, tell me the two or three things I should try out, and then try it again. A lot of times you’ll still see that it doesn’t work, and that’s okay. But what I’ve learned is that the things that today almost work but don’t are definitely getting solved soon. That’s the signal of where there is value coming in the future.
Aakash: One of my favorite prompts now is to say, analyze all my past chats, look up all the latest features, what new features should I be using from how I use you. It seems like it’s able to incorporate all that knowledge now where it wasn’t able to before.
Abi: This is something we’ve focused on with both Codex and ChatGPT, model self-knowledge. It’s helping the tool understand what it is capable of, which is a really interesting recursive problem. By doing that it can help you onboard to the tool. You can come to it and say, I’ve never used Codex before, here’s what I do, help me understand what I could be using, what skills would be relevant.
Filing his own taxes with a Codex-built 1040 app (37:00)
Aakash: Is there anything else in people’s personal lives they should be using Codex for that they might not be?
Abi: I have a funny example. I’m not going to say I recommend it to others, but every year you have to file taxes. Last year I switched jobs and I thought, I work with this accountant who is great, but what if I could just do it myself? If you go to ChatGPT today and ask it to file your taxes, it will give you advice, but it won’t create the end-to-end output of a 1040. I get why ChatGPT doesn’t let you do that, it’s risky. But what if I could build a web app? Over the last few months I built a 1040 filing web app, which takes as input all your tax documents and spits out an output, not just an analysis, but a full 1040 you can submit to the IRS. All you need to do is sign it. I was still cautious because this is tax returns, this is Uncle Sam, I don’t want to screw it up. So I also had my accountant do my taxes and did an AB test comparison. The accountant was saying my refund was much higher, which didn’t make sense. So I sent my accountant what Codex did, and he said, oh crap, I forgot one income source. That was a mind-blowing moment, that this Codex agent which has no knowledge of accounting practices was able to spot a mistake in what my accountant had done. I’m still glad I used my accountant for liability purposes, but the fact that I was able to do this was insane. I also used it to check my accountant’s work in this latest tax cycle.
Aakash: The red alarm going off for everybody is, how do I do this safely? How exactly do you feed sensitive information into an AI model while maintaining peace of mind?
Abi: There are two elements to safety people think about. First is regarding data, and the second is regarding control. On data, there are known solutions. If you turn training off and delete that chat, it won’t be in your memory. If you use an enterprise account with us, it comes with a lot of other protections that enable you to securely use enterprise data. Those elements have existed and will continue to harden. The other element is control. In this OpenClaw era, everyone’s seen those threads on Twitter where OpenClaw went and deleted something. I have a lot of respect for what the team has built with Codex, because there are different levels of permission. You can say I want to review every action, or I want you to just run with it and find the right steps in between. Every time it’s connecting to a new data source or pulling something on your laptop, it can ask you for permission. In my prompts I also tell it where I want to give feedback. I want you to get it to WhatsApp and before you send the message, give feedback. Establishing that control relationship is really important, and that’s what helped me use it well.
What an international growth PM at OpenAI actually does (40:22)
Aakash: We just walked through how an OpenAI PM uses Codex in his work and personal life. Now I need to learn more about your PM job specifically. It’s on international growth. I’ve been on this personal agenda of getting rid of this term AI PM. I feel like it’s too umbrella. There are many different types of AI PMs. Can you help me understand your own personal taxonomy of AI PM and what type you are within that?
Abi: The broader question is I think every PM needs to be an AI PM. That’s what this new era is enabling. To operate at the velocity people do today, you need to use AI tools. Whether that means using it just to do parts of your job, or using it to build things and push the frontier, that’s up to you. As for my role, our mission statement is to make sure that AGI benefits all of humanity. Most of humanity does not live in the US. Most of humanity does not live in the developed world. It lives in India, Latin America, Southeast Asia, and all these other countries, and we care deeply about making sure all the tools we build benefit those users. My job is to figure out how we can continuously provide value to users outside the Silicon Valley tech bubble. We have grown a ton since we launched ChatGPT, and a lot of that growth has come from places like India. India is one of our fastest growing and now our second largest market. My job is to think about three layers of the stack. One, how do we improve our models. Two, how do we improve our product to surface relevant use cases. And three, top of funnel, how do we tell the story of what ChatGPT or Codex could be doing for you.
Aakash: The traditional growth and core split is that within core, PMs own specific application surface areas. I imagine at OpenAI there’s also the entire research model arm of product. So there’s research, then apps, and within apps is it like core teams that own specific features and growth sits cross-cutting, and you specifically are focused on the international part? How is that structured?
Abi: Generally we are still an extremely scrappy company and we have way more to do than we have people to do it. It comes down to what you care about, and people are very willing for you to drive that end to end. The org structure and boundaries are more loose, more of a suggestion than a hard boundary. That said, we do try to follow a similar structure where growth is cutting across the work happening across the core app and different features, trying to figure out how we drive value. My specific angle is what are the unique pieces of value that people around the world might have that maybe we’re not thinking about day-to-day, and maybe our audience is not thinking about day-to-day, that we can think about. That is at the model layer, the product layer, and the top of funnel marketing and partnerships layer.
The product moves that drove ChatGPT to 900M weekly actives (43:50)
Aakash: If we broadly think about, okay, you guys hit 900 million weekly active users, probably faster than anybody ever. There are going to be a lot of components that drove that growth. Can you walk us through, from a product point of view, the most important things for that growth? And which were the ones the growth team drove?
Abi: Most of the growth and the credit I’d give for what has happened is not because of me or my team. It’s because of the amazing work the rest of OpenAI is doing. My job is to channel that and to help ground the work they’re doing in real-life user problems that may be outside the US. The narrative arc of ChatGPT is that a few years ago when it launched it was really a chatbot tool, used widely, but specifically we had early product market fit in two segments. One was knowledge workers, people like you and I, and the other was students. What is interesting, and this is research done by ChatGPT so I’m quoting ChatGPT stats, when you think about a country like Germany, about 60% of adults with internet access are knowledge workers. In the US it’s somewhere like 40 to 50%. So in those places, catering to students and knowledge workers covers a lot of what we want to do. But when I joined, one of the things I looked at is how that differs for India or Brazil. In Brazil only 10 to 20% of the working population are knowledge workers. In India it’s lower than 10%.
These are all people who have jobs. They’re adults, but they’re not sitting day-to-day looking at a computer. Maybe they’re trading goods, running a small business, maybe work is happening on WhatsApp. So what are the use cases that can drive value to them? In 2024, one step change for us was launching search. Early ChatGPT had an arbitrary knowledge cutoff where it would say sorry, I don’t remember things past this date, which made it hard to do simple queries about how do I get to work or what should I be looking at. These are the kinds of things that matter to a larger world that are not just knowledge workers. The other step change was hands down image generation. When you think about how people interact with technology, a lot of people are not spending time typing and reading text. It’s multimodal. They’re talking, calling, viewing videos and content. Image generation was our first breakout moment in that vein, where we provided the rest of the world that wasn’t a knowledge worker an obvious way to experience the benefits of AI without having to be steeped in reading three paragraphs of text. It was so visually different than anything done beforehand, and that led to a breakout moment for ChatGPT.
What just became possible with the latest image model (47:20)
Aakash: We’ve had this recurring theme about figuring out the latest and greatest capabilities of the latest models. Image 2 is the biggest ELO jump of any model. If you look at one versus two, or Gemini Nano Banana 2, Gemini Pro 3, versus the image 2 model you guys just released, it’s an insane jump, way bigger than the jumps between two, three, and four. Can you show us what just recently became possible with the image model?
Abi: I want to start by saying huge kudos to our research team that work day and night. When I saw that LM Arena chart, I was stunned. I’ve never seen such a step change improvement, and I can viscerally feel it when I use the product because I use image gen every day. Here’s an example of something that when I tried it in the past did not work. I do a lot of user research and go talk to people in the field. This is a prompt I learned from someone in Bangalore who wanted to open a bookstore and imagine what it would look like, specifically wanting an image of book titles in multiple different Indian languages. If you know anything about India, the language changes every hundred kilometers, so you have to think about Hindi, Bengali, Marathi, and put these titles on books. The first image model struggled with this. Character rendering, especially in different languages, is very difficult, and we want to make it really realistic, not feel like AI.
While that runs, let me talk you through what we’ve worked on, from our blog post. First, we’ve worked on providing greater precision and control. Every time you create an image you probably don’t single-shot the final output. You want to edit and provide fine-grained ideas of what you want to change, kind of like what you might do on a Figma file, and with image gen we’ve now allowed you to do that. It’s able to do these fine-grained edits, to take something that was fun and make it work output. We’ve also allowed you to make multiple images at once, so that tells a story. And the thing I spend a lot of time with the team on, that I care deeply about and is mission-aligned, is working better at different languages.
This is one of my favorite examples, a Japanese manga comic. In prior versions of the model, we would never get the character rendering right. It would either not be in the right characters or mix up languages. While I can’t speak Japanese, it is able to get all the characters correct and point out what is happening in the story in a randomly cool fashion. You can see the level of detail in the image outputs is so real, and it’s stitching together multiple images at once.
Aakash: I think you guys also got the character consistency a lot better now. We saw in that manga it was the same guy across the frames.
Abi: Exactly. That’s definitely something that annoyed me in the past. So what you see here, which is really cool, is it imagined this bookstore in the book format itself and talked about the stories across multiple different languages. You have all these different states of India represented, Kerala, Himachal Pradesh, Assam, all the different languages, and the rendering for each of these to my knowledge looks pretty spectacular. If you zoom out, this looks like something that is actually a high-definition image taken of an actual book. Yet in the 30 seconds I was showing you something else, ChatGPT just made that. For those who are pro image creators, you can switch from instant to thinking, and that enables the model to up its game, have more realistic outputs, and provide high-definition images.
Image gen limitations, pro tips, and the charts breakthrough (52:08)
Aakash: At least one key tip is use thinking. In my personal use, I feel like it’s taking my prompt, understanding my actual goal, and basically writing a better prompt. What are the other things one needs to know if they’re about to go heavy into image gen? What are the limitations? Can I use this for charts in my upcoming product review, or is it still getting axes wrong?
Abi: One pro tip, if you want to describe edits, you can click the edit button and that opens up different options. You can say I want this ratio of an image, which is useful when you’ve got to edit for a specific format like Instagram. You can also select different areas of the image. Let’s say I don’t like this part, you can highlight it now and say everything else is good but edit this part. That’s why it’s going from using these images for fun to actually using it for work. As for breakout use cases, especially in countries like Japan, it’s been infographics. Even prior to building the model, a lot of people in Japan and East Asia were creating infographics to share on social media, on how Japanese growth has been. Image gen is able to create studio-level outputs. I still think the place we’ll continue improving is steerability, allowing users to say I want you to change this specific thing but keep everything else the same. We’ve made a lot of improvements there in both the product and the model, but it’s not still not perfect.
Aakash: I cannot emphasize enough, if you haven’t yet used the latest image model. I’m using it multiple times a day and my biggest use case is charts. It’s finally good at those. All the other AI tools I’d been using would do random things like use an out-of-scale chart. It feels like it’s crossed over something. It almost feels like, and I’m curious if you felt this way, that somewhere around December of last year the coding models and Codex got really good, and right now with image 2 in April the images got really good.
Abi: 100%. It was truly a step change and the evals show this, but sometimes evals may be disconnected from reality. This time I can see that every single use case I was trying before has just gotten better. It’s become so much more realistic, the edits are so much more precise, and the model is able to search the web in real time and bring that information in rather than using a knowledge cutoff. That’s why I encourage not only people who want to use images for fun, but also small businesses and creators, people who can’t afford working with an entire agency, that image gen can get you, where beforehand maybe 30% of the way, today I think 90% of the way.
Aakash: And I can’t afford creators and designers, sadly I’ve had to let some of them go, because the difference is image gen can make a full change for you in two minutes. Your iteration speed gets really high, versus when you worked with your designer you’d send that message, they’d have to prioritize that work, then do it, so there’s always a couple hours of lag. Image gen is instant.
Abi: What has been your favorite use case of image gen so far? This is useful feedback to our team.
Aakash: What I do is tell it, build a chart that looks like it could have been in the Economist or Bloomberg, a very high quality financial paper, of this recent data, and make sure this data is up to date. Number one, it’ll go find the updated version of the data. There’s this very famous chart of how the price of goods has gone in America, TVs have gone down and university has gone up. It was able to go find the extension of that data for 2026 for me and make it look beautiful, journalist quality. That has been the breakthrough for me.
Abi: While we were talking, I tried another use case. I recently went to Tokyo for work, and while all the business meetings were happening, I tried to sneak out and spend four or five hours absorbing the city and the culture, a lot about the Meiji dynasty and how it was powerful to the history of Tokyo and Japan. So I asked image gen, as we were talking, tell me about Tokyo’s relationship with the Meiji dynasty in a Japanese language infographic that’s relevant to that audience. What’s really cool is it went and searched for this information as we were talking on the web, looked at the history of the Meiji dynasty. I didn’t want to make something a tourist would be impressed with, I wanted something a local would be impressed with, so it shows the history of the Meiji dynasty over the period of time. When I showed this to people in Japan they were stunned. That was a big moment for me because I felt so proud of the work we’ve done.
What it’s actually like to be a PM at a frontier lab (57:47)
Aakash: We’ve walked through Codex, we’ve walked through image gen. In my mind, those are the two things you guys really need to be trying. Now I want to unpack PM at OpenAI. You’ve been a PM at Nubank, head of product at Tinder, and a PM at Meta. What is fundamentally the difference working as a PM at a frontier lab?
Abi: A couple things. First, in some ways the model is the product. The core thing we’re building is the model and then we build the product on top of it. The crazy thing about the model is that it is so general purpose, it can do so many different things. Also, no one exactly knows what the next model is going to be. Even we don’t. We have a hypothesis of what’s coming, but we’re not sure. There are always behaviors when we put it out to the world that people use that we didn’t even expect. We’ve seen this with image, text, and our voice models. So the key difference is you operate in such an ambiguous environment where so many things could be changing. Your road map has to be extremely dynamic and flexible and receptive to the improvements the research team is making. When we put something out and give it to beta testers, what is Aakash seeing, maybe I should be highlighting that in the product. The big change is that it is extremely fluid, but in a good way, because we want to adapt to where the research is going.
How to break into OpenAI and what to upskill on (59:33)
Aakash: If I am a PM at a regular company and I have it in my mind that I’d love to break into OpenAI, what are the things I need to be upskilling and learning? There’s too much noise. Some people tell me I need to learn how to vibe engineer. Others tell me I need to understand RAG and fine-tuning. What is the truth? What do I actually need to know as a PM on AI topics?
Abi: First, as a comfort to your audience, a lot of the core PM skills still matter at OpenAI. Structured thinking, analytical thinking, communication, storytelling. That matters wherever you go. But I’ll add two things. First, it’s important that you’re living and breathing AI. You’re using AI tools to do your work and even outside of work, so you understand where the frontier is today and where it’s going. The second thing is that the currency of progress, especially in a frontier lab, is eval. Anytime we think about a problem we want to get our researchers excited to improve, the question they ask is, can we build an eval. An eval is a rubric which helps us understand, for a specific problem, how do we measure progress. What are the types of scenarios we want to test, what are the expected outputs we want to have, then we look at where we are today, set a goal of where we want to be, and work with research to hill climb on that. Speaking the language of eval is another skill for all PMs.
Aakash: I can already see the pitchforks out because I’ve had enough comments on my eval articles and eval episodes. What is the actual level of depth a PM needs to go on eval? Where is the line between this is what the PM is doing and they speak the language, versus this is the research team taking it over?
Abi: The roles are quite fluid and there are different PM archetypes and people who spike in different things. Some PMs work very closely with researchers, co-embedded with them, going the entire way. They come with a hypothesis, write the eval, run the eval, so end-to-end workflows. Then there are others who are a little less involved and are basically helping research understand the problems we want to solve and working with them to figure out how to go from that to an eval that maybe research is driving. So it’s a little flexible depending on where you’re working. I wouldn’t say you need to come in and be an end-to-end expert at having run many of these, but you understand the value of it and what it’s trying to achieve.
His path to OpenAI and the builder attitude that got him there (1:02:18)
Aakash: Can you tell us your story? You’re now the fourth person I’ve had on from OpenAI. I’ve been collecting the stories of how you all have broken in, because when I ask my audience where they want to work, OpenAI always ranks first.
Abi: I consider myself lucky every day. Everyone here is way smarter than me, so it’s astounding I even got here, thankful to those who took a bet on me. But if you take a step back, the through line through my career is trying to figure out how we take technology to the next billion people. I grew up in India, spent a lot of my childhood there, and was always trying to figure out how we can build technology that helps people like that. When I started working, I worked at Meta, and one of the things I did was work on the election integrity team that was starting to stop misinformation around elections around the world, Brazil, India, EU, Africa. That’s when I first started interacting with this at work and the bug kicked in. Then I heard about this crazy company in Sao Paulo, Brazil called Nubank, which was then a growth-stage startup. They reached out and asked if I’d like to come work there. It was a wild thing in 2019 to consider, but I packed up my bags and was ready to move to Brazil. I spent some of that time working remotely from the US because of COVID, but then I went there, learned Portuguese, and worked in our Brazil, Mexico, and Colombia businesses, just hopping around.
This is a through line through my career. So when OpenAI was starting to think about this international role, I had thought about these problems for a while. In fact I was building an early prototype company around trying to solve real-time language translation. At my previous job, some people spoke Portuguese, some English, some Spanish, but no one spoke all of them. It was big chaos at work. So I built a Chrome extension on top of OpenAI APIs that would help solve this. I got in touch with the team, and a combination of my experience and the builder attitude is what led me to be here.
Aakash: The ingredients are really coming together. World class experience at Meta and Nubank training you on the fundamentals of PM and specifically international growth, so you have that extremely strong career base. But the other component, which I think some people are missing and now hear validated from your story, is you went out and actually built an AI product.
Abi: 100%. The latter is important not only from the perspective of a resume or an application, but for your own skill set and learning. When I started building, I started realizing what can work and what cannot work. I needed to create, I didn’t even know this word then, but I needed to create evals myself. The translation thing wasn’t working, so we had to create a rubric. As I got into the interview process, it was like, a lot of the things we’re talking about now, I hadn’t done to this standard, but I understood operationally. That really helped.
Where to find Abi (1:05:30)
Aakash: I could talk to you for another hour, but thank you so much for opening yourself up, sharing so much information. I have not gotten these insights after talking to so many people.
Abi: Thanks for having me. On a final note, I went to Michigan like you, and when I was there about 10 years ago there weren’t many people in the product management industry. When I started learning about it, I looked at you on LinkedIn. So it’s funny, a decade later, full circle being able to talk to you in this format. Thanks for all the help you do for everyone interested in product and growth.
Aakash: If people want to find you online, where should they find you?
Abi: LinkedIn is probably the best place. Not a big Twitter person yet, but LinkedIn, and my name is the same as my handle.
Aakash: If you guys are working on international AI products, you now know the person responsible for international growth. Until the next episode, we’ll see you later.