From LLMs to Agents - The Next Leap

Adam D'Angelo

Harrison Chase

Summary

AI and Agent Development: Key Takeaways from the Conversation

1. Rapid Evolution of AI Models: The tech landscape is evolving rapidly, driven by the frequent release of new models. This necessitates adaptability and continuous learning for professionals.

2. Role of Reasoning Models: These models are crucial for handling complex tasks, offering advanced reasoning capabilities that enhance application development.

3. Large Language Models (LLMs) in Development: LLMs are being utilized to build user interfaces for AI-powered agents, as seen in tools like App Creator on POE platforms.

4. Adaptability and Continuous Learning: The fast pace of change requires professionals to be flexible, constantly updating their skills and knowledge to keep pace with technological advancements.

5. Potential of Code Generation Applications: These applications could democratize software development, making it more accessible by automating code creation for a broader audience.

6. Understanding POE Platforms: These platforms allow users to interact with AI agents, providing insights into how agents are built and utilized in real-world scenarios.

Conclusion: The conversation underscores the dynamic nature of AI, emphasizing the importance of staying informed about emerging tools and techniques. Professionals should focus on adaptability, continuous learning, and exploring innovative applications like code generation to stay ahead in this rapidly evolving field.

Auto-Highlights

Auto-Highlights: Highlight: different models, Count: 4, Rank: 0.09 Highlight: Reasoning models, Count: 4, Rank: 0.09 Highlight: new models, Count: 1, Rank: 0.09 Highlight: language models, Count: 6, Rank: 0.09 Highlight: video Models, Count: 2, Rank: 0.08 Highlight: certain models, Count: 1, Rank: 0.08 Highlight: multiple models, Count: 2, Rank: 0.08 Highlight: AI model developers, Count: 1, Rank: 0.08 Highlight: text models, Count: 1, Rank: 0.08 Highlight: large language models, Count: 4, Rank: 0.08 Highlight: appendix models, Count: 1, Rank: 0.08 Highlight: many different markets models, Count: 1, Rank: 0.08 Highlight: open source models, Count: 2, Rank: 0.08 Highlight: deep SEQ models, Count: 1, Rank: 0.08 Highlight: people, Count: 42, Rank: 0.07

Transcript

Please help me welcome Adam Gamboulo to the stage. So I thought the best way to kick this off was to ask Po for a question that I should ask you. And so this is what it came up with. Can you share the inspiration behind POE and how it fits into your vision for the future of knowledge sharing and AI? Sure. Speaker B: So, first of all, thanks so much for having me. We started looking into AI at Quora, into large language models, when we saw GPT3. And we started experimenting with using it to generate answers on Quora. And we learned pretty quickly that sort of the right paradigm for. For humans, getting knowledge from large language models was going to be something that looked more like chat than like the sort of human library orientation of the Quora product. Speaker A: And so did you think that before ChatGPT? Speaker B: Yeah, yeah, this was. This was before ChatGPT. So we did. Through that experimentation, we decided that this was such a paradigm shift that it sort of called for a new kind of interface. And so we set out to build a private chat product. And for us, because we weren't building the large language models ourselves, we thought a lot about what is our role in this ecosystem that's going to emerge. And we decided to make a bet that there would be a wide diversity of both of language models and also of products built on top of those language models, which today we call agents. And we thought there would be a need for a sort of common interface to all of those. And so in the same way that the web browser was very important for the development of the Internet, we thought that there was a need for sort of a single interface that people could use. So before the web browser, if you were building an Internet product, you had to build client software, server software, build your own protocol. And that was just a ton of work. And so you had, you had a limited number of applications, you had things like SMTP for email, you had irc, FTP. But anyone who wanted to make something like, you know, a hobbyist trying to make a new product, they could not do that until the web browser came along and just greatly reduced the barrier to building basically an Internet product. And so after the web browser, you had things like homepages, where anyone could have a homepage. You had a lot of these FAQ pages of all kinds of explosion of Internet products that were enabled by the web browser. And so our hope with POE is to have the same effect for AI, where we make it so that anyone who is either training a model or building a product on top of a model can just plug into PO and we provide iOS, Android, Windows, Mac clients, a web interface, we provide monetization, we provide a history and sort of all this work that you need to get a an AI product to millions of consumers around the world. We just sort of have a single place to do all of that. Speaker A: Yeah, I think one of the interesting things that we'll probably dive into is I think you are one of the few kind of like consumer facing companies that we have on stage today. And so I'm curious how, how are you seeing consumers use and interact with AI? What are the common kind of like patterns and use cases? Speaker B: It's really varied and I think it's interesting. One of the great things about large language models is that they are so general and they're just so capable of doing almost anything and that makes it challenging to build on top of them, but it makes it just a great experience as a user. You can continually find these. So we have a very wide variety of use cases on poe. There's everything from writing assistance to, to question answering to things like there's role play, there's homework help, there's a lot of assistance and people like doing their jobs. There's a lot of media, a lot of marketing usage, a lot of people creating media. We support image and video and audio models and the sort of the central value proposition that we've landed on for consumers is pose one place where you can get all the different AI products under a single subscription. And that's very appealing for a certain set of people especially. It's especially true for people who are either developers building something on top of AI because they want to try a lot of different models. It's valuable for marketers. They often want to create things with many different markets models and it's just anyone who's sort of like an AI enthusiast, this is a pretty attractive value for them. Speaker A: Yeah. Going to poe, you can scroll down the list of all the different models on there and there's a lot. What are you seeing in terms of which ones are people using? Which ones are most popular? Speaker B: Why? Speaker A: You guys have a really unique vantage point to how people are using all these different models. Speaker B: Yeah. So we actually just published this. A blog post is called the AI Ecosystem Report which I would encourage people to check out if you're interested. If you just do a Google search for POE blog, you'll find it. It's the first entry right now. I'd say the interesting story of the last few months is really the growth of Reasoning models. So this includes 03.04, mini Gemini 2.5, it includes Claude, Sonnet 3.7 and there's deep SEQ models. There's going to be a growing set of models in this category, but those have sort of really grown in usage recently. It's just incredibly powerful what they can do, especially if you're doing something relating to writing code. The reasoning really adds a boost to your accuracy. Speaker A: I know Anthropic published some study on how people were using Claude and they found that coding was abnormally high percentage. Do you think that's the same of people who are using poe? Speaker B: I think it's probably a little bit different. I'd say we have a biased group of users, which are. We do surveys of our users and ask why are you using poe? And the top reason is that it's a place where you can get all the AI in one place. And so we sort of have a biased selection for the kind of people who want to use multiple models. I think in the same way, I would guess that Anthropic tends to get more developer oriented users. We do have a pretty decent percentage of usage that is code related, but I'm sure it's not as high as Anthropic. Speaker A: You mentioned a few different modalities. Voice images, which ones are you seeing? Kind of Resonate. I know OpenAI, when they launched their kind of image support, all the Studio Ghibli images went crazy viral. Are you seeing similar things or we had someone talk about Voice earlier and how powerful that was. Speaker B: Yeah, I mean there's certainly a lot of excitement around the new image models. And we have the OpenAI body model, the GPT4O Image Gen, it's called GPT Image 1 in their API and that's what we have it under POE. It's a great model, I'd say. Overall, to answer your question, text models still dominate usage and we've been thinking about why this is. I think that there's something where the image and video models are still not great. Like, you know, GPT Image one was a huge step forward, but we're still not at the point where you can reliably get graphics that are sort of like useful to use in say a presentation relative to what a designer can do. Whereas when you, you know, with appendix models, especially with reasoning, the quality is great. It's much better than what people can create themselves a lot of the time. And so the economic value there is a lot higher right now. I think this may change as the image and video Models get stronger and we'll just have to see. But our view is we're just trying to provide a general interface, so wherever the market goes, we want to provide the best service there. Speaker A: One question I have on that is, like, how much do people care about the model they're using and why do they care? Like, I think, you know, if you mentioned some of the use cases, it sounds like you have specific kind of like agents for, you know, doing different things. Do people care about the model under the hood? Like, this has always been something that I've thought about. Speaker B: Yeah. I mean, it varies and I think this is back to the selection effect where the kind of people use POE tend to care. I would say that a lot of the use cases are around someone trying to be the best they can be at some job they're trying to do. So you might have like a writer, someone doing creative writing, someone writing a book. They don't just want to like have one model they can consult, they want to try. They're trying to find like the right wording for something or generate ideas or find inconsistencies in some passage and they want to run that through a bunch of different models. And they'll be very particular about it. And we know because occasionally a model will, a provider of the model will go down and we'll have downtime and people are very unhappy. They'll just switch over to the other ones that are up. Speaker A: Interesting. Do you let them run it side by side to compare various outputs or one at a time? Speaker B: We let people query multiple models in parallel. So there's a syntax on POE where you say and then the model name and you can stack many of these entions in the front of a message and then in parallel we'll send it. Speaker A: To all the bots for the reasoning models you mentioned they're better at things. Do you think part of the appeal is also the reasoning traces that you can see? I know when Deepseek launched kind of their chat app that a lot of people really like that and they theorized that's why it was getting so much attention. Do you see that like UX trend being interesting or is it really they're just better? People just care about the quality, you. Speaker B: Know, I don't know where the market is going to shake out on sharing the reasoning traces. I, I don't know how much certainly people liked that with Deep seq, but I don't know how much of that was novelty. And especially as these models get optimized and they get faster, it's just a lot that you don't care to see. And so it may be that we end up on showing summaries of the reasoning traces or maybe that they're hidden, maybe they're hidden behind a click, something like that. For now, I think people like seeing the reasoning when you can provide it, but I don't know if that's something we're going to want long term. Speaker A: One of the things that O allows for besides just accessing the RAW models is accessing kind of like agents built in a variety of ways. So Asaf, who was talking earlier, who's the creator of GPT Researcher, they have a bot on poe. How do people create these bots on poe? Speaker B: So there's a few different options depending on your level of technical sophistication. So the first, the simplest option is we allow prompting. So you can just put in a prompt and a base model and you choose a base model. So you can choose an existing model on POE and you add a prompt to it and you create what's called a prompt bot. And that bot is just going to use the base model and follow the instructions in the prompt. And it's very simple. But this is actually pretty valuable for people because it saves them from having to repeatedly enter the prompt. It also means that you can, once you create a font like this, you can share it with other people. We have the whole category. You can explore the bots that other people have shared and see the most popular ones and search for them. Speaker A: Do you see that people are mostly coming creating bots for themselves or mostly public ones that they're sharing? Speaker B: A lot of it is creating bots for yourself. But then most of the usage of bots is using bots created by other people. But I think most of the bots are not are just used by the creator, so you're not involved feature at the same time. And so if you share the bot then you can monetize it and basically you'll get a cut of what people are paying POE to access it. Speaker A: We've talked a little bit about what the kind of like Agent Engineer or Agent Builder profile looks like, who's building these bots. Speaker B: So the prompt box, I mean it's a real art, prompting is a real art. And I think that the art has changed as the models get more powerful, especially the reasoning models, but I think it's still a real art. And the kind of people creating them, they tend to not be very technical, but they are people who can sort of like empathize with the model. And they're just very persistent in trying the model in many different cases and understanding what needs to go in the prompt. Speaker A: Then. Speaker B: So that's prompt bots. Then we have people who can create what we call server bots. And a server bot is pretty simple. You just give us the URL of your server and we will make an HTTP query over to your server every time a user sends a message to the bot. And whatever you return from that request, it will go into the poemessage as a response to the user. And so this is more useful for if you want to do something more complex than just a prompt, if you want to have an agent, if you want to query outside data sources. GP Researcher, for example, is a server bot. And this is more sophisticated developers who are creating, creating these and also AI model developers. So just to give you a good example, there's this really niche model I think is cool called Retro Diffusion. And there's two people, they train this model to generate pixel art for games and animations as well in the same kind of like pixel art style. And the model's just super tuned and for exactly that use case and they set up a server bot for their model. That's something that you have to use a server bot for because they're doing inference with GPUs and they're hosting all of that. Speaker A: Do you see that most people who create server bots are training and hosting their own model or is it more around the workflows and things like that? Speaker B: Originally it was mostly people hosting their own models, but over time it has become more of agents and kind of like more complex applications built on top. But, but we have, I mean we have companies like Together Fireworks, there's a whole fal. These companies that host open source models, they generally set up server bots and for these companies, for everyone, it's sort of a way to. If your company is not set up to reach consumers all over the world, like if you're a developer oriented company or if you're just a small team, then setting up a server bot is a way to reach a much bigger audience and just basically generate extra money and get feedback from a broad audience. Speaker A: How does the monetization work? Speaker B: So it's totally flexible. The creators can choose what they want to charge users and we turn whatever the creators want to charge in dollars, we turn that into a number of points that the users get. And so users have a limited number of points depending on how much they're spending on PO per month. We're basically just passing through the cost from the creator to the user. But we have creators that are making a lot of money. I mean there's, there's companies making in the millions of dollars per year. Speaker A: Millions of dollars per year? Speaker B: Yeah. And there's like individual people making in the hundreds of thousands and tens of thousands of dollars per year. So there's like a real economy developing here. Speaker A: What are these prompt bots, are these server bots or what are they doing that's providing that much value? Speaker B: I mean, it's a mix. It's, you know, some, sometimes it's training our own models, sometimes it's hosting open source models. Some, some people, some companies are very good at like optimizing the cost of inference of certain models. Sometimes it's it's agents, sometimes it's prompt bots where it's just like some people are really good at prompting, sometimes people are good at marketing. And that's another component of, particularly for prompt bots, just like building enthusiasm from Internet users about using your bot. But this is something that, it's an opportunity. If you're good at marketing and you're good at prompting, you can create these bots and people making a living off this. Speaker A: You mentioned agents. Are these bots taking actions for people at the moment or, or are they mostly kind of like just still conversational and read only, not write. Speaker B: They're mostly read only. One of the places we'd love to go as a platform is to enable more of these real world actions. We're not there yet, but so right now it's basically bots that effectively don't have side effects is a way to think about it. So the whole action of the bot is to create some artifact that it then returns to you in the chat. Speaker A: We've got hundreds of developers here. If you were a developer, what type of bot would you build or what would you recommend people explore? Where do you think there's an unmet need? Speaker B: I think the agents category is probably where there's the most opportunity. Obviously if you can train models, you should do that. But I think that like building on top of the models, building things that are more sophisticated than just a simple prompt, but not as sophisticated as like training a new model or fine tuning. There's like so much to be built. You know, you can and should build it with LangChain, but there's just this like incredible space that has opened up to make agents that are useful to people. And yeah, I would experiment with that and see where you get traction. Speaker A: And I said this earlier, but agents with Side effects. Will those be coming soon? How are you thinking about that in general? Because I imagine there's a bunch of risks obviously associated with that. Speaker B: Yeah, no, we'd love to support that. I think things like MCP have made it easier for there to be standards around how to do that. We will get to that at some point. Right now we have a lot of competing priorities, so we'vehad to focus and we're hard at work on some things that will unblock the way for us to provide a great experience around that. Speaker A: Zooming out a little bit, you made a reference earlier to kind of like the early days of the Internet and kind of like you also built Internet company. How do these two things compare? What are the differences? What are the similarities? Speaker B: There's. I mean, a lot is similar, but I think the big difference is sort of just like the pace that the environment is changing. Speaker A: So largely because of the foundation models. Speaker B: Yeah, because of the AI and just the change. You know, just every. Every month there are new models, there's new modalities, there's agents, there's xcp, there's tools. There's just the sort of like the environment that we exist within is changing incredibly quickly. Whereas with Quora, when we started in. We started in 2009 and launched the first version in 2010, it was, let's say it took maybe like every few years. There was a significant shift. You know, there was a big shift with mobile and there was a sort of shift with creators coming more. More of like a constituency, if among like the Internet. But it was, you know, we had to sort of grow with the Internet, with internationalization, as different countries started to grow online. But it was. The pace of change of the environment was. The environment was pretty stable. And that allowed us to sort of focus internally and sort of invest a lot in internal abstractions and just sort of like polishing things. And there's a certain way of doing things that works when you're in a stable environment. In this environment, things change so fast that we just need to sort of constantly adapt and react. Like it doesn't make sense to plan years ahead in. Speaker A: How do you plan? We. Speaker B: I'd say our plans go out about two months right now. That's how far we have some very. Hopefully you'll all see some great stuff launched in the next few months and then we will evaluate. I mean, we could plan further than that, but I just think, you know, we're gonna get two months from now, we're gonna launch the stuff we're working on and then we're gonna. It's gonna. The things you would think of to do then are different than what you would think of to do now. Speaker A: That may make answering my final question hard, but what do you see coming down the pipeline? What are you most excited about? Where do you think some of the space is going? Are there things POE that we should be keeping an eye out for? Speaker B: You know, I think the main thing is just the power of the models is going to keep increasing. The pace this year has been incredible, right? It's just as the reasoning models have gotten stronger and stronger. I'm particularly excited about code generation applications. So we have. We have a tool within POE we call App Creator that uses. Uses LLMs underneath to help you generate interfaces and interfaces for bots on poe. So if you want like a graphic interface, a web interface, and it's good now, but I can just see where that is going to get to in the next six months as the code generation abilities of all these models keep growing. I mean, it's just the amount of software that we're going to be able to generate and the ecosystem that's going to emerge around that is. It's going to be incredible. So I'm just really excited for that. Speaker A: It's a great note to end it on. All right, let's give Adam a big round of applause. Thank you guys all for being a big part of Interop the first time we did it. It's been a really fun day for myself and hopefully for all of you. And hopefully it inspires you to build lots of agents. I want to give a special thanks to our presenting sponsor, Cisco Experience.