They key thing to know is that they don’t actually remember anything. They are completely stateless. Every question, every reply, every interaction needs to have the entirety of the conversation history fed back in to it. That’s actually what the “context” is. It would be like talking to a person, and every time you spoke you had to repeat everything you’ve ever said to them (and what they said to you!) before you could say something new. It’s insanely inefficient and why they require data centers the size of a small city. “Agents” are even worse, because they talk to themselves and each other in loops, but still have to repeat that context for every single interaction.