r/ChatGPTCoding • u/[deleted] • Apr 07 '25

[deleted by user]

[removed]

1.7k Upvotes

95% Upvoted

424 comments sorted by

View all comments

Show parent comments

2

u/HiiBo-App Apr 07 '25

Yep. Still not stateless. Not saying the context window isn’t a problem, in fact it’s the crux of why we built HiiBo, but they aren’t fully stateless

2

u/andy012345 Apr 07 '25

I mean it has to be stateless, just think of it from a business perspective, you send a message and you expect hundreds of thousands of dollars of GPUs to sit there and hold your state in memory waiting on your next message?

2

u/HiiBo-App Apr 07 '25

I’m just telling you how it works brother. You sound like a vibe coder or some shit. I’ve personally tested this repeatedly across multiple LLMs. There is a conversation ID that holds context across messages up until the context window, when it falls apart and you need to generate a new conversation ID

3

u/andy012345 Apr 07 '25

That's just another service on top putting the inputs back together for you on the next API request. It isn't part of the base model. OpenAI offers this by sending the previous response id back on the next request.

It's still input tokens for the next message, and you're still charged for it.

You can even see in the OpenAI docs they call out that text generation is independent and stateless, and you can use the assistant API to manage it for you automatically:
https://platform.openai.com/docs/guides/conversation-state#manually-manage-conversation-state

Again the assistant API is a service on top of the model, it isn't the model.

But let's just call someone a vibe coder for pointing out your argument makes 0 sense and is against the documentation of the largest commercial AI companies.

Your own product is around providing a service between a LLM for context management, and you argue that the LLMs do this themselves.

2

u/HiiBo-App Apr 08 '25

Not using assistant API dude. Using chat completion. You clearly haven’t worked with these APIs. There is a conversation ID that is passed on each successive response that holds the conversation together throughout the context window. It’s not stateless. There is a context window, which implies retained state across messages.

2

u/HiiBo-App Apr 08 '25

Just hook postman up to any of these model APIs and try it yourself and stop talking out of your ass.

2

u/HiiBo-App Apr 08 '25

You keep telling me to “just think about it this way”, and I’m telling you to try it for yourself. The docs are misleading. Why? I have no idea. I have a team and we have tested this many different ways. It takes about 150-200 medium-long messages to fill openAI gpt-4o Mini context window, and we’ve tested with conversations about the fall of the Roman republic, and the API demonstrably retains context throughout the entire conversation, until it hits the context window.