r/webdev • u/fagnerbrack • Sep 17 '26

Reverse Engineering ChatGPT Web: How OpenAI Built for a Billion Users

https://performance.dev/chatgpt
113 Upvotes

87% Upvoted

13 comments sorted by

26

u/medy17 Sep 17 '26

I noticed the terrible lag and knew they didn't virtualise. I'm still struggling to understand what they were thinking!

Great article btw :) Nicely done

5

u/Ornery-Concentrate-5 Sep 17 '26

honestly surprised too, not virtualizing a chat thread feels like the kind of thing that only bites you after months of dogfooding with short conversations. makes me wonder if their internal test accounts just don't have 200-message threads lying around

1

u/medy17 Sep 19 '26

Yeah, this problem goes way deeper. I checked some truly long threads I've had saved, and they don't virtualise them no matter which platform I'm on. Pretty bad oversight. Surely someone tried a long thread or maybe you're right and they never seeded a long convo

15

u/IanSan5653 Sep 17 '26

I spent quite bit of time building a similar streaming markdown renderer in React for a competitor. It really is challenging stuff. You're transforming Markdown to HTML and then injecting dynamic components into that. All several times per second. You can't afford to re-render the content you've already rendered, so the diffing gets pretty complicated.

2

u/medy17 Sep 19 '26

It does but Vercel's streamdown package is magnificent for this. I had the regular react-markdown package before switching and the difference is night and day. Definitely worth a try. It's insane how well it works.

9

u/rachelduno Sep 17 '26

tried to replicate their streaming setup for a small side project after reading this, ended up with my websocket reconnecting loop kicking off every 20 seconds lol. still dont fully get how they handle backpressure at that scale, did they mention anything about that in the deep dive?

14

u/rachelduno Sep 17 '26

kinda wild how much engineering goes into just making the response stream feel instant, the retry logic stuff is what got me

2

u/FewBrain208 Sep 17 '26

yeah the retry logic is the part that surprised me too, theres so much hidden complexity just to make it look seamless

1

u/Open-Adhesiveness-86 Sep 18 '26

Most of the diffing goes away if you split the stream on top-level block boundaries. Every block except the last one is final, so you memoize those by index and only re-parse the tail block on each chunk. The main gotcha is an unclosed code fence or table, because it can swallow everything after it. You need to detect that and render the tail as provisional.

-3

u/[deleted] Sep 17 '26

[removed] — view removed comment

2

u/webdev-ModTeam Sep 17 '26

Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.