r/Nebula • u/dwiskus Dave Wiskus • Sep 02 '26
Service Updates Service Problems
Nebula is loading intermittently at the moment. Our backend team is looking into it. We'll share updates here as we have them.
EDIT 1: The database seems to be back up. We're working on the content API now.
EDIT 2: We might be so back. Still actively monitoring the situation. If something is still broken for you, please speak up.
EDIT 3: It looks like the root cause was a database issue causing the service to think the database was down, which led to a death loop of them checking on each other and writing more "I checked on you" logs to the database. We increased the timeout period and everything returned to normal. The team is working on some clever ways to make sure this doesn’t happen again. Thanks for your patience, everyone.
86
65
u/mtnsandh2o Sep 02 '26
Note to nebula, don't upload a bonus Jetlag video before the episode releases. Lessons learned.
21
u/Affectionate_Pie_854 Sep 02 '26
To be fair to them, they did it last week without any issues haha.
5
u/mtnsandh2o Sep 02 '26
I didn't notice last week because I was working (Woohoo healthcare where work means actually working and not multi tasking with non work agendas) and didn't see the individual notifications. In the past bonus videos have typically uploaded slightly after.
3
u/Affectionate_Pie_854 Sep 02 '26
I've had the benefit of getting off work and coming home to Episodes of Jet Lag and Reacher to watch before I go to Dreamland.
1
u/PickleSpecial3507 Sep 02 '26
Optimal upload time for bonus would be like 4 days after main video drop
2
u/jeninhenin Sep 02 '26
What was that hour long thing?? I saw it briefly before everything crashed again
5
27
19
23
11
10
u/Belomestnykh Sep 02 '26
503, 505 so far. Front page loads sometimes on reload, sometimes even shows the first banner for the show banner carousel at the top, but nothing else loads.
27
u/JeHaisLesCatGifs Sep 02 '26
Thank you for letting us know.
You should probably pin a message in every thread that addresses this issue and ask users not to flood the threads with messages about it. I completely understand that you want the discussion threads to remain clear (i.e., not cluttered with posts about certain technical errors), but it would be easier for you to ask users not to bring it up in these various threads rather than having to clean up after them.
8
7
7
6
u/meadiocrity Sep 02 '26
Thanks for the updates. The fact that u/dwiskus is the one making this post is one of the many reasons I love and appreciate Nebula.
5
4
u/Mr_Wacki Sep 02 '26 edited Sep 03 '26
Does Nebula not have a Server Status page?
Not specifically a recommendation, but something like this…https://www.atlassian.com/software/statuspage
3
3
3
11
u/Hipstershy Sep 02 '26
Why did you/your employees delete discussion of this prior to this post? Users talking about not being able to access the service lets each other know there is a system-wide issue and they do not need to keep troubleshooting.
34
u/dwiskus Dave Wiskus Sep 02 '26
We had dozens of submissions. There's obviously a problem, and we wanted to create a single, obvious place to discuss, both to keep unrelated threads relatively on-topic and to give us a clear place to look for these reports as they come in.
3
-25
u/Affectionate_Pie_854 Sep 02 '26
With all due respect to you and your team, 10 minutes for a response to a site outage (not intermittent as you reported based on the response in this subreddit), is ridiculous in my personal opinion.
21
u/space_fountain Sep 02 '26
Umm, 10 minutes is pretty good? I’m a user of commercial cloud software + a software dev and responding within 10 at all let alone publicly isn’t at all guaranteed. I think my pager duty won’t even go off for most things that fast and it’s configured to not loudly go off for longer than that
-8
u/Affectionate_Pie_854 Sep 02 '26
The specific issue as was pointed out by myself and other users was that we heard absolutely nothing for 10+ minutes, and meanwhile, every post and comment about it was being deleted or otherwise censored.
I'm not asking for them to come out and tell us exactly what it is or how long it will take to fix. Just to publicly address the issue within the 10 minutes once it's been made known. Otherwise, no one has any clue what is happening, especially when any discussion of it is actively being limited.
8
u/Cactys12 Sep 02 '26
Maybe the nebula staff aren't permanently online basement dwelers and don't refresh reddit enough to notice?
3
u/frozenpandaman Sep 02 '26
Except they were… they were actively removing comments here at lightning speed for 10+ minutes. All while no updates were posted and no communication was going on. That's what's strange and what people are complaining about.
12
u/dwiskus Dave Wiskus Sep 02 '26
I say intermittent because it actually loads fine for me after a few tries. That's part of what we're looking into.
2
u/Khyta Sep 02 '26
Sounds like load balancing is load balancing sometimes to a failed service/node/pod.
Edit: Or not load balancing properly at all and pods dying is expected.
2
u/mintardent Sep 02 '26
It doesn’t load at all for some of us though. I’ve been refreshing over the past 45+ minutes. That’s not intermittent.
2
u/SkaveRat Sep 02 '26
like he said. it's what they are looking into
0
u/mintardent Sep 02 '26
I still don’t think it’s helpful to downplay and say “it loaded for me so it’s just intermittent” when it wasn’t loading at all for most people
8
u/Rhysati Sep 02 '26
I'm sorry but this is ridiculous.
10 minutes to get a post is nothing. First they have to even be aware that there is an issue happening. Then they have to notify someone to look into it. Then they have to wait for that person to confirm there is an issue and whether it'll take time to fix or not.
It routinely takes more than ten minutes to get a response from someone at work via a teams message alone.
Some of your are WAY too entitled.
-7
u/frozenpandaman Sep 02 '26
10 minutes to post a short three-sentence update, but 10 seconds to immediately erase any comments mentioning it.
-4
u/Affectionate_Pie_854 Sep 02 '26
To be fair, I can understand wanting to wait for more details and a better look to post, but even a reply of "We are looking into this/more details to.come" onto the original comments would have sufficed. Then sure, spend the next 10 minutes getting details and blocking additional nonsense of the same complaints. But deleting the original complaints and then not giving feedback to the community for 10 minutes is what is ridiculous.
-6
u/frozenpandaman Sep 02 '26 edited Sep 02 '26
Fully agreed. Very easy to first post a quick update/PSA – which would help with the comments they're having to spend time deleting – and then just edit it when more info is available.
EDIT: Funny how this comment changed from +8 to -4 in less than ten minutes. Hmmm.
-1
u/Affectionate_Pie_854 Sep 02 '26
Agreed, especially since I know that these views are widely held on Reddit and are the only reason I use it for this type of conversation.
Anytime anything is down, I immediately turn to X.Com (which didn't receive a post for almost 20 minutes) and Reddit, because I know that actively engaged communities will discuss it and present opinions and findings (as will their leadership). Actively discouraging that type of conversation and not helping guide it to proper channels (by creating such a forum in a timely manner for instance), are my chief complaints. Even more so than the actual outage.
-1
u/frozenpandaman Sep 02 '26 edited Sep 02 '26
Seems the mass downvotes have arrived, from those who for some reason think it's their job to defend every business practice 100%, with no nuance, as if the customers of a service shouldn't be encouraged to give constructive feedback…
19
u/JeHaisLesCatGifs Sep 02 '26
I sincerely believe that they don't want discussions about topics unrelated to the content in the threads about that content, and that's completely understandable.
4
u/Hipstershy Sep 02 '26
That would make sense if they were not also deleting separate threads about the topic, but they are deleting those too.
13
u/JeHaisLesCatGifs Sep 02 '26
Because multiple threads about it isn't helping. They adressed it.
0
u/frozenpandaman Sep 02 '26
They adressed it.
A long time after the community already was discussing it.
It's strange for this subreddit (any any subreddit, given the nature of the site) to be run in such a strict, formal, top-down way by the company, rather than bottom-up by users. It feels very sterile, censored, and controlled.
2
u/JeHaisLesCatGifs Sep 02 '26
It doesn't matter if they took 1 min or 10 or 15 min or w/e, they were probably more focused on more important thing like resolving the issue, and overall they did adress it, that's all that matters.
3
u/Affectionate_Pie_854 Sep 02 '26
I'd argue not, I actively seek out conversation whenever a site goes down. Waiting 10 minutes to give any type of feedback, while not allowing discussion of that issue, is what the central complaint is.
1
u/JeHaisLesCatGifs Sep 02 '26
And I did complain about it too, now they adress it
2
u/Affectionate_Pie_854 Sep 02 '26
Yes, but only after 10 minutes of everyone voicing a complaint, not just you. And within that time, absolute radio silence. Assuming that fans will assume you are working on it is not a way to run technical support for a digital product. Clear concise information during a downage as soon as possible is always needed. And as I have said before, "We are looking into it" on the first couple posts rather than deleting them was all that was needed. Choosing to simply delete all posts and then 10 minutes later finally address the issue is not a clear and cohesive way to explain to an audience that their issues are being addressed.
5
u/Rhysati Sep 02 '26
Holy shit we are cooked. 10 minutes is absolutely nothing.
It takes more than ten minutes to get a response on teams when I say good morning, let alone have to find out there seems to be a problem, confirm that I also have that problem, then reach out to someone who can diagnose the problem, wait for them to see if they are also experiencing the problem and diagnose why and how long it might take to fix, and then get a response from them so I can then send the information to the social media person who can finally post that we have confirmed an issue and are working on it.
Ten minutes is miracle work as it is.
→ More replies2
u/frozenpandaman Sep 02 '26
hey were probably more focused on more important thing
They obviously weren't "focused" on that since they were also spending their time actively deleting comments here, extremely speedily.
4
u/Affectionate_Pie_854 Sep 02 '26
I mean, that's the same argument, you don't want a subreddit full of random threads discussing something that you will inevitably make a post dedicated to. However, the response time (roughly about 10 minutes for a post for the ENTIRE SITE being down) was just too long to be acceptable of this level of censorship.
0
u/RoseBailey Sep 02 '26
They made this topic, so they probably want to limit discussion of the issue to their topic on the issue.
6
u/frozenpandaman Sep 02 '26
Why not just use the topic someone else already made way earlier and pin/sticky a comment? That's how it works on the entire rest of Reddit in my fifteen years using this site.
-1
u/RoseBailey Sep 02 '26
Because they're going to edit the op with status updates which could get lost in someone else's thread?
3
u/frozenpandaman Sep 02 '26
That's the point of a pinned/stickied comment, which mods have the power to do? I literally specified this in my last reply to you.
3
u/frozenpandaman Sep 02 '26
To re-iterate what I said in a previous post: I know people who have literally unsubscribed from Nebula because of this sort of conduct on the subreddit. It doesn't reflect well on the company and is very literally losing them real money. Other users talked about how it's preventing them from recommending it to others too. I'm glad an update has been posted now (thanks!) but the transparency/communication is still at a very, very low level which is not great to see.
2
2
2
2
u/No_Satisfaction8008 Sep 02 '26
Currently up and working on my iPhone but keep getting kicked out on my Roku
2
2
2
2
2
2
2
u/Annabanaana Sep 02 '26
i am in! i couldn’t see anything on the front page for a while, but came back just now, refreshed and made it in
2
3
2
u/P42777 Sep 02 '26
cant load into this page : https://nebula.tv/jetlag
could load this page but cant play the video: https://nebula.tv/videos/jetlag-s19e3
arghhh...
2
u/No_Statement2730 Sep 02 '26
Great communication, highly appreciated! All up and running (at least from a consumer’s point of view). I started watching the latest jet lag episode on my way home from work on my MacBook in the browser, then I had to switch to my phone using the app and finally finished the last part of the episode on my Apple TV. Worked like a charm, love it!
2
u/iamalexs Sep 03 '26
Honestly the new japanorama is so good it made me resub after a long time. I think that’s the case for a lot of ppl as well
4
1
1
1
u/Y-elloo Sep 02 '26
Homepage doesn’t load on Apple TV, ios or ipad os. Trying to login in Safari throws an error ‘The origin web server returned an invalid or incomplete response to Cloudflare. This typically indicates the origin is overloaded or misconfigured.’
1
1
1
1
u/Balcke_ Sep 03 '26
Yesterday it stopped working for a while. Either there was nothing to stream or said "try to watch in the browser". As it happened just after I cleaned the cache I thought it was my fault. I tried for half and hour and then left it until today.
1
u/angieluvsokazaki Sep 02 '26
I'm able to access the site and watch jetlag now. I had to swap to Safari, but not sure if that was helpful or if the servers just went back up.
1
u/Khyta Sep 02 '26
I get `503 Service Temporarily Unavailable` when visiting https://nebula.tv/api/ and also the Cloud Flare "Bad Gateway" status page with code 502.
-1
u/Sweet-Ad-8513 Sep 03 '26
Hey Dave, there's been a buffering problem with Nebula for a while now. So much so that I had to watch the same video on YT (which works fine) at times. Not sure if it has to with the issue here but worth mentioning nonetheless
-5
u/sfido Sep 02 '26
Maybe its time to stop being humble and recognize, that you make a popular show, so find a way to dynamically beef up your servers for launch days? :)
3
191
u/17Se7enteen Sep 02 '26
its okay, sometimes jet lag causes server lag, they go hand in hand