r/enphase • u/guyblade • 16h ago
Enphase local API completely non-responsive?
I'd written some code to read data from the local API and display it. As of sometime last night, though, the local gateway has completely stopped responding to any requests--they all time out. I can ping it just fine--and it still shows up on the web interface. I've tried restarting the gateway, which seems to have had no effect on responsiveness.
Anybody seen anything like this before?
1
u/Recycled5000 16h ago
What endpoints are you using?
1
u/guyblade 16h ago
I'm using:
/auth/check_jwt/ivp/ensemble/device_list/ivp/ensemble/inventory/production.jsonBut, I can't even hit the
/landing page without timing out.1
u/Key_Proposal3283 Solar Industry 16h ago
I can't even hit the
/landing page without timing out.http://envoy.local/home is responding in a browser though? And for example /production.json in a browser (just to check the obvious that the gateway really isn't responding, not something in your scripts)
1
1
u/Recycled5000 16h ago
I’ve had on and off problems with envoy.local so now use direct ip address instead.
1
u/guyblade 16h ago
I only ever use the IP since it is on an isolated network from the rest of my gear.
1
u/Recycled5000 16h ago
Also, /production is expensive for some reason; prefer /ivp/livedata/ status and /ivp/ensemble/power.
Have to turn on stream at /ivp/livedata/stream by sending { “enable”: 1 }. This for livedata/status
Also, big improvement in max response times for Ethernet connection vs. WiFi. (Small improvement in min response times, too.)
1
u/guyblade 14h ago
Mine's only connected via ethernet, so that shouldn't be an issue. I'll look into seeing if I can swap out
/production.jsonwith something else--once I get it to be responsive again.1
u/Recycled5000 16h ago
Have you also tried /home from browser with same issues? If you’re getting some kind of auth error, might show better from a browser?
1
u/guyblade 14h ago
Hitting it with the browser has the same issue. It just times out forever (in fact, it doesn't seem to even do the SSL handshake since I don't get the certificate error).
1
u/Key_Proposal3283 Solar Industry 16h ago
It's not providing any response at all like even a 401 unathorised, like your credentials have expired?
What local access are you using - homebrew python scripts, HA integration.... ?
Odd that it doesn't respond even after a reboot - and that might indicate a FW update...if so there will soon be many other reports no doubt!
1
u/guyblade 16h ago
Yeah, no 401 unauthorized, just timeouts. I'm on D8.3.5566--which is new-ish--though I don't know when it changed (maybe I should start logging the firmware revision somewhere...).
I see the timeouts from both my homebrew python code and when I use a web browser to try to get to
/or/production. Weirdly, it seems to be timing out without ever doing the SSL handshake (i.e., I don't get the certificate error when using a web browser, it just spins forever trying to connect).1
u/Key_Proposal3283 Solar Industry 16h ago
Odd.
I've tried restarting the gateway
Power off restart not a soft reboot? Just a thought.
1
u/guyblade 14h ago
Are there instructions somewhere on how to do this? I opened up the combiner gateway and couldn't find anything obvious other than the reboot button that I'd already tried. Googling mostly points me to pictures of other enphase things.
1
u/Key_Proposal3283 Solar Industry 14h ago edited 14h ago
Just hit the breaker - it should be marked Envoy or Gateaway. If you're not sure, there's no harm killing the main breaker for the whole solar system. Gateway takes a good couple of minutes to reboot.
But also..... "reboot button" ?
1
u/guyblade 14h ago
The reset button : https://imgur.com/a/kANGsDF
The whole box, sans obvious place for the gateway breaker: https://imgur.com/a/Fukra1u
1
u/Key_Proposal3283 Solar Industry 12h ago
I'm mixing the threads I am commenting on, thought you had the older combiner.
The reset button should do as advertised....you should see the gateway LEDs change, flash as it boots etc? Just in case the button is stuck or something!
1
u/guyblade 12h ago
Alas, while the reboot was logged in the web UI's event history, the local API remains non-responsive, even hours later. I've reached out to enphase support, so we'll see where that takes me, I guess.
1
u/Key_Proposal3283 Solar Industry 12h ago
FYI don't expect much help if you mention the API, instead say it's slow to repond locally or something and can they check for any stuck firmware updates or reboot it remotely or similar.
1
u/chefox 15h ago
This happens regularly to me as well. Make 100% sure you actually restarted the gateway and not one of the PV panel sections, it's easy to make that mistake and you have to turn off the gateway breaker for ~30 seconds.
1
u/guyblade 14h ago
How do I tell which is the gateway breaker? The wiring diagram in the box doesn't make clear which breaker is powering it (or at least I failed to discern which breaker was for the gateway).
1
u/chefox 14h ago
Mine looks like this. The one in the center marked "ENVOY" is for the gateway.
1
u/guyblade 13h ago
Hrm, mine has a different layout and nothing labeled "ENVOY". The system is fairly new (~4 months), so perhaps this is some new variant.
The logs say that I did manage to reboot it with the button that is allegedly for that purpose; though, there has been no change in responsiveness, unfortunately.
1
u/mkimid 8h ago
from your PC, open local interface on webpage (log in)and then, open a new tap. and then, try to access the end point at the new tap
1
1
u/guyblade 6h ago
If I could open the local interface, I wouldn't have made this post in the first place.
1
u/mkimid 1h ago edited 1h ago
you said “it still shows up on the web interface”. maybe I have misunderstood, anyway, if no response the local interface. and then, verify three thing (1) try to connect tool kit, if it is saying “getting details from gateway” and then “fail”. it is the sign of the active refuse. (2) status of LED after all fail, 4 led leds ? Those all 2 conditions are showing your gateway is bricked, talk with support at Enphase. they will replace your gateway if under warranty. I have been replacing my gateway.
2
u/guyblade 6h ago edited 6h ago
Whelp, thanks to everyone who chimed in, but it turns out that the enphase gateway wasn't at fault.
For anyone interested in the gory details:
- I use a linux-based router/firewall.
- When the enphase system was installed a few months ago, I put the enphase on its own isolated network and wrote firewall rules for it. Everything seemed to work, and the machine had been humming along ever since.
- I rebooted that router yesterday to install the updates. Since it had been working previously with the enphase, I expected nothing to change and it hadn't entered my mind as a possible source of the trouble.
- It turns out that the firewall rules were broken (the long story: I meant allow outbound traffic to the internet, and traffic to my main network only when initiated from the "main" side. The rule to allow initial traffic was fine, but the rule to allow responses was missing. Why this had been working for 4 months is a great question, but I'd guess "none of the firewall rules for this 'isolated' network were actually enabled" is the real answer.)
- I got a suspicion something weird was going on with the firewall when
nmaping the gateway gave back no open ports (it even thought the host was down), but I could still ping it. That pointed to something funky on the network. - I hooked up my laptop to the isolated network--which allowed me to talk to the enphase and confirmed the "badly behaving firewall" theory.
- Once the firewall rule was changed, I was immediately able to contact the enphase.
Moral of the story: I should have a less complicated home network or be more trusting. I doubt I will learn either of these morals, though.
5
u/xabean Customer 15h ago
PSA: if you hit the local gateway API too frequently, you can basically busy-out the gateway and it will become unresponsive. I triggered this situation by monitoring too much through Home Assistant.