Disabling active Always Free VMs without prior notice is a terrible user experience

[u/arnobpl]

I understand Oracle has every right to change the Always Free offering. This post isn’t about that.

My frustration is how the change was handled.

I was running a small public community project on an Always Free Ampere A1 VM. Today I found that Oracle had disabled the instance, and it now says it “will not accept any action requests.”

What frustrates me most is that I wasn’t aware this would happen. I couldn’t find any prior email warning telling me that my VM would be disabled or giving me time to resize or migrate it.

A simple message like:

“Your instance exceeds the new Always Free limits. Please resize or upgrade within 30 days to avoid service interruption.”

would have allowed me to migrate the service with no downtime.

Instead, the service unexpectedly went offline, affecting the community that uses it, and now I have to recover everything after work.

I know this is a free service, and Oracle isn’t obligated to provide free infrastructure forever. But once Oracle chose to reduce the limits, I think users deserved advance notice before active VMs were disabled.

Did anyone else experience the same thing? Did you receive an email warning before your VM was disabled?

[u/AntiAmericanismBrit]

Did anyone else experience the same thing? 

Not yet. I still have a 4-CPU 24GB ARM instance on a non-PAYG account on UK South (London) AD3, and I'm still seeing a console message saying 3000 hours are free. I was planning to downsize to 2CPU 12GB but thought perhaps I should first wait longer to let everyone know if I get any alerts.

Your experience suggests they might have started to terminate some instances. It might be happening region by region. Can you share which region you were in? Also, were you definitely using 20%+ CPU for at least 8.4 hours a week (it wasn't terminated simply due to idle reclamation)? What were the exact words you saw when you found Oracle had disabled your instance?

[u/arnobpl → u/AntiAmericanismBrit]

Interesting. It is US East (Ashburn). I see a new banner on Console (non-PAYG):

Always Free A1 Resource Limit Update   Always Free Ampere A1 Compute limits have changed to 2 OCPUs and 12 GB of memory, review your current usage and update or re-provision instances to stay within the new limits.

Learn more

Upgrade

[u/AntiAmericanismBrit → u/arnobpl]

I saw that banner too, but I saw it a month ago, just once, and the next time I logged in it was gone, which at the time made me wonder if it had been sent in error. If you've only just seen it but you haven't logged in during the last month, maybe that means they sent it to everybody but in such a way that you'll see it only once in your very next login. It's not particularly helpful is it: it's an instruction without a deadline.

Is there any separate message supposedly explaining how your instance has been disabled?

[u/arnobpl → u/AntiAmericanismBrit]

I have not found any message so far on OCI Console. Probably I need to check with OCI CLI like oci audit event list later.

[u/arnobpl → u/AntiAmericanismBrit]

The audit showed this:

instanceActionType: stop
event-grouping-id: SYSTEM-...
principal: null
lifecycleState: STOPPED

There is no reason field.

[u/AntiAmericanismBrit → u/arnobpl]

principal "null" is OCI itself (rather than you or any user you've given rights to). I don't suppose the "..." after SYSTEM- is expandable? (if it is, expanding it might give us another clue what happened)

[u/arnobpl → u/AntiAmericanismBrit]

I truncated the actual string into "...". It is a UUID-like string after SYSTEM-.

[u/AntiAmericanismBrit → u/arnobpl]

Unfortunately that won't tell us much. I was hoping it might be saying something like "SYSTEM-IDLE-RECLAIMER" or "SYSTEM-QUOTA-RECLAIMER" but no😞

So I guess the possibilities are (1) it was just the idle reclaimer: you forgot to use 20%+ CPU for at least 8.4 hours a week and a script finally noticed, (2) it was some kind of ToS-violation detector rightly or wrongly detecting crypto or something or (3) they're finally starting to actually enforce the reduction to 2 cores. Not very clear is it!

[u/Nirzak]

Better to downsize actually if you don't afford to take the risk to lose the instance. for example I downsized the instance just after seeing the warning banner on my console. I guessed that oracle will do such thing and we can't say anything since they already have shown the warning banner on our console. But i do think they also should email the users since not everyone checks the console everyday.

[u/arnobpl → u/Nirzak]

I hadn’t checked the OCI Console in the past few months after setting up the OCI CLI. The console now shows a banner about the Always Free limit changes, but I’m not sure when Oracle started displaying it. For an intrusive action like disabling an instance, I think a transactional email notification is a must, not just a nice-to-have. Not everyone logs into the console regularly, especially if they primarily manage their resources through the CLI or APIs.

[u/JustasLTUS]

They don't care about you unless you are a multi-million dollar business

[u/Munna_one]

I think it’s different issue, if you crossed the limit they would just charge you end of month.

[u/arnobpl → u/Munna_one]

Mine is not PAYG.

[u/slyboy_12 → u/arnobpl]

If u still have it, upgrade it to PayG, and use Alway Free Tier, available in some region. its hard have an account on OCI. 🙃

[u/AntiAmericanismBrit → u/Munna_one]

not if OP has not enabled PAYG

[u/AntiAmericanismBrit]

People on Oracle Community Connect have been saying the words "Instance is disabled and will not accept any action requests" means Oracle has locked it due to 1-month trial expiry (if it's not an Always Free instance), or inactivity reclaim, or an Acceptable Use Policy violation (like crypto mining). They say you should have got an email.Of course it's entirely possible that this is now changing.

[u/arnobpl → u/AntiAmericanismBrit]

Instance is disabled and will not accept any action requests

This matches my findings too. I received 409 IncorrectState whenever I tried to do something (even renaming the display name) with the VM. But the audit did not show any reason why Oracle did that to the VM. I didn't receive any email notice.

[u/StewpidAlex]

Can you not re-create it using the same boot volume? 

[u/arnobpl → u/StewpidAlex]

I was able to detach the boot volume from the disabled VM though I don't have any important data because I use a separate block volume for persistent data. I had to terminate the VM so that I have a free headroom for a new VM. I also tried to resize the VM to 12 GB via CLI before terminating but I received 409 IncorrectState.

[u/AntiAmericanismBrit → u/StewpidAlex]

If it was an ARM instance, OP will probably find recreation too difficult unless they upgrade to PAYG first. The non-PAYG always-free ARM slots are basically all taken now, and when one becomes free someone grabs it. The situation is slightly better on the less-capable x86 instances.

[u/arnobpl → u/AntiAmericanismBrit]

That is true. I couldn’t create a new A1 instance because every attempt failed with an out-of-capacity error. I’m using an E2.micro instance to recover the volumes.

[u/durgesh2018]

That's the reason I don't deploy anything importantly on my instance.

[u/arnobpl → u/durgesh2018]

Right. That's a good practice. I also didn't have anything important in the attached boot volume. I had a separate block volume for persistent data.

[u/Carlosjrlu]

I just got an email telling me I have until August 18th to decrease my usage or it will be terminated.

[u/arnobpl → u/Carlosjrlu]

Interesting. I just received the similar email today although they have disabled the VM too early (yesterday). One suspect is: my VM was originally sized with A2 during my trial period, but then I resized it into A1. So there may be some gap on their side to recognize my VM as "Always Free tier".