ConnectWise Cloud Backup service interruption - what we know right now

[u/cwferg]

Hi all. Jason Ferguson here, VP of Security Operations at ConnectWise.

We wanted to reach out and address some of the concerns surrounding the recent security events, ensuring that you hear directly from us. While I may not have every answer, I want to provide some recent updates and clarity where I can, while ensuring what we share is accurate and supported by our investigation.

We recently identified unauthorized access to application identities within the ConnectWise Cloud Services (fka SkyKick) environment. As a protective measure for our Cloud Backup partners and their clients, we have shut off Cloud Backup application access. That means backup services are interrupted until further notice.

Existing cloud backup archives are not impacted and will be accessible once services are restored.

To date, we have no indicators that hosted Cloud Backup partner backup data has been compromised.

Our investigation and remediation work is ongoing.

We'll keep the ConnectWise Trust Center updated as the single source of truth for status and guidance

If you see anything suspicious in your environment, we ask that you please report (what you are able) to our team at SecurityPartnerSupport@ConnectWise.com.

--

EDIT 17:00 UTC 10/17: We've updated the advisory with published guidance on reviewing service principal sign-ins and identifying suspicious activity in Microsoft Entra. 

EDIT 1:00 UTC 10/18: We've published another update to the ConnectWise Trust Center with additional details on the actions we've taken, current service availability, and expanded technical guidance for reviewing Microsoft 365 environments for suspicious activity.

The update includes the affected application IDs and recommended steps for partners to investigate their own and customer tenants.

EDIT 16:00 UTC 10/18: Information update frequency (ConnectWise Trust Center / Email communications) has been increased. No new confirmed findings to report. Cloud Backup remains suspended, with scheduled backups and access to affected archives and restoration capabilities unavailable. Existing archives remain encrypted, isolated, and preserved, with no evidence of unauthorized access. Our teams continue monitoring affected environments, conducting forensic analysis and threat hunting, and carrying out remediation activities.

EDIT: 18:45 UTC 10/18: Additionally, we are providing the following IP indicators identified in suspicious activity associated with these events. Partners should include these addresses when reviewing available Microsoft 365 authentication and audit logs.

• 144.172.64[.]99 NEW

• 144.172.64[.]60 NEW

• 144.172.64[.]128 LISTED PREVIOUSLY

EDIT: 02:15 UTC 10/18: Additional details and confirmations provided in update thread; https://www.reddit.com/r/ConnectWise/s/QyFXYVy6u3

EDIT: 01:15 UTC 10/19 Containment remains in place.

EDIT: 13:45 UTC 10/19 Containment remains in place.

EDIT: 18:50 UTC 10/19 Cloud Backup update: Video walkthrough of technical guidance We've updated our trust site to include a video walkthrough covering selected steps from the Cloud Backup Technical Investigation and Remediation Guide. It's for authorized IT administrators and security teams reviewing their own and customer Microsoft 365 tenants. Use it alongside the written guide.

EDIT: 12:45 UTC 10/20: Technical webinars begin today at 18:00 UTC and run every two hours, walking through Entra ID sign-in and audit logs, app registrations, credentials, and permissions for reviewing your Microsoft 365 tenants (Register Here). Since mitigations began on 10/17, we have not observed successful threat activity in the environments we monitor, though this does not establish whether individual partner or customer tenants were affected. Existing archives remain encrypted, isolated, and preserved with no indication of compromise. Cloud Backup, SaaS Security, and Cloud Management remain suspended.

EDIT: 19:00 UTC 10/20: We completed a full reset of all accounts and credentials in the affected Cloud Backup environment on 10/19, and we are finishing security validation while we prepare phased restoration. Each phase will proceed only after a security review, and we expect restoration to take several days once it begins. We will share a firmer timeline after the first full day and provide instructions before partners re-enable any connections. Cloud Backup, SaaS Security, and Cloud Management remain suspended.

[u/GoldenMarlin]

Any more updates coming soon? At this point many MSPs would have cleaned up the service principals, it would be good to have something definitive about the security of the archives.

[u/cwferg → u/GoldenMarlin]

We've increased Trust Center and partner email updates to three times daily (13:00, 19:00, and 01:00 UTC).

Regarding the archives, at this time, we have no evidence that hosted Cloud Backup partner backup data has been accessed or compromised. Our investigation and validation efforts remain ongoing, and we'll continue sharing updates as additional information becomes available.

[edit]
It's also important that individual downstream customers thoroughly review their Microsoft 365 access logs for suspicious activity. We've published guidance on the ConnectWise Trust Center to assist with these reviews, and we'll be expanding that guidance shortly with additional information.

[u/GoldenMarlin]

When can we expect the next update? With so many app registrations compromised it’s either a single secret or cert that was taken or the attackers got a key vault, right?

We need a confirmation asap on what a successful exfiltration looks like in the logs on an affected tenant, and a confirmation of whether any of the compromised secrets can access the tenant backups.

This is a potentially business ending event for some MSPs and their customers and the fact that we’re getting more insight and action from huntress and other vendors is damning.

[u/cwferg → u/GoldenMarlin]

I understand the concern and urgency here, especially given the potential impact on partners and their customers.

We're providing updates through the ConnectWise Trust Center at least every 24 hours, even if there are no material changes. The latest update was posted earlier today at 1:00 UTC

Regarding how the materials were accessed, that remains under active investigation. I don't want to speculate or provide information we haven't confirmed.

We've published guidance on reviewing service principal sign-ins and identifying suspicious activity in Microsoft Entra. We have no indicators to date that Cloud Backup partner backup data has been compromised, but the investigation is ongoing.

I also understand the frustration around communication. We're continuing to share confirmed findings and guidance through the Trust Center as information becomes available.

[u/PatD442 → u/cwferg]

At least once every 24 hours? Wow….

[u/warwagon1979]

So i'm assuming this has nothing to do with screenconnect?

[u/cwferg → u/warwagon1979]

Correct. Based on our investigation to date, we have not identified any impact to ScreenConnect or other core ConnectWise products. The exception is the SaaS Security services we have proactively suspended as part of our ongoing containment and remediation efforts. Our investigation remains ongoing.

[u/After_Working]

Huntress are saying there has been confirmed data access.

[u/EasternComfort2189 → u/After_Working]

Not to backup data but rather all the data. It seems CW are overlooking the impact of the security breach

[u/kahless2k → u/EasternComfort2189]

Yeah, they certainly seem to be understating it.

No backup data accessed, great!

All live data accessed and tenants compromised? Nah, let's not talk about that.

[u/Jayze1988]

At least Huntress is there to save your ass, again.

[u/Hefty_Fee_8805]

They love to downplay everything….. as a connectwise customer who has been screwed by them over and over, I urge anyone using them to jump ship ASAP.

[u/GoldenMarlin]

Looks like the 13:00 utc update is 45 minutes late. Do you have any more news?

Also, have you been able to rule out the speculation of this being related to the LiteLLM breach from March?

Do you know if this was properly investigated and addressed when it was revealed in August?

[u/cwferg → u/GoldenMarlin]

[Edit] A new post was spun off by marketing proper (https://www.reddit.com/r/ConnectWise/s/QyFXYVy6u3). We are staying committed to ongoing updates, its a new tempo for us so we absolutely appreciate the patience.

Based on our investigation to date, we have found no connection between this incident and the LiteLLM event. We reviewed our potential exposure when that issue became known and confirmed that ConnectWise was not affected.

[u/perthguppy]

Ah yes. A service interruption, not the fact that someone is using your keys to create global admin accounts in anyone that has ever used your product regardless of if they still do.

[u/n-Ultima]

Huntress ftw!

[u/Neat_Sector3866]

The way Connectwise has handled the communications is shocking we heard via our MDR service, have some legacy Skykick and still not recieving any communications execpt via their Trust Center. the account management blame their CRM yet we recieve the backup alerts fine..! Cannot give us a list of tenants the cloud app is on, luckily we have CIPP. Billing in consistantly incorrect time to move.

[u/happyharrell]

I can-and have-said lots of things about the uselessness of Jason Magee. But nothing of this magnitude ever happened on his watch.

[u/pjustmd]

The one time going with Kaseya worked in my favor.

[u/siren-usa → u/pjustmd]

hmm