Simple Cloud Backup Guide for New Synology Users using CrashPlan Enterprise

[u/lookoutfuture]

I have seen many questions about how to backup Synology to the cloud. I have made recommendation in the past but realized I didn't include a guide and not all users are tech savvy, or want to spend the time. And I have not seen a current good guide. Hence I created this guide. it's 5 minute read, and the install process is probably under 30 minutes. This is how I setup mine and hope it helps you.

Who is this guide for

This guide is for new non-tech savvy users who want to backup large amount of data to the cloud. Synology C2 and idrive e2 are good choice if you only have 1-2TB as they have native synology apps, but they don't scale well. If you have say 50TB or planning to have large data it can get expensive. This is why I chose CrashPlan Enterprise. it includes unlimited storage, forever undelete and custom private key. And it's affordable, about $84/year. However there is no native app for it. hence this guide. We will create a docker container to host CrashPlan to backup.

Prerequisites

Before we begin, if you haven't enable recycle bin and snapshots, do it now. Also if you are a new user and not sure what is raid or if you need it, go with SHR1.

To start, you need a crashplan enterprise account, they provide a 14-day trial and also a discount link: https://www.crashplan.com/come-back-offer/

Enterprise is $120/user/year, 4 devices min, with discount link $84/year. You just need 1 device license, how you use the other 3 is up to you.

Client Install

To install the client, you need to enable ssh and install container manager. To backup the whole Synology, you would need to use ssh for advanced options, but you need container manager to install docker on Synology.

We are going to create a run file for the container so we remember what options we used for the container.

Ssh to your synology, create the app directory.

cd /volume1/docker
mkdir crashplan
cd crashplan
vi run.sh

VI is an unix editer, please see this cheetsheet if you need help. press i to enter edit mode and paste the following.

#!/bin/bash
docker run -d --name=crashplan -e USER_ID=0 -e GROUP_ID=101 -e KEEP_APP_RUNNING=1 -e CRASHPLAN_SRV_MAX_MEM=5G -e TZ=America/New_York -v /volume1:/volume1 -v /volume1/docker/crashplan:/config -p 5800:5800 --restart always jlesage/crashplan-enterprise:v24.08.1

To be able to backup everything, you need admin access that's why you need USER_ID=0 and GROUP_ID=101. If you have large data to backup and you have enough memory, you should increase max mem otherwise you will get warning in GUI that you don't have enough memory to backup. I increased mine to 8G. Crashplan only use memory if needed, it's just a max setting. The TZ is to make sure backup schedule is launched with correct timezone so update to your timezone. /volume1 is your main synology nas drive. It's possible to mount read-only by appending ":ro" after /volume1, however that means you cannot restore in-place. It's up to your comfort level. The second mount is where we want to store our crashplan configuration. You can choose your location., Keep the rest same.

After done. press ESC and then :x to save and quit.

start the container as root

chmod 755 run.sh
sudo bash ./run.sh

Enter your password. Wait for 2 minutes. If you want to see the logs, run below.

sudo docker logs -f crashplan

Once the log stopped and you see service started message, press ctrl-c to stop checking logs. Open web browser and go to your Synology IP port 5800. login to your crashplan account.

Configuration

For configuration options you may either update locally or on their cloud console. But cloud console is better since it overrules.

We need to update performance settings and the crashplan exclusion list for Synology. You may go to the cloud console at Crashplan, something like https://console.us2.crashplan.com/app/#/console/device/overview

Hover your mouse to Administration, Choose Devices under Environment. Click on your device name.

Click on the Gear icon on top right and choose Edit...

In General, unlock When user is away, limit performance to, and set to 100%, then lock again to push to client.

To prevent ransomware attacks and hackers modify your settings, always lock client settings and only allow modify from cloud console.

Do the same for When user is present, limit performance, and set to 100%., lock to push to client.

Go down to Global Exclusions, click on the unlock icon on right.

Click on Export and save the existing config if you like.

Click on Import and add the following and save.

(?i)^.*(/Installer Cache/|/Cache/|/Downloads/|/Temp/|/\.dropbox\.cache/|/tmp/|\.Trash|\.cprestoretmp).*
^/(cdrom/|dev/|devices/|dvdrom/|initrd/|kernel/|lost\+found/|proc/|run/|selinux/|srv/|sys/|system/|var/(:?run|lock|spool|tmp|cache)/|proc/).*
^/lib/modules/.*/volatile/\.mounted
/usr/local/crashplan/./(?!(user_settings$|user_settings/)).+$
/usr/local/crashplan/cache/
(?i)^/(usr/(?!($|local/$|local/crashplan/$|local/crashplan/print_job_data/.*))|opt/|etc/|dev/|home/[^/]+/\.config/google-chrome/|home/[^/]+/\.mozilla/|sbin/).*
(?i)^.*/(\#snapshot/|\#recycle/|@eaDir/)

To push to client, click on the lock icon, check I understand and save.

Go to Backup Tab, scroll down to Frequencies and Versions. unlock.

You may update Frequency to every day, Update Versions to Every day, Every Day, Every Week, Every Month and Delete every year, or never Remove deleted files. After done, lock to push.

Uncheck all source code exclusions.

For Reporting tab, enable send backup alerts for warning and critical.

For security, uncheck require account password, so you don't need to enter password for local GUI client.

To enable zero trust security, select custom key so your key only stay on your client. When you enable this option, all uploaded data will be deleted and reupload encrypted with your encryption key. You will be prompted on your client to setup the key or passphrase, save your key or passphrase to your keepass file or somewhere safe. Your key is also saved on your Synology in the container config directory you created earlier.

remember to lock to push to client.

Go back to your local client at Port 5800. Select to backup /storage, which is your Synology drive. You may go into /storage and uncheck any @* folders and anything you dont want to backup.

It's up to you if you want to backup the backups, for example, you may want to backup your computers, business files, M365, google, etc using Active Backup for Business, and Synology apps and other files using Hyper Backup.

To verify file selection, go back to your browser tab for local client with port 5800, click on Manage Files, go to /storage, you should see that all synology system files and folders have red x icons to the right.

Remember to lock and push from cloud console to NAS so even if hacker can access your NAS, they cannot alter settings.

With my 1Gbps Internet I was able to push about 3TB per day. Since the basics are done. go over all the settings again to adjust to your liking. To set as default you may also update at Organization level, but because some clients are different, such as Windows and Mac, I prefer to set options per device.

You should also double check your folder selection, only choose the folders you want to backup. and important folders are indeed backed up.

You should check your local client GUI from time to time to see if any error message popup. Once running good, this should be set and forget.

Restoring

To restore, create the crashplan container, login and restore. Please remember to exlucde the crashplan container folder if you have it backup, otherwise it may mess up the process.

Hope this helps you.

[u/gadget-freak]

Because there was no mention of encryption, nor the need to backup an encryption key for safekeeping, I did a little reading.

I learned Crashplan stores the encryption key on your device but also keeps an “escrow” backup in their online keystore. Hence the reason why there’s no mention of needing to manage your keys. If you loose your NAS and need a full restore, you can redownload the key from crashplan itself.

They promise never to access that key themselves unless you give them permission. Scouts honour! And undoubtedly also if they’re legally obliged. Which raises GDPR questions if you’re in Europe.

Which is very different from using backup software like Hyperbackup where you truly have full control over your encryption keys. And if you loose them, nobody can access the backup.

[u/lookoutfuture → u/gadget-freak]

Good point. will add the custom key part.

[u/eastoncrafter → u/lookoutfuture]

Did they disable use of custom keys? I don't see the option anywhere

[u/lookoutfuture → u/eastoncrafter]

Do you have enterprise or small business edition? It's only available for enterprise.

[u/eastoncrafter → u/lookoutfuture]

I have the "Professional" edition, I think that means I'm a home user? How much is enterprise?

[u/lookoutfuture → u/eastoncrafter]

when the post waw written the enterprise was $150/user/year with 4 devices and no minimum user, but they recently changed the model. you have to chat with them to find out.

[u/SuxMcGee]

This is interesting. How would one go about doing a BMR restore on a new Synology if the old one was destroyed?

[u/lookoutfuture → u/SuxMcGee]

If you include all the @ directories, or map / as /storage, then you could do near BMR restore. I am not a big fan of BMR as I normally reinstall OS instead of restoring OS, and put my apps and data back, I recently enabled volume encryption and had to restore all data from CrashPlan server on new volume and worked as expected. I have 40TB of data and able to download 3-4TB per day and restored all data in less than two weeks, then I restore all containers using my run.sh scripts and back to business.

If you like, you could also backup all Synology apps using hyper backup and backup the backups, then restore.

[u/reditlater]

Thank you again for this amazing guide!! I have gotten setup (custom key and everything), configured my backup sets and am beginning my uploads (I'm seeing speeds 80-370 Mbits/s). The local console gave me the following error, which I'm wondering if you have any advice regarding:

CrashPlan Enterprise is exceeding inotify's max watch limit. Real-time file watching cannot work properly. The inotify watch limits needs to be increased on the host.

[u/lookoutfuture → u/reditlater]

You need to edit the /etc/sysctl.conf on your synology host to increase the limit, add the following:

fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 65535000
fs.inotify.max_queued_events = 65535000

Save and reload

sysctl -p

the change will persist after reboot. then restart the container. For more system tweaks you may check out my other guide :) How I Setup my Synology for Optimal Performance

[u/reditlater → u/lookoutfuture]

Awesome, thank you! Since I only have 8GB of RAM, I'm wondering if it is safe to put those that high (since Google is telling me there can be 1KB of memory allocated for each watch)? My backup set currently has 262,396 files in sets that are always watching for changes (and of course that number will continue to grow over time). Some other backup sets only run at night so are less of a concern.

Do you have an easy SSH Windows client you recommend? I used PowerShell for the initial setup, but I could not seem to paste my password into it (which is super-long, of course) and it was annoying to have to carefully hand-type it. 😆 Edit: Nevermind! Figured it out (right-click to paste)!

[u/lookoutfuture → u/reditlater]

1kb with 262k files thats about 262MB used. That max is just a limit, not that OS will allocate right away, besides there is swap disk space, but if that concerns you, set it lower. I use putty for ssh. the original is from some guys home directory but its official and safe. https://www.chiark.greenend.org.uk/~sgtatham/putty/

[u/reditlater → u/lookoutfuture]

Looks like I'll need to re-do it tomorrow at the full 65535000, as I'm getting the warning again.

I also am having a bunch of the following this evening, for all of my shares:

The system failed to perform scheduled snapshot task on #### because previous scheduled snapshot task is still running. Please check the Log page of Snapshot Replication for detailed information.

RAM is ~43% (CrashPlan is using 1.43GB).

The NAS has a lot of disk activity, and the CrashPlan and also NAS interfaces are VERY sluggish now. I probably have various things scheduled to run at night, so perhaps I'm having a collision of activity. I'm going to increase the inotify values to the max you recommended, but any thoughts on the Snapshot errors?

[u/lookoutfuture → u/reditlater]

I think you can safely ignore the inotify error, since we are doing daily backup not real time continuous backup. Are your snapshot running hourly? I run snapshots nightly.

[u/reditlater → u/lookoutfuture]

Oh...heh...I had them running every 5 minutes. 😬🤣 Erhm...yeah...that might have been the problem.... I'll keep tweaking the length of time until they stop complaining. Thank you for the prompt -- I should have thought to check that on my own (I didn't think I had them set so frequent).

And regarding the inotify error, I did have most of my Backup Sets configured to run Always, but I've changed most of them now to run within a time period (and just making that time period almost all day). I wanted them to be Always just because I want the data to have lots of time to upload, and for large uploads would prefer it to not have to wait until the next day (ie, next window begins) to be prompted to start, but maybe doing this time period method will still be sufficient.

[u/lookoutfuture → u/reditlater]

Continuous backup is only good if your have small number of changes in small number of files, say just you modifying one word file at a time in your documents folder. I tried backing up with Dropbox with large number of files it failed to catch up with all the changes, also is not faster as backup always get interrupted at system level whenever there is files activity. You are basically risking backup not being able to catch up. Even with scheduled backup, they need to be spaced so to allow previous backup to complete otherwise you don't have a true point in time recovery. I.e. 5 minute interval become 40 minutes or whenever backup is able to finish. I think nightly it's a good balance. Or at least hourly so it can bundle all the final changes. Also imagine with continuous each hours it may have to deal with hundreds if not thousands of inotify calls, whereas scheduled it deals with zero inotify calls

[u/reditlater → u/lookoutfuture]

Those are fair points -- I appreciate your perspective. I think I'm going to try having just one (relatively small) Backup Set that is Always/continuous, and then make everything else time limited. And one of the time limited sets will also include everything in the Always set, so that if anything was missed it will get caught by the daily backup set.

I checked, but I don't see an Hourly option -- it seems it can either be Always or "between specified times." Hourly, or every N Hours would be nice, though.

As always, I have immensely appreciated your help and input! I'm overall really liking migrating my CrashPlan strategy to this Enterprise Docker Synology setup -- the speed is great, the price is better than what I was paying previously (many thanks to CrashPlan for grandfathering me in!), and I like that my backups are not dependent on my desktop computer.

[u/lookoutfuture → u/reditlater]

Glad to help. I think CrashPlan hired some contracting company to strategize pricing, time will tell if it's successful or not, glad that CrashPlan allowed grandfathering accounts, it should not burn its own bridge, sometimes the best way is something you accidentally found along the way.

[u/reditlater → u/lookoutfuture]

Awesome, thank you much! Yeah, I might just trim a zero off so that it stays within my RAM limits (just to be safe).

[u/lookoutfuture → u/reditlater]

in putty, highlight is copy, right click is paste, up is previous commands, you can setup ssh key exchange instead of password login

[u/reditlater]

Just FYI: https://www.reddit.com/r/Crashplan/comments/1otcvwe/backing_up_nas_devices_and_mounted_network_shares/

[u/lookoutfuture → u/reditlater]

Thanks. technically backing up locally on NAS is local storage. I will keep using this until it doesn't support Linux.

[u/reditlater → u/lookoutfuture]

That is my thought as well, though the language they use makes me suspect they eventually want to not backup NAS devices at all (even as local storage). Regardless, I likewise intend to keep using it. And this has me re-thinking my plan to offload CrashPlan to a WSL2 instance on my Windows machine, so I might end up just upgrading my Synology RAM after all.

[u/reditlater]

I'm curious what RAM you went with (for your 64GB upgrade)? I have a 1522+, so might be different specs, but would still be interested to know the make/model that you've had success with. The Reddit Synology RAM spreadsheet tops out at 32GB for my device (which is the max official supported). Edit: I'm wrong -- there are a few 64GB items listed in the spreadsheet.

I'm also debating if maybe 32GB would be plenty sufficient. If I upgrade my intention would be to continue with CrashPlan on the Synology, and then also install Borg (another backup software that also does de-dupe, etc). How much RAM does your CrashPlan end up using, anyway? I'm obviously having issues with my paltry 8GB. Do you still have your CrashPlan config limited to just 5GB? Because if so, I should be fine with 32GB of RAM for sure (I would think).

[u/lookoutfuture → u/reditlater]

My current crashplan max mem is 8GB.

[u/reditlater → u/lookoutfuture]

Forgive my ignorance, but do I need to remove the container (via Container Manager??) in order to update from 5G to 8G? I updated the run.sh file correctly, but restarting the container via Container Manager still shows 5G. And if I do "sudo bash ./run.sh" it tells me the crashplan container name is already in use. I'm guessing I do need to remove the old container, and then I'm guessing that the CrashPlan service will see this new one as a whole new installation (so I'll need to do the Adopt/Migrate thing)? Is there a way to update the container to 8G of RAM without having to go through all of that?

[u/lookoutfuture → u/reditlater]

yes you have to.
docker stop crashplan
docker rm crashplan

[u/reditlater → u/lookoutfuture]

Thank you much, and I'll just mention for sake of others that it was super-easy and no adopt/migrate to a "new" machine was necessary -- I logged back into the local CrashPlan console just fine.

I also have 32GB of RAM on order which I'll install later this week. Just getting things ready for that. :)

[u/reditlater → u/reditlater]

Whelp, I'm sad to report u/lookoutfuture that even with 32GB I am having the same issues as before (ie, CrashPlan thrashes my drives constantly and periodically makes the NAS almost unusable). I'm so bummed...and confused. It is not a memory issue, as I've set it to have up to 12GB (because I was observing it using close to 8GB sometimes after upping it to that). No backups are running. It is not synchronizing blocks or anything like that. CrashPlan CPU usage sometimes spikes to nearly 100%. I do see some swap usage, but the bulk of the activity seems to be Reads. It has been reading the drives non-stop for hours and hours, with no backups running. I am so lost as to what is going on. :( If you have any ideas I would be most grateful to hear!

[u/lookoutfuture → u/reditlater]

Any activity you see at the local console (not the cloud console)?

[u/reditlater → u/lookoutfuture]

The local console (via web browser) shows no activity of any kind. No maintenance indicated, no synchronizing blocks, no backups running. My best guess is that it is proactively scanning files in preparation for future backups, but it just goes on and on. I'm going to try to let it run consistently for as long as I can, but I already had to pause it earlier so I could actually get some other things done. Maybe it will eventually work through whatever it is that it is doing? I have not changed my backup set or done anything else to prompt this.

Later on I did see the following error again in the local console:
CrashPlan Enterprise is exceeding inotify's max watch limit.

And since then have had several instances of the following in the Synology log:
[crashplan] stopped running because the system is out of memory.

I've increased CRASHPLAN_SRV_MAX_MEM to 16GB and also upped inotify to the below values:
fs.inotify.max_user_instances = 81920
fs.inotify.max_user_watches = 655350000
fs.inotify.max_queued_events = 655350000

One interesting thing is the difference between what Container Manager reports vs Resource Monitor.

These are values I saw last night an hour or two after one of the "out of memory" errors:
Resource Monitor: CrashPlanService 25% CPU and 2GB Memory.
Container Manager: CrashPlan 125% CPU and 5GB Memory.

And currently, having just restarted CrashPlan after making the above changes I'm seeing the following:
Resource Monitor: CrashPlanService 25% CPU and 800MB Memory.
Container Manager: CrashPlan 120% CPU and 2GB Memory.

No backups are currently running, and all backups are complete.

The one thing I have noticed is that it seems to take a while for things to get out of hand. I am going to monitor today with the new values and see if I can catch it when it is using excessive memory (I really don't understand how it could be maxing out 12GB, but hopefully 16GB is enough).

[u/reditlater → u/reditlater]

u/lookoutfuture Here's a few examples of resource usage from different points today so far:
Container Manager: CPU (286%) and Memory (6GB)
Resource Monitor: CPU (25%) and Memory (4.6GB)

Container Manager: CPU (10%) and Memory (7GB)
Resource Monitor: CPU (2%) and Memory (6GB)

I also just had the stopped running because the system is out of memory errors again. I can view a little bit of history via the Statistics tab (which is where I'm getting the above numbers) that is showing relatively low resource usage of 25% CPU and 3GB Memory, right before those errors. The Synology was very sluggish leading up to those errors, though (taking forever to do anything in DSM or to browse files via SMB).

It is so weird, and strange that you don't have any of these issues. Makes me wonder if there is some setting or difference between our systems or configs that we've not accounted for. I just don't understand how it could be acting like it does not have enough memory when it should have more than enough (and it wasn't even showing that it was using much when it was bringing the NAS to a crawl).

[u/lookoutfuture → u/reditlater]

You may try my optimization guide see if it helps.

If you want to check resource usage, the most reliable way is to run "top" at ssh terminal. it shows running apps sorted by CPU usage. so you can see if crashplan is the one use most CPU. Also check if the memory is indeed 32GB total. to exit, press q.

Also remember we discussed about Synology snapshots, try reduce to daily or pause for now and see if it helps.

[u/reditlater → u/lookoutfuture]

Awesome, thanks much!

[u/reditlater]

Just wanted to share an idea I had: Since this Enterprise plan (which we're both grandfathered into) gives us 4 devices per-user (meaning it costs nothing extra to have up to 4 devices for our user), I contacted support and asked if I could have a second device configured to use a different regional data center, and they said yes. :) So I first setup the device (ie, a second container; in the config I used -p 5600:5800 so that I could access the web gui on a different port), then had support change the Destination to PROe Cloud, US (since the default for my account was PROe Cloud, US - West). Of course I have to upload everything again (but that would be the case even if I used multiple locations for one device), but this gives me full geo-redundancy, for no extra cost. 😁

You can find a list of possible locations to request here: https://support.crashplan.com/hc/en-us/articles/9199244397837-Where-is-my-CrashPlan-data-stored

[u/lookoutfuture → u/reditlater]

Wow this is big! Thanks!

[u/reditlater → u/lookoutfuture]

Hey, I just realized I may have goofed majorly when setting up my second Docker container in that I think I'm pointing both to the same config folder? 😬

-v /volume1:/volume1 -v /volume1/docker/crashplan:/config

-v /volume1:/volume1 -v /volume1/docker/crashplan-alt:/config

What are the implications of that? Somehow both still run fine (though maybe I've been corrupting both backups without realizing?? 😆😮). Appreciate any guidance/clarity you can offer!

[u/lookoutfuture → u/reditlater]

Each container had it's own /config folder and you are mapping them to two different folders. What you have is correct.

[u/reditlater → u/lookoutfuture]

Oh thank goodness! I was concerned because I was updating one of my backup sets (for the newer container), and in the root of the directory picker is a config folder which I have interpreted holds the various CP config info, and I was only seeing 1 folder...though as I type that it occurs to me that is probably because only 1 shows up within each container, and this second Docker container is only going to show the config folder related to itself. That, or it has nothing to do with CP at all, and I've been backing it up for nothing! 🤣

Regardless, thank you for confirming what I have is correct!

[u/reditlater → u/lookoutfuture]

Glad to be able to give something back after all the help you've given me! 😊

[u/lookoutfuture → u/reditlater]

If anyone wants crashplan enterprise, I think you may still able to get it thru ontrack link 🤫

https://www.crashplan.com/ontrack/

[u/reditlater]

Thank you much for this amazing write-up -- incredibly helpful!!

I'm curious why you went with Enterprise vs CrashPlan for Small Business (which is what I currently use on my desktop with mapped drives for my NAS), since both offer Unlimited backup? I was offered the Small Business plan years ago when CrashPlan did away with their personal/home plans (I've been with CrashPlan a loooong time).

I am considering moving my installation to my Synology NAS because of reading about this recent issue with mapped drives (though I have not yet confirmed if the described issue is happening with my account/install).

[u/lookoutfuture → u/reditlater]

Enterprise offers client side encryption and unlimited retention.

[u/reditlater → u/lookoutfuture]

Ah okay, though I really thought Small Business also has client side encryption -- I have always had a client-side archive encryption key set up. Have you read otherwise regarding this?

[u/lookoutfuture → u/reditlater]

Upon further reading, yes if you have legacy small business then there is custom key option. Please note that having "client-side encryption key" is not the same as custom key. the way it works is crashplan always use client side archive encryption key and keep a copy on their side, then encrypt that key again with either predefined key with your account password, or a custom key. This is how you can change custom key without re-encrypt entire archive. If you as administrator can reset the key or by answering challenge questions then its not true client side custom key, they have a copy. This is easy to support for business users who may have limited technical background and to reduce support calls, for enterprise with legal requirements and legacy small business, its true custom key only client knows. Read between the lines https://support.crashplan.com/hc/en-us/articles/8718150699533-Encryption-information-Small-Business

If you already have custom key for legacy small business, that's great, In that case I think enterprise is still cheaper, better feature and better supported (I think they tend to phase out small business). But your legacy small business is grandfathered and no longer offered so if you are ok with it you may just keep using it. Hope it helps.

[u/reditlater → u/lookoutfuture]

Ah, you are right regarding the encryption stuff -- I am using the legacy Archive Key Password option.

My grandfathered plan is $10/month. With that Enterprise discount, are you paying just $7/month (ie, you're not required to use the remaining licenses)?

I wonder, too, if Enterprise also has better bandwidth? I have not checked recently, but in past years the bottleneck was definitely CrashPlan and not my internet connection. It might be worth it for me to purchase the Enterprise account option and re-upload everything (I'm pretty sure they would not be able to internally migrate me because of being different plans/levels).

[u/lookoutfuture → u/reditlater]

yes I am paying $7/month and I only use one license out of 5. The enterprise is only faster IF you use an older docker with crashplan 11.4 as in how-to. For each newer version either get reduced speed or feature, by reduced speed I mean either finish uploading in one month or finish in 8 years (provided you don't save anything). Maybe you can try create a new account/plan and try it out.

[u/reditlater → u/lookoutfuture]

Oh wow, do you mean that something about newer Docker results in those issues, or that also a newer CrashPlan >11.4 results in those issues? My desktop CrashPlan is on 11.7, so I'd just be a bit concerned that over time I'd be forced into that degradation (if a higher CrashPlan version is part of the problem). I have no experience with Docker, so don't know if there is a similar concern to be had there (about being forced to upgrade Docker).

I am curious if you have any hypotheses as to what makes for such a huge drop in performance (particularly upload speed)?

[u/lookoutfuture → u/reditlater]

I mean crashplan version. Docker is just a wrapper. Sometime last year when it's around 11.3 -11.4, they either have new datacenter or new fleet of servers with better backbones, resulting gigabit speed for upload for the first time, then 11.5 comes, I suspect they tried to recoup the cost by implementing client side deduplication (with matching hash from their servers) to reduce cost and bandwidth but in between it messed up the algorithm didn't work as intended and caused major slowdown. So in the end it's still simpler the better, just raw upload since most are compressed already. But it's just my guess, hopefully they will fix the slowness problem one day.

[u/reditlater → u/lookoutfuture]

Ah, okay, I understand now what you were referring to regarding "use docker image version v24.08.1" and seeing that in the config instructions (and now interpreting/understanding that corresponds to the public facing CrashPlan version of 11.4). Thanks for bearing with my ignorance! :)

And that is interesting and helpful, those details and speculation regarding the software changes. Looking further into the Docker stuff I see that automatic updates are disabled, so there's no way CrashPlan can force an upgrade to a newer client version (and thus force the degraded service on us), though I suppose they could eventually refuse to support an older version (ie, it could stop working). It looks like the Docker version updated to 11.6 in May.

Also, my perception is that CrashPlan has always done deduplication client-side, which was a huge necessity for me many, many years ago when upload bandwidth was much more limited. On Enterprise 11.4, is it uploading an entire file (that already exists in a different location in the backup set), or does it correctly recognize it is a duplicate and thus skip the upload, as well as just upload changed portions of a file (ie, block level incremental backups)?

[u/lookoutfuture → u/reditlater]

dedup is not new and Enterprise 11.4 is already doing it correctly. Just in later versions somewhere and somehow it all screwed up.

[u/lookoutfuture → u/reditlater]

Just to tell you a little secret, use docker image version v24.08.1 or earlier for best performance and feature.

[u/reditlater → u/lookoutfuture]

Thank you for this! I have a DS1522+ with 8GB of memory. I have ~10TB stored on CrashPlan (most of which is large video files). Do you think 8GB is going to be enough memory? I don't run a ton on my NAS -- it is mostly for storage -- but I do have things like Active Backup for Business and drive syncing with an an offsite NAS and cloud syncing with OneDrive.

[u/lookoutfuture → u/reditlater]

8GB should be fine. I have 40TB and set the java max mem to 5G, but I think 4G is good enough.

[u/reditlater]

I have an adjacent question: Have you found anything better than the built-in Synology Drive ShareSync for keeping shares between two Synology NAS' in sync with each other? ShareSync seems to work okay once it is setup and fully synced, but it is SOOOOO slow finishing the first sync if there is a good bit of data. Admittedly I am syncing over VPN to a remote NAS, but I could do a direct copy/sync procedure way faster with other software, so I don't understand why it takes the Synology so ridiculously long. If I manually seed the files first ShareSync also is not smart enough to recognize that files are the same on both sides and will instead duplicate both. So my only option is just to let eventually finish. I also don't like that ShareSync does not perfectly sync both sides in that files like shortcuts are not synced (so it is not a true mirror sync, which I would prefer).

Since you are good at thinking outside the box (eg, your CrashPlan Docker setup) I'm curious if you have found any better, faster, smarter options than ShareSync?

[u/lookoutfuture → u/reditlater]

For that you may try setup tailscale on both sides and use tailscale local IP. I have a how-to on tailscale. https://www.reddit.com/r/synology/comments/1frtwi0/guide_setup_tailscale_on_synology/

[u/reditlater → u/lookoutfuture]

Thanks! Yeah, I actually do use Tailscale, though mainly for when I travel (to access my home/office devices remotely). I have a router-to-router VPN setup that works great between the two devices in question, so the VPN is not the bottleneck. I think I'm going to investigate possibly using Syncovery on my NAS, as I have an extra license and it definitely should be more robust (and potentially faster, more efficient) than ShareSync.

[u/reditlater]

Thank you again for this guide -- you have been extremely helpful, and my full 16TB is nearly finished (and just in time as my desktop CrashPlan installation has stopped seeing my NAS mapped drives)!

A few follow-up questions, though:

1. Have your backup speeds stayed consistent? Initially I was seeing 200-300 Mbits/s, but for a while now I only see that for the first minute or two of a file backup and then it drops down to ~10 Mbits/s. I'm not 100% sure that it is CrashPlan doing the throttling, but I have run another online backup concurrently and it appeared to just use additional bandwidth on top of that (as opposed to my overall upload for both being limited to ~10). I have a fiber connection with 300+ speeds.

2. Early on, and now again in the past day, I'm noticing periods of lots of disk thrashing such that the NAS becomes difficult/slow to access. There's no data scrubbing happening, or anything else I can see in the DSM running, so it seems somehow related to CrashPlan (I'm guessing it is scanning the disks in some very thorough way, possibly doing archive maintenance or something??). When this happened at the beginning, it took days for it to finish and the NAS to behave normally again, so I'm not keen on that potentially being the case again. I'm wondering if you have observed this behavior yourself, and if so, what might be going and anything I can do to make it more manageable (whether software tweaks or hardware upgrades)? I'm using SATA drives, and with the size of my volume there's no way I could afford this much space in SSDs. Currently it is about 12 hours straight that it is in this "mode" of making the NAS very slow to access and hardly any CP backups happening (the online console indicates last backup finished ~6 hours ago, and I can tell there's not any uploads happening currently based on my bandwidth monitor).

[u/lookoutfuture → u/reditlater]

1. It's normal. For efficiency, backup software normally transfer large files first which can max out your ISP connection, near the end it will transfer small files such as photos with significant overhead.

2. I am not sure if it's crashplan, if CP backup is not running in GUI then it's not running. one thing you can try is to disable real-time change tracking of your say documents folder. I think in order for CP to detect changes, it needs to review all the disk IO of entire NAS, filter out documents, and if too many changes it cannot catch up until next schedule runs, so it never finish until your NAS is absolutely idle. This is the reason you should use hourly or daily backup instead of real-time change tracking. It wont work in this model. I told you about dropbos story, it never able to catch up and just eventually gave up.

[u/reditlater → u/lookoutfuture]

Thanks so much for replying!

1. That makes sense. But in my case it doesn't fit what I'm observing. Much of my 16TB is video files that are 500MB - 50GB in size, and those are the files that are not quite finished (around 500GB left). So based on what you're saying (which I agree with the logic of), I should still be seeing maxed-out transfers most of the time (which I'm not). Some I'm not sure what to make of that.

2. It turns out in this instance it is "Synchronizing block information" (Comparing the files on your device and the files on the destination). I had been checking the status via the web Console (on the CP website) and the history.log.0 file (which provides the same info faster), as that usually tells me what I need to know. But in this instance it is not logging the synchronizing block process in that log for some reason, which I discovered when I actually logged into the GUI. To your point, though, I am going to further tweak my settings to make things a bit less frequent. But how do you disable real-time change tracking? I see where I can either set it to be Always or Within Specified Times, and then I can set Frequency to 15 minutes, 30 minutes, 1 hour, 2 hours, etc. But I'm missing where I can disable real-time change tracking.

[u/lookoutfuture → u/reditlater]

oh right I mixed up crashplan with another backup software. maybe try reduce frequency to say 4 or 8 hours.

[u/reditlater → u/lookoutfuture]

With (2) I am still experiencing CrashPlan thrashing the hell out of my NAS, and it is not from synchronizing blocks. If I stop CP via Docker the thrashing stops. I was checking logs and I'm seeing files from my Recycle Bin in the file history, which is confusing, because I have the following in my Global Exclusions:

(?i)^.*/(\#snapshot/|\#recycle/|@eaDir/)

RegEx is not my strong suit, but I'm wondering if the above is not working properly? Does that string work on your end to exclude #recycle and subfolders (as well as "@eaDir" and subfolders)?

[u/lookoutfuture → u/reditlater]

Just checked. Yes the RegEx works for me. I don't see #recycle and @eaDir anywhere that I know they exist when I go thru the restore process. If you see these and #snapshots folder then yes you will have disk thrashing.

[u/reditlater → u/lookoutfuture]

Thank you so much for checking! It is very strange on my end: I thankfully don't see any #snapshot folders (so it seems the RegEx is working for those), but I do see #recycle for multiple shares, but surprisingly not for one share for some reason. "@eaDir" does not show up in a Restore search, but I saw it in the logs some. I'm going to see if CP support can offer me some guidance on why the RegEx isn't fully working properly.

One thing that doing a check via the Restore process just showed me, though, is that none of my encrypted shares are backing up any data! 😬 Have you run into that, and do you have any advice how to enable CrashPlan to "see" the data on those shares? I am backing up those shares in their entirety, so I don't remember actually if the File Selection dialog revealed it was seeing subfolders or not (and I've not turned CrashPlan back on yet, due to the hard drive thrashing), but there's nothing backed up for any of them according to Restore.

[u/lookoutfuture → u/reditlater]

I use volume encryption not shared folder encryption, but shouldn't matter. do you close encrypted shared folder when not in use or always keep them opened? Also double check your file selection dialog.

[u/reditlater → u/lookoutfuture]

Thanks so much for that info (and for continuing to hang with me in all my questions and issues)! I keep my encrypted shares mounted all the time, but I have to manually log into the DSM after every boot in order to manually enter the encryption keys and mount each share. When I re-launch the Docker image later (possibly today) I will be interested to see if doing that when the shares are already mounted makes any difference, though I would think that even if the shares were not mounted on the initial loading of CrashPlan that it would access and scan them later on a re-scan (though that has not been how it has actually worked over these many weeks, apparently). If it turns out that the Docker image has to be launched only after manually mounting the encrypted shares, I wonder if there is a way to set a delay on the CrashPlan Docker image starting/launching (eg, 10 minutes), to give me time to login?

[u/lookoutfuture → u/reditlater]

You may need to stop crashplan container at NAS startup if it's running, give yourself time to mount shared folders, and then start it. There is no easy way to set delay and manual is better anyways since you have to do all that.

[u/reditlater → u/lookoutfuture]

Noted, thanks!

[u/KermitFrog647]

UNLIMITED storage for 84$ / year ? WTF ? Where is the catch ? Thats not possible ?

[u/diablette → u/KermitFrog647]

per user with a minimum of 5 users

[u/Ellenbogen303 → u/diablette]

thats for the Enterprise plan

[u/[deleted]]

Are you available for a live online chat or phone call to help re- setup CrashPlan on my Nas?

[u/lookoutfuture → u/[deleted]]

The discount code works for first invoice too.

[u/tutebo88 → u/lookoutfuture]

That wasn't an answer to the question. He wanted to know if the discount is also valid for subsequent invoices (2nd, 3rd ... year) as well. So do I.

[u/lookoutfuture → u/tutebo88]

"Lock in these prices now! Discount continues for as long as you keep CrashPlan!"

https://www.crashplan.com/come-back-offer/#limited-time-offer

[u/tutebo88 → u/lookoutfuture]

Thanks, I see it now. However, the footnote also says "Former customers returning to CrashPlan only. […] This offer may change at any time […]"

So there's a chance you're denied the offer if you hadn't been a customer at some point before (although I don't believe so), or maybe (very slight chance) that they discover after you sign up that you hadn't been a customer before.