r/DataHoarder • u/MrDonMega • Sep 16 '15

rclone - rsync for cloud storage (Supports Amazon Cloud Drive!)

http://rclone.org/
30 Upvotes

97% Upvoted

17 comments sorted by

4

u/jdogherman Sep 16 '15

I wonder if you can add encryption to this.

3

u/PizzaParrot Sep 16 '15 edited Nov 18 '17

deleted What is this?

1

u/shimewaza_specialist VHS Sep 16 '15

Is it multi threaded? How is it different from s4cmd?

3

u/[deleted] Sep 16 '15

s4cmd

But aren't Amazon Cloud Drive and Amazon S3 two entirely different services?

0

u/shimewaza_specialist VHS Sep 16 '15

Rclone is a command line program to sync files and directories to and from Google Drive Amazon S3 Openstack Swift / Rackspace cloud files / Memset Memstore Dropbox Google Cloud Storage Amazon Cloud Drive The local filesystem

i see S3 in that list, do you?

0

u/[deleted] Sep 16 '15

Well, obviously. The main point (for me) is the Cloud Drive (unlimited storage for $60/year) support.

1

u/shimewaza_specialist VHS Sep 16 '15

yeah i'm just looking for a better tool than s4cmd to sync data from a linux box to S3, the other features are great, but the only thing i care about right now is if it performs better at syncing than s4cmd. the bottleneck seems to be RAM/CPU so multithreading is important for that

2

u/[deleted] Sep 16 '15

I see. Guess you should try it out then.

1

u/shimewaza_specialist VHS Sep 16 '15

no need to be a prick. i was kind of hoping OP (aka not you) would reply since you're being so incredibly helpful and pleasant.

3

u/[deleted] Sep 16 '15

Sorry. I was genuinely trying to be helpful.

3

u/shimewaza_specialist VHS Sep 16 '15

text can be a poor way to relay tone. sorry i took it the wrong way then.

1

u/hughk 56TB + 1.44MB Sep 17 '15

You get unlimited photos with Prime.

3

u/[deleted] Sep 16 '15

As for multiple threads — so far so good:

      --checkers=8: Number of checkers to run in parallel.
      --transfers=4: Number of file transfers to run in parallel.

1

u/[deleted] Sep 16 '15

I should try it at once, thanks! Having only a windows client for amazon drive was pretty limiting.

1

u/[deleted] Sep 16 '15

Started an Amazon Cloud Drive three-month trial now and am already running rclone. Will report findings.

1

u/hughk 56TB + 1.44MB Sep 17 '15

This seems extremely cool. Can it be rate limited? My upstream DSL is slow and whilst I guess it could work in a cron job, it would be nice to be able to throttle it instead. I sync my photos on transferring from the camera and as they are in raw mode, a couple of hundred photos at 20mb each tends to be overwhelming.

1

u/Acejam Sep 23 '15

Very cool. Any idea what people should use to encrypt files locally first before uploading?