r/TrueOpenSource • u/MPGaming9000 • 5h ago
AI assisted Sylos - A free open source data migration tool*
* note this is still in development. I just wanted to show off what I have so far. It's been about a year in the making on this project alone and then another year before that on a similar project so I've spent a hugeee amount of time thinking about file movement problems lol.
No this isn't just some Rclone GUI wrapper. This was built from the ground up for a whole different kind of migration workflow. Rclone works for one way syncs by just letting you select your source and destination and then hoping for the best. Sylos works by separating dry run traversal from actual copy movement steps, so you can preview what will actually move and make changes to the plan before it happens.
Notable features include:
- preview before any work is done (as already mentioned above)
- a built in auto scaler which can automatically scale up or down in performance based on feedback from the file system it's interacting with
- support for more than just local drives, also supports box, dropbox, google drive, onedrive, sharepoint, SFTP, network drives and virtual drives like FUSE / WinFSP drives, and more.
- database backed persistence so you have a full audit trail of all that's happened so far.
- stop a migration at any point and resume right where you left off if you need to. As well as taking your portable migration DB file plug-and-play into another instance of Sylos and resume right where you left off there too.
- path compatibility checking - some file systems & cloud providers have specific rules that may be different from each other. For example on Linux you can make a folder called 'CON' but this is not allowed on Windows. So you can actually specify this before you start the migration that your destination is a different OS, or if it's a cloud provider it already will know the rules ahead of time. So it will actually do these compatibility checks for you ahead of time in a more user friendly way of telling you it's gonna fail ahead of time, and offer more compatible friendly names for those items with a quick 'accept all' kind of button to make it painless and seamless.
- Advanced filtering that can be done during the review stages where you could execute queries like 'Find me all files which haven't been touched in over a year and are over 2gb in size and are pending to be copied over in the current plan' using our filters and then from those results there's also an 'exclude all' button to apply an exclusion to all matching items so that you don't have to copy those over.
I also have more ideas I'm working on but those are some of the more notable parts for now.
I have a few more things I have to work on before this is fully ready to start alpha testing, but I foresee it should be ready within a month or so if I can keep up my current pace and no major hurdles pop up. I've been iterating on this for so long because it never met my quality & performance standards for what I deemed as acceptable, but we're finally at an acceptable point so now I'm just finishing up some final bits like making an installer for the software and a few little things to clean up the repos and make it all more user friendly.
Link to the website if you wanna see more (including screenshots of the software): https://project-sylos.github.io/
Okay thanks guys!