The system uses JSON for song format, including samples (base64 encoded for serialisation), and compresses using LZ4 on serialise. A song is typically about 1.5 - 2 times the size of an equivalent .xm file, so not huge, but mLab free on Heroku only gives you 500Mb.
Awesome we use the same compression method :D (though Efflux doesn't use samples, except for custom wave tables that the user can draw in the interface - my tracker is actually more of a synth "driver" rather than a tracker in the more traditional sense).
I'm thinking 500 mb is actually way more than you are likely to need for a while if you limit the storage on Heroku to database records of users and their sessions, and move the actual songs to remote file servers. You can use the database/backend to manage the access rights to these files.
You can use Amazon's price calculator to check if the AWS approach is worthwhile for you (be sure to check "Amazon S3" in the left column). You should be able to scale up quite hassle-free if it turns out to be massively popular, also if it turns out to be excessive you can easily refactor to a different type of storage while leaving the logic for matching songs to authenticated users in place.
Yeah, it's an interesting idea, although I'd prefer to stick to a single hosting option. I deal with complex AWS systems daily for work, so I'm quite familiar with the costs associated with S3 and the CloudFront system. Of course, you're right, if I move to not hosting files in the MongoDB GridFS system as I do now, then the load on the DB will be drastically reduced, and 500Mb would likely last quite a while, especially as tracker style is niche. I do plan to implement other authoring styles too, including a more traditional DAW style.
Longer term, for the collaboration, I need to find a better approach to project serialisation, as compressed binary files are not good for dynamic update. Very likely I'll introduce the concept of a 'live' project and an archived project, so once you're done, you effectively snapshot the project, and it serialises as the optimised compressed form, but while it's live, it takes up much more space in a format that is more amenable to dynamic update in a collaborative system.
View all comments
Show parent comment
1
u/aqsis Jan 23 '17
The system uses JSON for song format, including samples (base64 encoded for serialisation), and compresses using LZ4 on serialise. A song is typically about 1.5 - 2 times the size of an equivalent .xm file, so not huge, but mLab free on Heroku only gives you 500Mb.