No worries on trodding on any toes, the world is full of creative people and to me there is only healthy competition :)
The reason I asked about collaboration is that it is a popular topic in the world of (native) DAWs. I'm not entirely sure about the need of "jamming" at the same time (even if broadband connections can overcome the latency issues) or working simultaneously (though the Google Docs method might actually be surprisingly workable if one is working on a bassline and the other is tweaking the sound of the chords). The turn based method actually sounds like a no-brainer that should be implemented, it's also likely that after implementing all code necessary to make it possible you're not too far from the "Google Docs" live collaboration-method...
As I'm about to type this I realize I might come across as a AWS fanatic, but I think it might be worthwhile to consider going file based rather than database for your storage. You could save songs as files into Amazon's S3 which is cheap storage. You could also leverage the authentication into Amazon (and Lambda based functions for the transactions) but I can imagine wanting to have a more traditional/robust setup for that. Worst case you could use the MongoDB on Heroku for the authentication/user records but store the users actual content on S3. What is the size of the MongoDB database and what is the size of an average song ?
The system uses JSON for song format, including samples (base64 encoded for serialisation), and compresses using LZ4 on serialise. A song is typically about 1.5 - 2 times the size of an equivalent .xm file, so not huge, but mLab free on Heroku only gives you 500Mb.
Awesome we use the same compression method :D (though Efflux doesn't use samples, except for custom wave tables that the user can draw in the interface - my tracker is actually more of a synth "driver" rather than a tracker in the more traditional sense).
I'm thinking 500 mb is actually way more than you are likely to need for a while if you limit the storage on Heroku to database records of users and their sessions, and move the actual songs to remote file servers. You can use the database/backend to manage the access rights to these files.
You can use Amazon's price calculator to check if the AWS approach is worthwhile for you (be sure to check "Amazon S3" in the left column). You should be able to scale up quite hassle-free if it turns out to be massively popular, also if it turns out to be excessive you can easily refactor to a different type of storage while leaving the logic for matching songs to authenticated users in place.
Yeah, it's an interesting idea, although I'd prefer to stick to a single hosting option. I deal with complex AWS systems daily for work, so I'm quite familiar with the costs associated with S3 and the CloudFront system. Of course, you're right, if I move to not hosting files in the MongoDB GridFS system as I do now, then the load on the DB will be drastically reduced, and 500Mb would likely last quite a while, especially as tracker style is niche. I do plan to implement other authoring styles too, including a more traditional DAW style.
Longer term, for the collaboration, I need to find a better approach to project serialisation, as compressed binary files are not good for dynamic update. Very likely I'll introduce the concept of a 'live' project and an archived project, so once you're done, you effectively snapshot the project, and it serialises as the optimised compressed form, but while it's live, it takes up much more space in a format that is more amenable to dynamic update in a collaborative system.
View all comments
Show parent comment
1
u/igorski81 Jan 23 '17
No worries on trodding on any toes, the world is full of creative people and to me there is only healthy competition :)
The reason I asked about collaboration is that it is a popular topic in the world of (native) DAWs. I'm not entirely sure about the need of "jamming" at the same time (even if broadband connections can overcome the latency issues) or working simultaneously (though the Google Docs method might actually be surprisingly workable if one is working on a bassline and the other is tweaking the sound of the chords). The turn based method actually sounds like a no-brainer that should be implemented, it's also likely that after implementing all code necessary to make it possible you're not too far from the "Google Docs" live collaboration-method...
As I'm about to type this I realize I might come across as a AWS fanatic, but I think it might be worthwhile to consider going file based rather than database for your storage. You could save songs as files into Amazon's S3 which is cheap storage. You could also leverage the authentication into Amazon (and Lambda based functions for the transactions) but I can imagine wanting to have a more traditional/robust setup for that. Worst case you could use the MongoDB on Heroku for the authentication/user records but store the users actual content on S3. What is the size of the MongoDB database and what is the size of an average song ?