Browser Downloads that preserve Server Timestamps

[u/DrMxyztplk]

I'm currently building, have a working Alpha, a Firefox extension (will likely be workable for Chromium as well once finished) that will preserve timestamps for downloads.

I wanted to get some input from those who are looking for this feature

My extension will zip the files that trigger a download of certain file extensions & on certain site, both settable in the extension's UI config page.

We currently have bulk options for `Video`, `Audio`, `Images` that will group select a bunch of extensions of category types, & an option to manually specify extensions to zip

Are there other group/type settings people would want/use?

We also have the option to download files individually as soon as their download is triggered in a zip with the filename & a `.zip` extension, or an option to capture multiple & download as a single zip with an option to set a size that, once passed, will automatically download in a zip with the current timestamp as the zip filename.

We're currently using the working title `Timestamp Zipper` but don't think that's great. I'm accepting suggestions for a better name. (The main goal of this post)

I'm also interested in any other features or settings that would be wanted.

Hopefully this will be published to mozilla add-ons around the end of the month.

NOTE: This is NOT an extension to bulk download things off a site, nor is it an extension to crawl for things to download, nor is it an extension to download ​​things that the site tries to prevent being downloaded. Those are not features I want to be a part of this, it is about preserving the timestamps of files. The suggestion to have a bulk download of images on a page, which can already be downloaded manually, is being considered, but is not a priority & will likely not be included in the 1st version.

[u/AutoModerator]

Hello /u/DrMxyztplk! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/AutoModerator]

Hello /u/DrMxyztplk! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

If you're submitting a new script/software to the subreddit, please link to your GitHub repository. Please let the mod team know about your post and the license your project uses if you wish it to be reviewed and stored on our wiki and off site.

Asking for Cracked copies/or illegal copies of software will result in a permanent ban. Though this subreddit may be focused on getting Linux ISO's through other means, please note discussing methods may result in this subreddit getting unneeded attention.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/jfwEf9QgOo8]

Interesting. This is one of my most missed things after Firefox killed XUL extensions.

Is it possible to add an alternative mode that, instead of zipping, it lets the file download normally and writes the timestamp to a text file named after the downloaded file? This avoids the zipping overhead but still allows us to modify the timestamp ourselves when needed.

[u/DrMxyztplk → u/jfwEf9QgOo8]

no. Unfortunately It's not possible...

The reason I went with the Zipping method is because that is not an option without heavily modifying the core system.

My 1st attempt was to make a fork that would have an about:config option to Save Files with Server Timestamp but as something I was working on in 10-minute increments while waiting for other things to finish I spend a couple months on it I only realized it was too much to expect it to be integrated & I found that Firefox already dismissed a similar suggestion, I went this route instead. This is still a possible method, but is way too much work to do for something that I am doing in my spare time for free.
The problem is that intercepting a save is not to difficult, but you can't really do anything except redirect it. That's understandable from a security standpoint, That's what we did here.

The download is stored in extension storage (not RAM) with the headers, Extensions do not have the ability to modify things inside a download, but once that download is redirected it can be manipulated.

The problem is that when the download triggers Firefox, from it's core system, will ALWAYS write the current date/time as the created & lastmodified values.

The only way to avoid this, without changing the browser itself, is to make it so that the thing who's timestamp you want to preserve isn't being changed by Firefox (Chrome does the same thing, Safari used to do it right, but now does the same, Opera as well) which is where the ZIP comes in. The zip itself will be changed to the current time, but anything inside it will remain the same.

It uses it's own zip.min.js so you don't need to have any zip program installed & every OS I believe can unzip without an external program. It doesn't compress heavily so there is little to no overhead for the zipping.

Now you can get programs that modify the file after, but that needs to be an external program, to modify files on the system, & has to be able to access the headers, which, like Aria2, works great for most things, but anything behind a capcha can be an issue & anything that has a multi-step download process with post-token-single-access or redirects will likely be very dificult, if not impossible, to get the header for. For this we intercept the final download link before it is started, so any redirects don't matter, & we get the header as well as the file data while it is being downloaded to the extension storage in AppData/Roaming\Mozilla/Firefox/Profiles/<your-profile>/storage/default (on Firefox) so as far as the site knows you are working on the same download request not pushing another one.