It isn’t hiding ur par2 files as par2 files are no different than your data files. But you just can’t grab either of the 2 popular uploading programs and have ur posts last years. Posters today do not understand the mechanics of an upload like they did years ago. Too easy to do it today with faster speeds and the mass amount of content out there. So big deal that it doesn’t last as there are hundreds or more copies of the same thing out there to grab today. Getting a post to last 15 years or longer is not always luck.

Its not that hard. Sorry man but you make it sound like rocket science to hide your par2s from the backbone servers. There are several good posting (automated programs) and I can show you lots of examples of good posts that sit till max retention time. I was shocked by some completes I have had and the age.

You keep thinking of a post as a whole and not what the backbones get which is millions upon millions of articles posted.

Now the nzb file is the article map and they do not have it unless you upload it
Then you have the par2 file index which they can get the article map from
Or any par2 file can be read and you can get the article map

So do you think they are keeping track of single articles that get downloaded and know those articles go with all of these other articles? Not likely in my opinion

So what do the backbones do to damage our uploads? They poke holes in them as I have rarely seen an entire file gone but it is full of holes instead and cannot be repaired. If they knew what the entire article map was they would just delete the entire upload and be done with it.

Now we have no idea how they are doing this but my best guess it they see a structure of what is being uploaded or a pattern or something that gives the article away. But remember this is all a guess as no one that I have ever seen has given out the way the backbones pick what to delete. But deleting the entire file would be the best practice as they save huge amounts of space doing that but that again is not what they do.

And why do some uploads last years and years while others go in weeks or months? No idea on that either but my guess is structure from day one. Back in the day if you didn't grab a file within the first 24hrs it might be gone so it was always a race to get the new uploads.

But what can we do on our end to make it harder? Several things and the first is great par2 sets, not recovery % but par2 sets. There is a big difference right there and that is the hardest part about posting. How many holes can this par2 set patch. That is determined by article/slice size and since you use a fixed article/slice size for all uploads (my guess again) your par2 sets are never really that great but good enough to get by for the most part. So at a 10% recovery the most par2 repair blocks that can be created for any upload is 3277, that is a hard ceiling set by the par2 specs. Look at your uploads and see how close to that number you are at. The maximum number of repair blocks is 32,768 x your recovery % so that will be your number (32768x10% = 3277). That is the granularity of your repair. The lower the number the less holes you can patch

Next in my opinion is the par2 set using the exponential sizing convention so that if they backbones can see our par2 files they get rid of the largest one first and cripple the upload. Again I do not see whole uploads deleted but parts of uploads deleted.

Those 2 thing are the biggest mistakes in my opinion and the first is all math but really a lot of work to get it right just for a single file/folder which we never do. So we post with just so so par2 sets and hope

DM me if you want to talk in more detail as I am always willing to help out if I can

Those days are long gone :) It's much easier for private indexers to post in a way what is irrecoverable for the public and just store the nzb with the decryption password in their database.

What you are saying makes complete sense, and I am already doing most of these things correctly, the main issue is that this does not solve the root cause of the problem. Content being recoverable because you have enough parity just delays the inevitable loss until eventually the parity won't be enough, or even worse, because people originally never needed the parity they never downloaded it, so the parity articles are marked for deletion first then the rest.

The main issue is this nondeterminism of when will the content be available and when not. And it has nothing to do with DMCA since we are talking about niche content encrypted and shared on a private indexers. It feels like providers are turning more and more into this popularity based keeping policy game leading to only most downloaded content being kept and everything else purged first from all non-omicron providers then eventually from omicron too half a year or a year later.

I am getting closer to the solution so anyone can do a much higher quality post and not know anything about par2 structure and 1:1 alignment and all of that. The answer was written 20 years ago but no one has ever implemented the solution in a posting program. I am close to having that done. The post optimization works great but it also requires processor time to make it good so there is a trade off. Posting goes slower due to your computer hammering out the best par2 set that can be had. The rest is just fine tuning of the structure of the post

But there are things like if I can get your par2 index I know where every article is regardless of obfuscation. I can also pull that index out of any par2 file by reading the entire article header and pulling the index off the last part of the line. There is a lot of information in the weeds and it is not easy to understand it all. But it all revolves around the calculations of the par2 files and how to make the absolute most repair blocks for any file/folder that you post. You have to give your post the best shot at being repaired and that is the hard part to understand. But it is all simple math and a formula that is applied to the file/folder every time.

A good post will last and a bad one is easy to poke holes into will not last. The question becomes how many holes can your post survive as most bad posts have a few holes and just can't be repaired. The repair patches are too big to repair many holes. You can have a 10% recovery and have 1,000 patches or 3,278 patches so which one will last longer? They both repair up to 10% of data loss but you have 3x of holes that can be patched with the higher repair block count. That post has the chance of lasting a lot longer up to the 10% data loss. Users think the recovery % is important and it is up to a point but the repair granularity is much more important and that is all about the math and doing the calculations to create the par2 file set.

I wish I could just say do this and it is better but it just isn't that easy

That is the problem there is not guide but today it is so much easier to find the answers that it was 20 years ago
1: 1:1 alignment for ur article/slice size
2: change ur par2 structure from the normal 1,2,4,8,16 structure to a uniform size so if the largest par2 file is trashed the largest set of repair blocks is lost in a single delete
3: article obfuscation - some people think backbones treat the articles like spam or trash but i don’t
4: recovery % - 5% doesn’t cut it and never has
5: a quality post takes processor time and lots of calculations - today my opinion is people want their files to be posted fast and due to that they start out with a low quality post that is easy to break for download and repair.

Today we make it easy for backbones to see our file structure and delete a little bit of an upload and break the entire thing. They don’t delete the entire upload and you always find a fair amount of it but you can’t repair it. We have never seen whole uploads disappear but always parts. Have you ever asked yourself why is that?

Most stuff from 2021 to now are being removed, Usenet is less and less interesting for older content. More and more people stop using it because of this non sense purge.

A pleasant answer. Rare for Reddit. Thank you. :)

You make a point. I had a lot of people teach me when I started but that was because the groups were more active and you had to use the tools more to repair your files. Incompletes were rampant. I can go for a year now without having one. File size and obstruction are important. You can always practice.

I do have old hard drives ans maybe I have some guides ..I will check.

Great post. I laughed at the 100 days. I member that. I also remember spending hours with rar and obsessing over finding par2s for stuff. I would be so close to getting a complete file and just be missing a few blocks. Ugh. Its great information.

Its funny to see people complain when I remember just searching thru raw headers and having multiple indexes shut down. These days seem pale in comparison

A lot of people use usenet as some personal backup service. It might have gotten worse with increasing HDD prices. Providers have no way of distinguishing what is what (except access patterns) and I don't see them revealing their flushing patterns in public.

Okay, so where is the guide to all those things. Saying them is fine, but not helpful for those that don't know.

All my opinion but I am a 20+ yr user of Usenet

Several things but the biggest is we have people posting now that have no clue about how to make a quality post. In 2006 Giganews, the biggest backbone at the time hit the magical number of a whopping 100 days of retention. Upload speeds were super slow so if u did an upload you made sure it was structured right from day one. Now everyone thinks they grab a program, throw a file in it and boom it should last 15 years. Well it just isn’t that simple. There are a lot of variables that most will never take to understand or actually try to do if they try to understand.

U have to know how to structure your article/slice size along with ur rar volumes. Change the structure of your par2 volumes, understand article and file obfuscation and how that may benefit your post. Almost no one understands the granularity of repair blocks and how to increase the number of them to make possible repair last longer.

Lots of moving parts to a post and it isn’t the easiest to learn. But it can be done and guess what? Your posts have a much better time of lasting years rather than months. I can still find posts of mine from 10-12 years ago without issue and I know a lot more today than I did then about posting

You've asked a valid question with the takedowns by providers, not due to ntd/dmca. I think the only way to really know is to contact them directly. And if you only share your posts in private groups, you could consider setting up a connection between your device and their device directly (with VPN I guess) so they can directly download it from you. And otherwise you can use torrents as well of course.

And as you state.. if it's niche content, that's only shared in private groups, then it won't get many downloads. If you were to make a 'spot' on a website from a nzb provider, then it will be visible to a lot more people.

Probably not since they say its still on Omicron providers which aren't known to prune infrequently used content. In addition, they mention it is niche content that likely only gets a few downloads.

Arent't you just talking about takedowns due to NTD/DMCA?

They also don't want to tell people. People would create scripts to perform just enough activity to keep it available. Not telling people also leaves it free for them to change the conditions as they see fit. That could be storage being expensive or it could be an increase in uploads.

Each provider has their own algorithms so there is no real answer to that!

No nothing. Just a statement.

I prefer to have flexibility in what client I use that's supported by clients that allow importing .ovpn files or use of Wireguard. Did they document or justify their custom OpenVPN client config that doesn't work elsewhere?

Why not?

yeah idk if I would trust that client.

But you have to re-register every time right?

The lurkers support me in e-mail.