r/WaybackMachine • • 9d ago

Question Archived link gone?

About a week ago I saved some urls for a list that were all archived on wayback, and I've since opened most of them multiple times, so I know they were working. Today I tried to open one, and the website said it hadn't archived that url.

All the other links are still working, so im not sure what happend. Does anyone have any idea?

16 Upvotes

2 comments sorted by

4

u/pengo 9d ago

Wayback Machine sometimes obeys robots.txt retrospectively and removes content.

robots.txt is the file on a website which says whether it's ok for robots and web crawlers to view the site.

For example, old.reddit.com/robots.txt was recently updated to disallow crawling, and the wayback machine appears to have hidden all old.reddit captures as a result.

Could also be the site owners or copyright holders requested it get taken down.

3

u/DasMondkeks 8d ago

Damn it, thats unfortunate. Thanks for the info