Archive.today Blog

same, even over tor interesting that the spanner is a base64 gif in the source, not a separate file subdomain blog (last updated 2 years ago) is still up: https://blog.archive.today/

More than a theory -- they've talked about this on their blog[1] before. [1] https://blog.archive.today/

>What scraper or headless browser are you using? it works so well. >Before 2019 - PhantomJS, after - ordinary (not headless) Chromium/80 with few small patches. https://blog.archive.today/post/618635148292964352/what-scraper-or-hea… (2020) >Archive.today launches real browsers (not even headless) and tries to load lazy images, unroll folded content, login into accounts if prompted with login form, remove “subscribe our maillist” modals https://blog.archive.today/post/642952252228812800/people-often-compar…

Why don’t you ask him? https://blog.archive.today/ Also last I heard it was only $3k a month.

Noticed that too. I've assumed that the root cause is the Russian aggression/war/invasion. https://blog.archive.today/ is perhaps are good place to ask that question. (I've somehow gotten the impression the person running this service is either in Russia or at least russian.)

http://archive.fo/K5Fpv > .IS domain will likely stop working soon[0] [0] https://blog.archive.today/

Archive.today has a different approach to the baseline archive technology (executing javascript at archival time and saving the DOM instead of saving and replaying server responses verbatim). Additionally, Archive.today employs a number of site specific mitigations which aren't visible to the end user. In some cases, for instance, they use accounts, but then retroactively modify the DOM to mask this mitigation. [0] While the exact strategy they use for Twitter isn't known to me, they are doing something by their own admission. [1] [0] https://blog.archive.today/post/708008224368001024/why-isnt-instagram-… compounded with personal observation. [1] https://blog.archive.today/post/708565142782246912/pretty-please-remov…

The blocks tend to be implemented at the DNS level, so you can usually connect by using a different DNS server. The Tor onion link works fine as well, and that URL is on the archive.today Wikipedia page. https://en.wikipedia.org/wiki/Archive.today > archiveiya74codqgiixo33q62qlrqtkgmcitqx5u2oeqnmn5bpcbiyd.onion via this post on their blog: https://blog.archive.today/post/711271973835227136/did-something-chang…

Clickable links: [0] Archive.today: on the trail of mysterious guerrilla archivists of the Internet - https://news.ycombinator.com/item?id=37009598 - Aug 2023 (58 comments) [1] "archive.today is doomed to die" - https://blog.archive.today/post/657822115776610304/not-respecting-peop…

archive.today or archive.is - Wikipedia: https://en.wikipedia.org/wiki/Archive.today Help:Using archive.today - Wikipedia: https://en.wikipedia.org/wiki/Help:Using_archive.today archive.today - FAQ : https://archive.md/faq archive.today - wiki : https://wiki.archiveteam.org/index.php/Archive.today Archive Team wiki : https://wiki.archiveteam.org/ archive.today - Blog : https://blog.archive.today/ Tumblr : https://archive-is.tumblr.com/ Twitter : https://twitter.com/archiveis archive.today archive.ph archive.is archive.li archive.vn archive.fo archive.md archiveiya74codqgiixo33q62qlrqtkgmcitqx5u2oeqnmn5bpcbiyd.onion Launched May 16, 2012; 11 years ago

According to their blog they use AMP: https://blog.archive.today/post/675805841411178496/how-does-archive-by…

Doesn't archive.today basically replace the ads with their own? https://blog.archive.today/post/680555809289224192/why-are-there-ads-o…

archive.ph also uses (donated) logins to archive (paywalled) content, however those accounts do get blocked from time to time. https://blog.archive.today/post/678202832257794048/why-cant-instagram-…

Works for me. Do you have CloudFlare DNS? https://blog.archive.today/post/681415425930723328/%D0%BF%D1%96%D0%B4%…

https://blog.archive.today/post/623568857709395968/i-from-the-region-w… There was another answer I could not find quickly where that is named here "another free dns service" was named Amazon.

To add, apparently another reason is that he believes using Cloudflare as your recursive resolver could lead to phishing[0]: > the same entity which answers your DNS queries is able to issue SSL certs for any domain, so using CloudFlare DNS you never know whether you access the original website or a fishing one Generally this is protected via certificate transparency+CAA records. If CF's CA were to issue a bad certificate, it'd be blocked by the browser and, should it get out, jeopardize the entire company, likely DigiCert as well given they cross-signed Cloudflare's issuing CA. 0: https://blog.archive.today/post/634795612966125568/when-will-your-site…

Also some interesting numbers in this post: >How much does hosting cost you per month at the moment? >about ~$2600/mo of pure expenses on servers/domains, not counting “work time”, “buying laptop/furniture”, etc. ($100…300/mo covered by donations + $300…500 by ads) https://blog.archive.today/post/659383959382294528/you-said-that-befor…

No, they run their own crawler against the URL you provide (IIRC, it's a headless Google Chrome [1]). For some special / often-requested (?) sites the people behind the project have: - implemented post-processing to strip ads and tracker beacons - bought legitimate accounts that they use to bypass paywalls (some of them show up as remnants in the archived pages' code, some of them show up when you request an archive of the site's account management portal, some show up as HTTP traces during the archival process - the crawler will dump every URL it requests on a "WIP" page, and for some, auth information like JWTs is passed in the URL) - implemented anti-nagwall features (some sites are "vulnerable" in their paywall/"you read X articles, end of free limit" by pressing the ESC key to disrupt the JS of the site) [1] https://blog.archive.today/post/618635148292964352/what-scraper-or-hea…

Brave got a tap on the shoulder from the government and was told they had to collect all that data in order to release crypto payments to the owners of the websites they'd been collecting micropayments for: https://preview.redd.it/52bkflf0cld31.jpg?width=462&format=pjpg&am… https://support.brave.com/hc/en-us/articles/360032158891-What-is-KYC- I disagree with a lot of Brave's approach. However their experience shows that this problem isn't theoretical. One company has already managed to deliver frictionless micropayments, and was told by the government that if they didn't add some friction back in (AML/KYC), executives would be going to prison. Not cool. This also led to a real mess with the operators of archive.today, who don't live in the USA and apparently aren't supported by the AML/KYC process being used. So a fairly large amount of money collected in order to support them is stuck and can't be paid out: https://blog.archive.today/post/626174398020403200/please-provide-all-…