I was trying to archive some posts on the
other "Bitcoin Forum" and quickly found out that is explicitly an disallowed action.
https://bitco.in/robots.txtUser-agent: ia_archiver
Disallow: /
User-agent: *
Disallow: /designertest/
Disallow: /forum/attachments/
Disallow: /forum/forums/*/?direction
Disallow: /forum/forums/*/?order
Disallow: /forum/help/
Disallow: /forum/login/
Disallow: /forum/lost-password/
Disallow: /forum/members/
Disallow: /forum/misc/
Disallow: /forum/online/
Disallow: /forum/posts/
Disallow: /forum/register/
Disallow: /forum/search/
What is more interesting is that the owners of the "bitco.in forum" have taken the time to explicitly block the archival site:
https://archive.is/For a site that claims:
Welcome to Bitcoin Forum.
No censorship. No BS.