Do they actually respect that? Did you saw the requests going away/being stuck in redirects? I always expected them to use a generic user agent if that happens. I mean they are arguably already disregarding copyright? Why should they adhere to a standard.
Do they actually respect that? Did you saw the requests going away/being stuck in redirects? I always expected them to use a generic user agent if that happens. I mean they are arguably already disregarding copyright? Why should they adhere to a standard.