Vittelius@feddit.org to Fuck AI@lemmy.world · 10 days agoai.robots.txt: A list of AI agents and robots to block. (and the configuration files to block them)github.comexternal-linkmessage-square10linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkai.robots.txt: A list of AI agents and robots to block. (and the configuration files to block them)github.comVittelius@feddit.org to Fuck AI@lemmy.world · 10 days agomessage-square10linkfedilink
minus-squareUlrich@feddit.orglinkfedilinkEnglisharrow-up0·9 days agorobots.txt doesn’t do any sort of blocking. It’s nothing more than a request. This is active blocking. Although I’m not sure how successful it will be, given the determination of these bots.
minus-squarezod000@lemmy.dbzer0.comlinkfedilinkarrow-up0·9 days agoA few of them are quite good at randomizing their user-agent and using a large number of IP blocks. I’ve not had a fun time trying to limit them.
minus-squareUlrich@feddit.orglinkfedilinkEnglisharrow-up0·9 days agoYeah dude, they’re extremely malicious and not even trying to hide it anymore. They don’t give a fuck that they’re DDOSing the entire internet.
robots.txt doesn’t do any sort of blocking. It’s nothing more than a request. This is active blocking.
Although I’m not sure how successful it will be, given the determination of these bots.
A few of them are quite good at randomizing their user-agent and using a large number of IP blocks. I’ve not had a fun time trying to limit them.
Yeah dude, they’re extremely malicious and not even trying to hide it anymore. They don’t give a fuck that they’re DDOSing the entire internet.