csantosb@lemmy.ml to AI@lemmy.ml · 25 days agoAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comexternal-linkmessage-square4fedilinkarrow-up14arrow-down10 cross-posted to: [email protected]
arrow-up14arrow-down1external-linkAI haters build tarpits to trap and trick AI scrapers that ignore robots.txtarstechnica.comcsantosb@lemmy.ml to AI@lemmy.ml · 25 days agomessage-square4fedilink cross-posted to: [email protected]
minus-squarebountygiver [any]@lemmy.mllinkfedilinkEnglisharrow-up1·25 days agodid you even read the article? It specifically only traps scrappers that don’t respect the robots.txt and put them in an endless maze of garbage information. If you enter a site that clearly warns you “malware ahead”, that’s on you.
did you even read the article? It specifically only traps scrappers that don’t respect the robots.txt and put them in an endless maze of garbage information.
If you enter a site that clearly warns you “malware ahead”, that’s on you.