dramaticcat to Lemmy Shitpost@lemmy.world · 1 year agoChad scraperimagemessage-square95fedilinkarrow-up11.07Karrow-down128
arrow-up11.04Karrow-down1imageChad scraperdramaticcat to Lemmy Shitpost@lemmy.world · 1 year agomessage-square95fedilink
minus-squareViscloReader@lemmy.worldlinkfedilinkarrow-up75arrow-down2·1 year agoJust a heads up for companies thinking it’s wrong to scrap: if you don’t want info to be scraped, don’t put it on the internet.
minus-squareDAMunzy@lemmy.dbzer0.comlinkfedilinkarrow-up17·1 year agoBut, but, norobots.txt! Chad doesn’t care!
minus-squarePoolloverNathan@programming.devlinkfedilinkEnglisharrow-up6·1 year agouses curl manually instead
Just a heads up for companies thinking it’s wrong to scrap: if you don’t want info to be scraped, don’t put it on the internet.
But, but, norobots.txt!
Chad doesn’t care!
uses curl manually instead