molave@reddthat.com to Selfhosted@lemmy.worldEnglish · 7 days agoRunning web sites makes me not want to - Lowkey Partylowkey.partyexternal-linkmessage-square48linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkRunning web sites makes me not want to - Lowkey Partylowkey.partymolave@reddthat.com to Selfhosted@lemmy.worldEnglish · 7 days agomessage-square48linkfedilink
minus-squareMonkderVierte@lemmy.ziplinkfedilinkEnglisharrow-up0·7 days agoYou can set a trap that isn’t or is forbidden in robots.txt. Anyone who ignores your robots.txt is not friendly.
You can set a trap that isn’t or is forbidden in robots.txt. Anyone who ignores your robots.txt is not friendly.