• MonkderVierte@lemmy.zip
    link
    fedilink
    English
    arrow-up
    0
    ·
    7 days ago

    You can set a trap that isn’t or is forbidden in robots.txt. Anyone who ignores your robots.txt is not friendly.