• daq@lemmy.sdf.org
    link
    fedilink
    English
    arrow-up
    1
    arrow-down
    3
    ·
    14 days ago

    I’m not sure how they actually implemented it, but you can easily block ML crawlers via cloud flare. Isn’t just about every small site/service behind CF anyway?

    • grysbok@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      14 days ago

      Last I checked, cloudflare requires the user to have JavaScript and cookies enabled. My institution doesn’t want to require those because it would likely impact legitimate users as well as bots.

      • daq@lemmy.sdf.org
        link
        fedilink
        English
        arrow-up
        1
        ·
        14 days ago

        Huh? I can reach my site via curl that has neither. How did you come up with this random set of requirements?