Support Opting out of GPTBot Crawling | World Anvil

Support Opting out of GPTBot Crawling


Remove these ads. Join the Worldbuilders Guild
Go back to the Voting Board

Support Opting out of GPTBot Crawling

New Feature Addition · World management · Created by notavi
accepted
chatgpt crawlers

What functionality is missing? What is unsatisfying with the current situation?

While OpenAI supports a way to opt-out of having your work crawled via robots.txt but since that is sitewide it obviously isn't something that can be managed on a per world or per account basis.   Documentation here: https://platform.openai.com/docs/gptbot   Ideally there would be an option in world settings that could be set that would block crawling. Since it's probably too cumbersome to put an entry in robots.txt for each individual world (especially given its 500kb limit) it may be necessary to change the world URL either for opted in or opted out worlds - for example:   robots.txt User-agent: GPTBot Disallow: /wn/*   Where worlds that choose to block crawling would have their world URL change slightly to match the pattern by changing the /w/ in their base URL to /wn/. A similar approach could be taken if this feature is opt out (block for /w/, but allow crawling for /wc/).   An alternate option would be to simply return a 403 Forbidden based on the user agent whenever the GPT crawler attempts to access a world that does not wish to be crawled by AI. This would work well as either an alternate implementation or a supplemental control.

How does this feature request address the current situation?

It allows creators who don't want their work crawled by OpenAI or used as AI training data some ability to prevent that.

What are other uses for this feature request?

As is, there aren't any other uses for this request but an extended version of this feature request might provide multiple levels of opt-in (All Crawlers, Search Only or No Crawlers) depending on the creators preferences.

The Team's Response

Thanks for your suggestion and to everyone who participated! Unfortuantely, the current solution provided by ChatGPT doesn't work for us, as we have a large amount of users with different opinions regarding this matter and don't want to implement an all-or-nothing solution. That said, we are accepting the suggestion pending a better solution from them. If they implement a method we can use on a user level, we will implement it. While we don't have a specific stance about AI, we know that many of our users do, and we'll always defend your work and ownership.   Note that private content can't be crawled by bots, so depending on your usecase this could be a workaround.
Login to Vote
Current score

161/300 Votes · +39507 points

Votes Cast

  • +300

    by ThePouil
    on 2023-09-01 11:35

  • +300

    by AnxiousArachnid
    on 2023-09-01 08:13

  • +300

    by raptor973
    on 2023-09-01 04:27

  • +300

    by Familiars-Table
    on 2023-08-31 23:54

  • +300

    by MKHETH II
    on 2023-08-31 17:34

  • -300

    by Bonus Action
    on 2023-08-31 07:54
    Unpopular opinion: If this website supports the use of AI in creative work, it does not really have a right for an option to block it on the other side. That is hypocrisy. I would only support this if WorldAnvil takes a stand (as many other creative platforms have done) concerning the use of AI, especially in contests and competitions.

  • +300

    by Codykusy
    on 2023-08-31 06:48

  • +300

    by Kwyn Marie
    on 2023-08-31 04:42

  • +300

    by warchild1x
    on 2023-08-31 04:06

  • +100

    by Observer of the Stars
    on 2023-08-30 21:50

  • +300

    by Imper1um
    on 2023-08-30 21:09

  • +100

    by A Beloved Goblin
    on 2023-08-30 15:04

  • +300

    by Angantyr
    on 2023-08-30 14:48

  • +100

    by Kebrevo
    on 2023-08-30 10:19

  • +300

    by PurpleFeathers
    on 2023-08-30 02:53

  • +300

    by Cobblers95
    on 2023-08-29 19:59

  • +300

    by R.W. Winton
    on 2023-08-29 19:00

  • +100

    by Fearandramen
    on 2023-08-29 17:48

  • +300

    by Gam1ngSk1ttle
    on 2023-08-29 16:05
    Yes, please. People's creative works should be actively protected on this site, and have an opt-in feature.

  • +300

    by LordAsterion
    on 2023-08-29 07:12
    default should be opt-in!

  • +300

    by lkmorris
    on 2023-08-29 06:49

  • +300

    by A Wild Elf
    on 2023-08-28 22:34
    Default shoudl be opt-out

  • +300

    by A Adorable Skeleton
    on 2023-08-28 21:58

  • +300

    by TherenMoog
    on 2023-08-28 16:23

  • +300

    by A Cute Mimic
    on 2023-08-28 14:08

  • +300

    by Zeer Gales
    on 2023-08-28 05:03

  • +300

    by A Lusty Bananafolk
    on 2023-08-28 00:47

  • +300

    by SirFibbles
    on 2023-08-28 00:27

  • +300

    by honeyultra
    on 2023-08-27 23:16

  • +300

    by Stikkychaos
    on 2023-08-27 19:43

  • +300

    by incedio7
    on 2023-08-27 18:11

  • +1

    by oOSilverwingOo
    on 2023-08-27 14:35

  • +300

    by Voodooking75
    on 2023-08-27 12:38

  • +300

    by Blasphemo
    on 2023-08-27 07:13
    Bot access feels like, if allowed at all, should be opt-in, not opt out.

  • +100

    by Mike_d_20
    on 2023-08-26 15:42

  • +300

    by Rinzler
    on 2023-08-26 04:50

  • +1

    by Thorolf_Skallagrim
    on 2023-08-26 03:39

  • +300

    by Jontaro
    on 2023-08-26 02:45

  • +300

    by bladesmoss
    on 2023-08-26 00:22

  • +1

    by Blossom Tree Art
    on 2023-08-25 22:43

  • +300

    by SyntaxChick
    on 2023-08-25 17:15

  • +100

    by Tobus
    on 2023-08-25 14:11

  • +100

    by Jauvil
    on 2023-08-25 14:08

  • +1

    by Aria Nicest Reynard
    on 2023-08-25 12:35
    If it's easy to implement, then sure - why not.

  • +300

    by aldreaalien
    on 2023-08-25 08:49
    I support a site-wide block.

  • +300

    by LisaGrimm
    on 2023-08-25 08:40

  • +300

    by Nightone
    on 2023-08-25 06:55

  • +300

    by VioletLegacy
    on 2023-08-25 04:14
    I have no desire for anything I create to be used by an AI/

  • +300

    by A Roaring Orc
    on 2023-08-24 20:21

  • +300

    by MaxPlay
    on 2023-08-24 20:10

  • +300

    by Patchwork Veneficium
    on 2023-08-24 19:25

  • +300

    by Plethoramania
    on 2023-08-24 16:32
    I'd rather delete all of my work and my subscription than have some robot touch my work.

  • +300

    by clonetone
    on 2023-08-24 06:57

  • +300

    by A Goodhearted Bananafolk
    on 2023-08-24 02:52

  • +300

    by A Enfeebled Hobbit
    on 2023-08-24 02:29

  • +300

    by devinsxdesigns
    on 2023-08-24 01:14

  • +100

    by Jockey_world
    on 2023-08-23 22:58

  • +300

    by Drasvin
    on 2023-08-23 22:49

  • +300

    by ChrisJWalker
    on 2023-08-23 22:38

  • +300

    by shyredfox
    on 2023-08-23 17:48

  • +300

    by A Rambunctious Skeleton
    on 2023-08-23 16:57
    I would love to see this to protect our community.

  • +300

    by narwaller
    on 2023-08-23 16:26

  • +300

    by Lnphysics
    on 2023-08-23 15:59

  • +300

    by George Sanders
    on 2023-08-23 15:50

  • +300

    by RueMarr
    on 2023-08-23 15:48

  • +100

    by Those2Nerds
    on 2023-08-23 15:18

  • +300

    by A Glamorous Dwarf
    on 2023-08-23 01:22

  • +100

    by A Revolutionary Orc
    on 2023-08-23 00:54

  • +300

    by sabinala
    on 2023-08-22 21:44

  • +300

    by Star Runner
    on 2023-08-22 13:20

  • +300

    by Everwild
    on 2023-08-22 13:00

  • +100

    by Lone Mountain Artificer
    on 2023-08-22 12:37

  • +100

    by Tijmuhn
    on 2023-08-22 07:53

  • +300

    by A Sneaky T-Rex
    on 2023-08-22 06:14

  • +100

    by Demongrey
    on 2023-08-22 01:15

  • +300

    by Bakugoes
    on 2023-08-21 19:18

  • +300

    by WeirdCorvo
    on 2023-08-21 19:03

  • +300

    by Sirjwinsalot
    on 2023-08-21 18:42
    I feel like it's important for people to be reading content not bots. If people are fine with the bots maybe have it as an opt in feature. I think it should be a default opt out with ability to opt in for people that don't mind bots.

  • +300

    by A Mischievous Dragon
    on 2023-08-21 15:19

  • +300

    by A Rambunctious Kobold
    on 2023-08-21 12:15

  • +300

    by erevainsgirl
    on 2023-08-21 03:25

  • +300

    by A Fierce Ooze
    on 2023-08-21 01:08

  • +300

    by franghiskhan
    on 2023-08-21 00:51

  • +300

    by twevan
    on 2023-08-20 22:14

  • +300

    by morganarcher
    on 2023-08-20 20:51

  • +300

    by Mikanis
    on 2023-08-20 19:15

  • +100

    by fantasticat
    on 2023-08-20 18:47

  • +300

    by PrincessESH
    on 2023-08-20 15:50

  • +1

    by cow2face
    on 2023-08-20 14:17

  • +300

    by A Rambunctious Ooze
    on 2023-08-20 02:50

  • +300

    by bonkleshorts
    on 2023-08-20 01:14

  • +300

    by Telyn Ethuil
    on 2023-08-19 20:12

  • +1

    by arsenicalchemist
    on 2023-08-19 16:06

  • +300

    by GrayWarden
    on 2023-08-19 07:39

  • +300

    by 1337spectra
    on 2023-08-19 06:14
    Being someone who has been researching machine learning, AI, quantum AI, etc. for many years now, I don't feel threatened by my work being scraped (because I understand the algorithm of how my data would be used to train a model), but I can definitely get behind giving users that choice over their creative work.

  • +300

    by Grogus The Mad
    on 2023-08-19 06:10

  • +1

    by BosephWasTaken
    on 2023-08-19 02:25

  • +300

    by VeryTired
    on 2023-08-19 01:38

  • +300

    by DarkhorseMB
    on 2023-08-19 00:00

  • +300

    by A Adorable Hobbit
    on 2023-08-18 19:12

  • +300

    by BCGR_Wurth
    on 2023-08-18 18:41

  • +300

    by Goddexx Devine
    on 2023-08-18 16:23

  • +300

    by speagle
    on 2023-08-18 15:35

  • +300

    by Elestre27
    on 2023-08-18 15:16

  • +300

    by MDarkKing
    on 2023-08-18 13:32

  • +300

    by Dalf32
    on 2023-08-18 05:06

  • +300

    by voidchimaera
    on 2023-08-18 03:31

  • +300

    by TinkerTech
    on 2023-08-18 01:30

  • +100

    by olixfarwall
    on 2023-08-17 18:45

  • +300

    by A Beloved Goblin
    on 2023-08-17 15:40

  • +300

    by GoodTato
    on 2023-08-17 10:53

  • +300

    by SierraKomodo
    on 2023-08-17 00:52

  • +300

    by jace0420
    on 2023-08-16 23:54

  • +300

    by Writing Bishop
    on 2023-08-16 22:44

  • +1

    by skorpeon450
    on 2023-08-16 20:20

  • +300

    by ClarissaGosling
    on 2023-08-16 19:43

  • +300

    by AbbieBridgewalk
    on 2023-08-16 19:33

  • +100

    by Milvi
    on 2023-08-16 19:07

  • +300

    by aidanbird
    on 2023-08-16 17:26
    PLEASE. I don't want my content trawled. I do not give consent for any AI/LLM to use my works.

  • +300

    by A Rambunctious Devil
    on 2023-08-16 16:56

  • +100

    by Tailspin
    on 2023-08-16 16:39

  • +300

    by namako
    on 2023-08-16 07:58
    I like using Chat GPT but (as opposed to a speaker before me) in my opinion the data collection for any AI and machine learning should be done in a legal and fair manner, not like this. Have my points, wise stranger.

  • +100

    by connor.phoenix
    on 2023-08-15 23:45

  • +300

    by A Roaring Ooze
    on 2023-08-15 04:42

  • +100

    by MoonlightKall
    on 2023-08-15 03:51

  • -1

    by A Thundering Dwarf
    on 2023-08-15 00:32
    I really don't think this is worth the Developers' time, I'm afraid. I like Chat GPT, and would like to see it improve. I don't see the harm in it using World Anvil for AI training. I kind of feel like, if you don't want anyone to learn from your work (AI or human), then don't make your work publicly available online.

  • +300

    by Sehanbrel
    on 2023-08-14 11:21

  • +300

    by Zanatos
    on 2023-08-14 10:20

  • +100

    by A Lusty Unicorn
    on 2023-08-14 08:02

  • -300

    by A Rambunctious Velociraptor
    on 2023-08-14 07:46

  • +300

    by Kaleidechse
    on 2023-08-14 05:33
    Yes, absolutely! And as others said, worlds should be opted out by default so that those unaware of it are protected.

  • +100

    by JoellaKay
    on 2023-08-14 00:46
    I support a site-wide block for AI scrapping bots, but only if search engines can still index/rank public pages.

  • +300

    by Rains Up Games
    on 2023-08-14 00:31

  • +300

    by PrincessHoneytea
    on 2023-08-13 17:56

  • +300

    by hwcarver
    on 2023-08-13 14:37

  • +300

    by Michael Chandra
    on 2023-08-13 11:24
    I support a site-wide block myself.

  • +300

    by kitoypoy
    on 2023-08-13 11:00
    I'd like this option too!

  • +300

    by A Enfeebled Mimic
    on 2023-08-13 09:15

  • +300

    by Quartz563
    on 2023-08-13 08:47

  • +300

    by RB98
    on 2023-08-13 02:26

  • -300

    by Xhy720
    on 2023-08-13 02:17

  • +300

    by A Lusty Mlem
    on 2023-08-13 00:45

  • +300

    by Lucifer__oO
    on 2023-08-12 21:45

  • +300

    by Jacob-W
    on 2023-08-12 17:37

  • +300

    by A Beloved Mimic
    on 2023-08-12 16:48

  • +300

    by AuthorCallin
    on 2023-08-12 15:34

  • +300

    by tjtrewin
    on 2023-08-12 13:36

  • -100

    by A Rambunctious Goblin
    on 2023-08-12 12:50

  • +300

    by Nanotide
    on 2023-08-12 11:15

  • +300

    by DramaticDragons
    on 2023-08-12 03:05

  • +300

    by Johe
    on 2023-08-12 03:05

  • +300

    by A Cute Mimic
    on 2023-08-12 02:04

  • +300

    by A Roaring Ooze
    on 2023-08-12 01:32

  • +300

    by raptorbricks
    on 2023-08-11 22:37
    Honestly, I'd say to should be enabled site wide, no member of this website benefits from their work being scraped by such tools.   Short of that, a per world option to opt out (enabled by default) feels like the bare minimum that should be done.

  • +300

    by melisiphe
    on 2023-08-11 21:55

  • +300

    by Rahjar
    on 2023-08-11 18:45
    I really think this is a good idea, I wanted to suggest this feature

  • +300

    by Jarhed
    on 2023-08-11 13:10
    This is absolutely necessary in any case where AI is implemented or used, I would even say it should be opt-in rather than opt-out, otherwise that makes it so those that may not even be aware of it may be having their writing scanned by AI without their knowledge/consent.

  • +300

    by Ratha
    on 2023-08-11 11:20

  • +300

    by milanistauk
    on 2023-08-11 11:20

  • +300

    by Nimsy
    on 2023-08-11 08:30

  • +300

    by A Fierce Mimic
    on 2023-08-11 07:50
    I wish I could put all of my points into this. This needs to happen, no matter what. OpenAI is a predatory company. If WA is as serious about protecting creatives as they say they are, this should be top priority.

  • +300

    by notavi
    on 2023-08-11 07:29

Find your way!

Resources

WHO WE ARE

LEGAL

Get the news

Listen to our Podcast

Our Allies

Dungeon Fog
Gold Ennie 2019 for Best Digital Aid
by 3 Armored Kittens · Prodromos v.1.1 - World Anvil 0.9.15.009 © 2017-2024 [en] | Anvil Time: 04:23, May 2 2024