Battling bias and other toxicities in natural language generation | InfoWorld - 0 views
-
NLG (natural language generation) may be too powerful for its own good. This technology can generate huge varieties of natural-language textual content in vast quantities at top speed.
-
Today’s most sophisticated NLG algorithms learn the intricacies of human speech by training complex statistical models on huge corpora of human-written texts
-
The algorithm can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. It can also generate a complete essay purely on the basis of a single starting sentence, a few words, or even a prompt
- ...3 more annotations...
-
In addition to human authors who may not be able to keep up with the models’ output, the NLG algorithms themselves may regard as normal many of the more toxic things that they have supposedly “learned” from textual databases, such as racist, sexist, and other discriminatory language.
-
Recent months have seen increased attention to racial, religious, gender, and other biases that are embedded in NLG models such as GPT-3. For example, recent research coauthored by scientists at the University of California, Berkeley; the University of California, Irvine; and the University of Maryland found that GPT-3 placed derogatory words such as “naughty” or “sucked” near female pronouns and inflammatory words such as “terrorism” near “Islam.”
-
Recognizing that algorithmic bias may be a dealbreaker issue for the entire NLG industry, OpenAI has announced that it won’t broadly expand access to GPT-3 until it’s comfortable that the model has adequate safeguards to protect against biased and other toxic outputs.