Specification gaming is great fun. When humans do it, it is sometimes called malicious compliance. The poster on X only asked for "a pope", not the pope, nor an image of anyone who ever was pope. After all, an Indian woman could become pope someday!
I just saw your response. My point was that lots and lots of testing is done by various people over times during system and safely testing, and NOT that I have evidence of specific prompts. So it's extremely unlikely, given the amount and variation of testing, that people did not realize that so many prompts - for example - would depict Vikings as Indian and British kings as Black.
I thought the "black Nazi" issue was blown out of proportion by the media and other people. Who really thinks that kind of thing is dangerous? Who expected that a model that is trying to be diverse and not perpetuate bias, and given finite amounts of data, might not do exactly that as a bug in its beta or early stages? (I think people who don't understand the technology well). Why be offended by these things and play the "anti-woke" agenda? I found the media's reactions a bit ridiculous, and Google's leadership and subsequent freak-out even worse. They should have had "more balls" and handled it with grace and honor and said that these kinds of mistakes do not matter in these nascent systems and that people should worry about really harmful content. It's not like drawing black or brown Nazis is going to affect the way we teach history or convince anyone that Nazis had brown and black skin, for example (if a person is convinced by that, they're going to be convinced by a ton of other much more harmful or stupid stuff). To me, it actually read in the opposite direction of "what the hell, being a Nazi is a horrible and disgusting thing, and who cares about their skin color, why would you even want to draw them, and if they are drawn black or brown, it actually disempowers real Nazis". The point of an image generator is not to create factual content (at least at this point). I may be missing the point of the drama, but I thought to myself "this was bound to happen, and at this point in the 'AI race' it's not such a big deal"). The fact that Google didn't really have a strategic team that could have managed to take advantage of the situation and attack the critics was really a sign of weak leadership in my opinion (I may sound a bit Machiavellian, but just for the entertainment of what they could have done instead of letting the issue deter their product development and negatively affect them, they could have come out with sophisticated arguments about why this was OBVIOUSLY something that was going to happen and belittle their critics). As for the "anti-whiteness," they could have argued that it was likely the result of their attempts to prevent white supremacist radicalization and attacked Elon Musk and other critics in the process. I think Google culture is not the kind of culture that defends itself and has a lot of PR and HR and public opinion boundaries and considerations. They should have attacked the critics and turned the situation in their favor. On the other hand, we know that Google is owned by white people and the brown Indians and Chinese and other developers are mostly just workers, so maybe my point of view is a bit naive (also considering that I am a Latin American man with brown skin, I might be missing the point, I guess if I were white and had problems asking for pictures of people with my own skin color I would feel differently? not sure though, it feels a bit stupid to be offended by perceived "anti-whiteness" unless you come from the camp that thinks "white privilege" is not real). Anyway, sorry for the long tangent, I know that the point of your article was precisely not to indulge in culture wars, but to point out what the real dangers are, and how people have missed the point by getting caught up in culture wars. My comment is just to point out my perspective on the issue.
I generally agree with the article. But I'm quite surprised that the author said things like "it’s impossible to safety-test all the prompts that users come up with once the model is released" [the "all" is irrelevant and misleading - "anti-whiteness" would show up for lots of questions] and "I don’t think they expected it to depict Vikings as Indian and British kings as Black." Well, of course they knew images would reflect "anti-whiteness" because such systems go through loads of system-testing and safety-testing - many Google staff entered many questions such as "show me pictures of the first US president", or Vikings or British kings.
The failure of Google Geminin makes me wonder if anyone on the team ever read a book called "The Robots of Gotham". It is a long book, but a very eye opening potential future of AGIs that heavily utilize specification gaming to carry out their machinations around the world.
All this neutrality thing is based on false assumptions, no wonder it leads to such a disasters. It’s a well-know result from formal logic, if axiomatic system has a false statement in it - any theorem can be proven true. So Asian-German soldiers during WW2 is just an illustration.
Specification gaming is great fun. When humans do it, it is sometimes called malicious compliance. The poster on X only asked for "a pope", not the pope, nor an image of anyone who ever was pope. After all, an Indian woman could become pope someday!
I just saw your response. My point was that lots and lots of testing is done by various people over times during system and safely testing, and NOT that I have evidence of specific prompts. So it's extremely unlikely, given the amount and variation of testing, that people did not realize that so many prompts - for example - would depict Vikings as Indian and British kings as Black.
I thought the "black Nazi" issue was blown out of proportion by the media and other people. Who really thinks that kind of thing is dangerous? Who expected that a model that is trying to be diverse and not perpetuate bias, and given finite amounts of data, might not do exactly that as a bug in its beta or early stages? (I think people who don't understand the technology well). Why be offended by these things and play the "anti-woke" agenda? I found the media's reactions a bit ridiculous, and Google's leadership and subsequent freak-out even worse. They should have had "more balls" and handled it with grace and honor and said that these kinds of mistakes do not matter in these nascent systems and that people should worry about really harmful content. It's not like drawing black or brown Nazis is going to affect the way we teach history or convince anyone that Nazis had brown and black skin, for example (if a person is convinced by that, they're going to be convinced by a ton of other much more harmful or stupid stuff). To me, it actually read in the opposite direction of "what the hell, being a Nazi is a horrible and disgusting thing, and who cares about their skin color, why would you even want to draw them, and if they are drawn black or brown, it actually disempowers real Nazis". The point of an image generator is not to create factual content (at least at this point). I may be missing the point of the drama, but I thought to myself "this was bound to happen, and at this point in the 'AI race' it's not such a big deal"). The fact that Google didn't really have a strategic team that could have managed to take advantage of the situation and attack the critics was really a sign of weak leadership in my opinion (I may sound a bit Machiavellian, but just for the entertainment of what they could have done instead of letting the issue deter their product development and negatively affect them, they could have come out with sophisticated arguments about why this was OBVIOUSLY something that was going to happen and belittle their critics). As for the "anti-whiteness," they could have argued that it was likely the result of their attempts to prevent white supremacist radicalization and attacked Elon Musk and other critics in the process. I think Google culture is not the kind of culture that defends itself and has a lot of PR and HR and public opinion boundaries and considerations. They should have attacked the critics and turned the situation in their favor. On the other hand, we know that Google is owned by white people and the brown Indians and Chinese and other developers are mostly just workers, so maybe my point of view is a bit naive (also considering that I am a Latin American man with brown skin, I might be missing the point, I guess if I were white and had problems asking for pictures of people with my own skin color I would feel differently? not sure though, it feels a bit stupid to be offended by perceived "anti-whiteness" unless you come from the camp that thinks "white privilege" is not real). Anyway, sorry for the long tangent, I know that the point of your article was precisely not to indulge in culture wars, but to point out what the real dangers are, and how people have missed the point by getting caught up in culture wars. My comment is just to point out my perspective on the issue.
I generally agree with the article. But I'm quite surprised that the author said things like "it’s impossible to safety-test all the prompts that users come up with once the model is released" [the "all" is irrelevant and misleading - "anti-whiteness" would show up for lots of questions] and "I don’t think they expected it to depict Vikings as Indian and British kings as Black." Well, of course they knew images would reflect "anti-whiteness" because such systems go through loads of system-testing and safety-testing - many Google staff entered many questions such as "show me pictures of the first US president", or Vikings or British kings.
The failure of Google Geminin makes me wonder if anyone on the team ever read a book called "The Robots of Gotham". It is a long book, but a very eye opening potential future of AGIs that heavily utilize specification gaming to carry out their machinations around the world.
BTW, goody-2 at least detects the diagonal argument/barber paradox, though it refused to answer (of course)
All this neutrality thing is based on false assumptions, no wonder it leads to such a disasters. It’s a well-know result from formal logic, if axiomatic system has a false statement in it - any theorem can be proven true. So Asian-German soldiers during WW2 is just an illustration.
> I don’t think they expected it to depict Vikings as Indian and British kings as Black
If they didn’t test these simple tests then they are not fit for purpose. Do they not have testers and prompt engineers (of course they do)