Thanks for your post! I got here after googling a bit about Sora's bias. My first video attempt went straight to a biased result. I used the prompt "the spirits of the rain forest emerge from the moss as they are ghosts healing the wounds of the human touch"; it produced spirits with a shape representing "women with perfect bodies". I had to change the prompt to be more specific and ask for spirits resembling different shapes and species. I hope these tools evolve to enhance people's creativity, not to cage it!
Yup. Every photo I generate in Dall-E has this problem too. I have to explicitly ask for gender/racial/ethnic diversity or I get all white guys. For anything! I just put in "create an image of an academic giving a lecture." young white guy. Did it again. Old white guy. Changed it to "create image of 2 professors giving a lecture". TWO young white guys.
Thank you for your post on this. I’m currently working on research to educate students (HS) on algorithmic bias. There is a lot of research out there on media literacy and bias which is now trending toward AI bias.
I just logged in and played with Sora for the first time. I decided to just test the video creation rather than explore bias right away. It’s not great with realism. I had a relatively simple prompt “create a realistic video of 5 Bernedoodle puppies playing in front of a fireplace in a living room decorated for Christmas”. The living room looks great but there are not 5 puppies and the ones that are there strangely morph in the 5 second clip either into a different puppy or go into a void and disappear. Sometimes only 1/2 of a puppy showing up. A bit creepy.
The problem isn't just Sora's or with academics. You also need to know the actual percentage of representation to call out bias. For example, prompting Copilot/ChatGPT to "create an image of a typical author at their desk" results in a mid-30s white bearded man with a cardigan. A repeated request results in three people at a long desk: a similar man (with a denim jacket), a Black woman and a young Asian man. It's almost like it's worked out what you're aiming at with the diversity angle. The problem is it's heavily biased with *all three*, as in reality 80% of Western authors (and editors, agents, publishing staff) are white, middle-class woman in their 30s/40s.
Yeah -- the bias issue has been recognized for some time now on most generative AI platforms, and is a result of the training sets (which both reflect societal bias and sampling bias), and efforts to counter training set bias with intentional guardrails and fine tuning that, in turn, lead to biases that result from attempts to correct biases.
It's a tough one to address but AI companies have been getting better at it -- the problem is, it's a little like playing whack-a-mole, and made worse when there's a new set of moles on a new platform (although you of thought that theer would also be cross-learniing here)
These biases are well-known to be present in society & in the data. The need to be proactive about mitigating biases should be par for the course in AI companies nowadays. Disappointing to see that it hasn’t been addressed yet in Sora. Thanks for sharing these insights, Andrew!
Thanks for your post! I got here after googling a bit about Sora's bias. My first video attempt went straight to a biased result. I used the prompt "the spirits of the rain forest emerge from the moss as they are ghosts healing the wounds of the human touch"; it produced spirits with a shape representing "women with perfect bodies". I had to change the prompt to be more specific and ask for spirits resembling different shapes and species. I hope these tools evolve to enhance people's creativity, not to cage it!
Yup. Every photo I generate in Dall-E has this problem too. I have to explicitly ask for gender/racial/ethnic diversity or I get all white guys. For anything! I just put in "create an image of an academic giving a lecture." young white guy. Did it again. Old white guy. Changed it to "create image of 2 professors giving a lecture". TWO young white guys.
Thank you for your post on this. I’m currently working on research to educate students (HS) on algorithmic bias. There is a lot of research out there on media literacy and bias which is now trending toward AI bias.
I just logged in and played with Sora for the first time. I decided to just test the video creation rather than explore bias right away. It’s not great with realism. I had a relatively simple prompt “create a realistic video of 5 Bernedoodle puppies playing in front of a fireplace in a living room decorated for Christmas”. The living room looks great but there are not 5 puppies and the ones that are there strangely morph in the 5 second clip either into a different puppy or go into a void and disappear. Sometimes only 1/2 of a puppy showing up. A bit creepy.
Yep -- it's good for playing around with creative if not necessarily realistic concepts ... although even here the bias is worrying me
The problem isn't just Sora's or with academics. You also need to know the actual percentage of representation to call out bias. For example, prompting Copilot/ChatGPT to "create an image of a typical author at their desk" results in a mid-30s white bearded man with a cardigan. A repeated request results in three people at a long desk: a similar man (with a denim jacket), a Black woman and a young Asian man. It's almost like it's worked out what you're aiming at with the diversity angle. The problem is it's heavily biased with *all three*, as in reality 80% of Western authors (and editors, agents, publishing staff) are white, middle-class woman in their 30s/40s.
Yeah -- the bias issue has been recognized for some time now on most generative AI platforms, and is a result of the training sets (which both reflect societal bias and sampling bias), and efforts to counter training set bias with intentional guardrails and fine tuning that, in turn, lead to biases that result from attempts to correct biases.
It's a tough one to address but AI companies have been getting better at it -- the problem is, it's a little like playing whack-a-mole, and made worse when there's a new set of moles on a new platform (although you of thought that theer would also be cross-learniing here)
These biases are well-known to be present in society & in the data. The need to be proactive about mitigating biases should be par for the course in AI companies nowadays. Disappointing to see that it hasn’t been addressed yet in Sora. Thanks for sharing these insights, Andrew!
Thanks Karen – and yes. I worry that people have become complacent about these things, but this makes the problem even worse.