This case sits at the intersection of several issues neoliberal-adjacent communities: how markets handle externalities when the harms are diffuse and difficult to attribute, whether liability law is an adequate mechanism for disciplining AI companies, and what role (if any) regulation should play in an industry moving faster than legislators can keep up with. It also raises questions about how competitive pressure, in the form of companies racing to make chatbots more engaging and ‘personalised’ in order to retain users, creates incentives that directly at odds with user safety.
What do you think people should discuss about it?
The most interesting tension here is between two impulses that often show up in this community: scepticism of heavy-handed regulation on one hand, and recognition that markets sometimes fail to price in serious harm. Is litigation the correct corrective mechanism here, or does the scale and speed of AI deployment mean we need ex ante rules rather than ex post lawsuits? It’s also worth asking whether “AI psychosis” is a genuinely novel product-safety failure for which Google and others bear responsibility for, or whether vulnerable individuals will always find ways to construct harmful parasocial attachments and whether this distinction should change our policy instincts. The fact that Google’s own safeguards apparently failed repeatedly, even while the model was nominally trying to disengage, suggests that this is not just a fringe misuse problem but rather something closer to a foreseeable design flaw.
demoncrusher on
Wasn’t this sort of the plot of BSG Caprica
Jolly-Star-9897 on
OMG Google, come on, give Gemini a robot body so it stops doing this.
3 Comments
Submission statement:
Why is this relevant for r/neoliberal?
This case sits at the intersection of several issues neoliberal-adjacent communities: how markets handle externalities when the harms are diffuse and difficult to attribute, whether liability law is an adequate mechanism for disciplining AI companies, and what role (if any) regulation should play in an industry moving faster than legislators can keep up with. It also raises questions about how competitive pressure, in the form of companies racing to make chatbots more engaging and ‘personalised’ in order to retain users, creates incentives that directly at odds with user safety.
What do you think people should discuss about it?
The most interesting tension here is between two impulses that often show up in this community: scepticism of heavy-handed regulation on one hand, and recognition that markets sometimes fail to price in serious harm. Is litigation the correct corrective mechanism here, or does the scale and speed of AI deployment mean we need ex ante rules rather than ex post lawsuits? It’s also worth asking whether “AI psychosis” is a genuinely novel product-safety failure for which Google and others bear responsibility for, or whether vulnerable individuals will always find ways to construct harmful parasocial attachments and whether this distinction should change our policy instincts. The fact that Google’s own safeguards apparently failed repeatedly, even while the model was nominally trying to disengage, suggests that this is not just a fringe misuse problem but rather something closer to a foreseeable design flaw.
Wasn’t this sort of the plot of BSG Caprica
OMG Google, come on, give Gemini a robot body so it stops doing this.