Oh sure there is definitely a risk of that. China introduces rules to rein in AI companion bots amid emotional dependency concerns - Chinadaily.com.cn.

the new rules explicitly prohibit AI systems from encouraging addictive use, requiring mandatory reminders after more than two hours of continuous use, pop-up interventions when signs of dependency are detected, and intervention mechanisms in cases of extreme emotional distress

I do imagine such regulation will prevent the worst outcomes, making developers responsible. I’m not completely opposed to such “AI relationships” like for example in games. Portal is one of my favorite games because you had this companion who constantly comments. Video games could change dramatically and enable completely new genres.

The most dangers I see in AR VR porn. Like once generative AI becomes able to generate and render photorealistic characters in 3D on a GPU in realtime, a kind of hybrid rendering approach. With virtual reality and cyberdildonics I can imagine many men being perfectly satisfied with such incredibly immersive porn, tailored perfectly for you, interactive etc. This could replace real world relationships. Or maybe save them, I don’t know lol. I would much prefer open source application for such VR AI porn that do not have a profit motive. The upside is that it generates porn in an ethical way and could drastically reduce demand for sex work, which I see as a moral good.