Development of AI technologies and adaptation of society: the example of Im2be

From interesting observations.

Today, the development of AI technologies is much faster than society's adaptation to their perception. And this regularly leads to what we at Im2be, among ourselves, call a “rollback of functionality”, or a “rollback to better times”.

The combination of AI and psychology is a complex and very delicate topic in itself. There were times when the team thought: Here it is, a really cool functionality. On the one hand, when used correctly, it can significantly help in resolving difficult family situations. On the other hand, it can potentially carry risks and unpredictable consequences.

One of such “features” that we had to partially disable “until better times” is Im2be Sentiment Analysis.

In essence, professional psychologists involved in the Im2be project trained/taught the AI model to analyze user behavior in the system: communication style in family chats, the speed and quality of task completion by family members, data from mood diaries, activity patterns, etc.

As a result, this made it possible to: form a kind of Alert for parents if a child showed markers of depressive states, remind partners that they had not spent time together for a long time, and help adapt family conversations to the behavior of a specific family member.

Sounds cool, right?

But when we began systematically to exclude negative cases and align them with the requirements of the European AI Act, it became clear that perhaps it was still a little too early. Technically, everything worked. But from the ethical, responsibility, and societal-readiness perspectives, it was still too early. As a result, most of this functionality had to be disconnected, deliberately postponing it until the moment when society is more adapted to such tools.

At the same time, our FlowCoach - an AI-psychologist and a kind of coach for families, which we, together with family psychologists, are constantly adapting to real-life scenarios, is currently showing itself quite well. More carefully, simpler, with a greater focus on support, not on "diagnosis".

The main rule that the Im2be team adheres to is not about "even more AI", it is about a balance between technology, psychology, and responsibility.

Based on the material https://dou.ua/forums/topic/57096/