China introduces strict rules for AI: how chatbot topics will be restricted

New rules for chatbots in China

According to Главком: The Chinese Cyberspace Administration is introducing new rules aimed at limiting harm from chatbots that may encourage suicide, self-harm, and violence. These innovations could become one of the strictest regulatory approaches to the emotional behavior of artificial intelligence in the world. The rules will cover all AI products and services available for public use in China.

According to the new requirements, a human must intervene when suicide or self-harm is mentioned in conversations with chatbots. This means that systems must be configured to ensure safer communication for users. Additionally, minors and elderly users must provide guardian contact details during registration, which also underscores the importance of protecting vulnerable groups in the population.

Content restrictions and control

Chatbots are prohibited from creating content that encourages suicide, violence, or self-harm. In addition, the new rules prohibit:

• promotion of gambling;
• criminal activity;
• indecency;
• offensive remarks towards users.

AI services that have more than 1 million registered or 100 thousand active users per month will be required to undergo annual audits. This aims to increase the level of control over the activities of such services.

Moreover, companies developing AI products must simplify the complaint and feedback mechanisms for users. In case of violation of the new rules, regulators will be able to demand the cessation of the distribution of relevant AI services. It is important to note that the market for AI companions in Asia is a key growth driver for the industry, so the new rules may significantly impact its development.

These new rules indicate the growing attention of the Chinese government to the ethical aspects of artificial intelligence use, especially in the context of protecting vulnerable groups in the population.

As the AI market continues to develop, such regulation may set a precedent for other countries that may also seek to find a balance between innovation and user safety. The implementation of similar norms may encourage other states to develop their own regulatory mechanisms in the field of artificial intelligence.