ICO warns tech industry not to ignore data protection risks when using GenAI

The Information Commissioner’s Office (ICO) has concluded an investigation into Snap’s launch of ‘My AI’ chatbot.

In February 2023 Snap launched ‘My AI’ for its premium Snapchat+ members. Two months later, the multimedia instant messaging app and service made the chatbot available to all Snapchat users.

ICO however was worried that ‘My AI’ posed data protection risks for its subscribers. Therefore the Data Protection Authority opened an investigation into ‘My AI’, following concerns that Snap had not met its legal obligation to adequately assess the data protection risks posed by the new chatbot.

The investigation resulted in a Preliminary Enforcement Notice to Snap on October 2023. According to ICO, Snap didn’t carry out a thorough review of the risks posed by the generative artificial intelligence (GenAI) chatbot.

Since then Snap has taken measures to look more in-depth into the matter. The company also demonstrated it had implemented appropriate mitigations. “We are satisfied that Snap has now undertaken a risk assessment relating to ‘My AI’ that is compliant with data protection law”, ICO says in a statement. The regulator promises to monitor the rollout of ‘My AI’ and how emerging data and privacy risks are addressed.

Almond: ‘ICO investigation should act as a warning shot’

Stephen Almond, ICO Executive Director of Regulatory Risk, hopes that ICO’s investigation into ‘My AI’ acts like a warning shot for the tech industry. “Organisations developing or using generative AI must consider data protection from the outset, including rigorously assessing and mitigating risks to people’s rights and freedoms before bringing products to market”, he says.

ICO will continue to monitor organisations’ risk assessments. Almond ensures that the British regulator will use the full range of its enforcement powers, including fines, to safeguard the public from harm.

ICO’s final decision in this case will be published in the coming weeks.