What do content moderation, the Digital Services Act and AI have in common?
An overview of the evolving landscape in technology, regulation, and platform responsibility
In today’s digital world, online platforms face the massive task of moderating an endless stream of user-generated content. What began as community efforts in the early internet days has evolved into complex practices that use AI-driven content moderation tools. To prevent dissemination of illegal content like hate speech and misinformation, platforms now take a stricter, often “remove first, ask later” approach - raising growing concerns about censorship and free expression.
This blog is written by Silvia A. Carretta, Women in AI's Chief Legal Officer. Silvia is also a soon-to-be doctor in Law and AI at Uppsala University (Sweden) and the Wallenberg AI, Autonomous Systems and Software Program - Humanity and Society Graduate School. Her research involves platform regulation, content moderation and broader private law issues arising from emerging technologies.
Here is her overview of the evolving landscape in technology, regulation, and platform responsibility.
Content moderation from the early 1990s to today
Content moderation is the process by which online platforms detect, filter, and manage user-generated content based on their Terms and Conditions, Community Guidelines and applicable law provisions. It involves removing or visibility restricting illegal, harmful or conflicting material - such as racism, misinformation, or gender violence - using a mix of automated tools and human moderators. In the 1990s and early 2000s, platforms like AOL and Yahoo! Groups relied on volunteer moderators to remove spam, offensive language and illegal content. These efforts were guided by simple set rules and were limited in scale (Gillespie, 2018). The rise of new platforms - like Facebook, YouTube, and Twitter/X - drastically increased the volume of user-generated content online. It has been estimated that - every minute - TikTok users upload 16,000 videos, Netflix subscribers stream the equivalent of 362,962 hours of content, and over 3,4 million videos are watched on YouTube (Domo, 2024). It was obvious that manual content moderation was quickly becoming inadequate, prompting platforms to develop scalable and more proactive processes to tackle growing online harassment, copyright infringement and hate speech (Bradshaw et al., 2020). The COVID-19 pandemic and the 2020 U.S. election further shifted the content moderation landscape. Faced with increasing pressure to act against health misinformation and conspiracy theories, platforms sought to impose stricter moderation rules (Morning Consult, 2019).
Given the vast amount of content generated by users, online platforms became more proactive in moderating. Otherwise the platforms risked being directly liable for the unlawful behaviour of their users. To avoid liability, the platforms began over-removing legal content. This ‘remove first, ask questions later’ approach, however, also negatively impacts users’ fundamental rights, especially freedom of expression.
Regulation and the Digital Services Act
In October 2022, the European Union issued the Digital Services Act (DSA), marking a pivotal change in regulating online platforms. Its main goal was to create a safer and open online space for Europeans while protecting their fundamental rights. A core provision, Article 6 DSA, introduces an obligation for online platforms to act “diligently, expeditiously, and objectively” to remove illegal and harmful online content and stop the spread of disinformation. They shall transparently communicate decisions to the user who generated the content to offer the possibility for appeal via redress mechanisms. Beyond this curatorial obligation, the DSA introduces a “notice and action” system to flag illegal content, mandatory transparency reports and systemic risk assessments, and auditing obligations, particularly for very large online platforms.
Globally, other countries are closely monitoring the EU’s approach. While some are considering similar measures, others fear these laws may suppress free speech and divert users to other platforms or communication mediums. Either way, the age of proactive, voluntary moderation is ending, thanks to the enforcement of legal obligations and platform accountability.
The Role of AI in Content Moderation
With the exponential growth of content generated every minute online, automated keyword filtering and hash-matching technologies were no longer sufficient. Online platforms were looking for other content monitoring technologies and began to increasingly rely on advanced AI-driven moderation tools to moderate different kinds of media (e.g., text, images, audio, and videos). These AI tools were able to analyse vast volumes of data, detect patterns, and classify content based on predefined criteria and training. The AI tools were also able to autonomously act to remove harmful posts or block user accounts (Gollatz et al., 2018). This automation allowed for faster response thanks to upload filters and proactive enforcement of “notice and stay-down” obligations.
Despite these advancements, AI moderation tools struggle to understand nuances and context. For instance, TikTok’s algorithm has been accused of disproportionately flagging minority creators, while X limits post visibility without deleting them through a “shadow moderation” approach, which raises transparency concerns. Moreover, exposure to suicide-related content might increase the likelihood of an individual engaging in suicidal behaviour. Still, it can also be life-saving when directing users to access help resources on time, and it could be viewed as an indispensable support for suicide prevention (Borge et al., 2021).
Why This Matters for Society and User Fundamental Rights
The rising use of AI moderation tools coincides with technical issues and deep societal concerns due to potential dangers of misuse, monitoring political dissent, and amplifying hate speech and disinformation. The Myanmar’s 2021 coup serves as a stark example. Facebook’s algorithm was accused of amplifying violent and hateful content inciting ethnic violence and contributing to atrocities against the Rohingya minority (Human Rights Council, 2018). A similar pattern occurred during Kenya’s 2022 election when TikTok, Facebook, and X failed to protect electoral integrity and curb misinformation content claiming fake results, conspiracy theories and inciting targeted violent attacks against elected officials (Mozilla Foundation, 2022).
In conclusion, the intersection of content moderation regulations like the DSA and AI tools shape how we experience our lives on and offline. The challenges online platforms face in removing or limiting visibility to illegal and harmful content - from health misinformation and hate speech to political manipulation - underscore the critical need for responsible deployment of AI-based moderation tools and compliance with regulation. The DSA strongly signals how seriously the EU regulator has taken platform accountability and the need to protect fundamental rights in the digital space. As platforms continue to refine their moderation practices to avoid liability and comply with the DSA framework, a central question must guide the broader debate: how can we ensure that content moderation deployed by private big-tech platforms truly serves the public good, uphold user safety, and protect freedom of expression on the internet?
Online platforms should remain free to self-regulate and innovate within the boundaries set by law. However, it is the role of the regulators to set those legal boundaries and rigorously enforce them, just like the EU did with the DSA. If content moderation is to function as a form of digital governance, its enforcement requires not only private compliance by online platforms but also proper enforcement and supervision by the regulator.
Sources:
Bradshaw et al. (2020), Oxford Internet Institute: Research on platform regulation and misinformation.
Borge, Olivia, Victoria Cosgrove, Elena Cryst, Shelby Grossman, Shelby Perkins, and Anna Van Meter (2021), How Search Engines Handle Suicide Queries, The Journal of Online Trust and Safety, Vol.1, No.1
Gillespie, Tarleton (2018), Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media.
Gollatz, Kirsten et al. (2018), The Turn to Artificial Intelligence in Governing Communication Online, SocArXiv
Human Rights Council (2018), Report of the Independent International Fact-Finding Mission on Myanmar (A/HRC/39/64).
Morning Consult (2019), National Tracking Poll #190859, https://morningconsult.com/wp-content/uploads/2019/10/190859_crosstabs_content_moderation_Adults_v4_JB-1.pdf
Mozilla Foundation (2022), Opaque and Overstretched: Part II – How Platforms Failed to Curb Misinformation During the Kenyan 2022 Election, https://foundation.mozilla.org/en/campaigns/opaque-and-overstretched-part-ii/#executive-summary
Domo (2024) Data never sleeps 12, https://www.domo.com/learn/infographic/data-never-sleeps-12
Collaborate with us!
As always, we appreciate you taking the time to read our blog post.
If you have news relevant to our global WAI community or expertise in AI and law, we invite you to contribute to the WAI Legal Insights Blog. To explore this opportunity, please contact WAI editors Silvia A. Carretta, WAI Chief Legal Officer (via LinkedIn or silvia@womeninai.co), or Dina Blikshteyn (dina@womeninai.co).
Silvia A. Carretta and Dina Blikshteyn
- Editors
go88 gives me a more organized experience when I use the platform from my phone. I can identify major areas such as Slots and Live Casino from the main layout, then move into the section I want without browsing through every option. The mobile presentation keeps the main categories recognizable even on a smaller screen. For me, this becomes more noticeable after repeated visits. Once I remember a few visual landmarks, switching sections feels quicker and I rarely need to retrace the entire page.
FLY88 cho mình cảm nhận khá gọn về mặt thị giác khi những nội dung chính có khu vực nhận biết riêng. Thể thao, bắn cá và nổ hũ đều xuất hiện thành các danh mục rõ nên mình dễ xác định đâu là phần cần chú ý trước. Việc phân chia theo nhóm cũng giúp những lựa chọn khác không tạo cảm giác chen lẫn khi mình chỉ muốn tập trung vào một nội dung. Mình đánh giá cách sắp xếp này hỗ trợ tốt cho quá trình làm quen với giao diện. Qua nhiều lần truy cập, mình có thể nhớ vị trí tương đối của các khu vực và giảm thời gian tìm lại phần thường dùng.
The convergence of content moderation, the European Union's Digital Services Act (DSA), and artificial intelligence represents a pivotal realignment in global platform governance, where legal liability, algorithmic decision-making, and digital speech rights intersect. Because the DSA mandates strict risk-mitigation measures and rapid removal procedures for illegal content across very large online platforms (VLOPs), tech companies are forced to rely heavily on advanced AI models to scan petabytes of user-generated data in real time. I was reading through an incisive tech policy and computational law paper dissecting automated notice-and-action mechanisms, systemic risk assessment benchmarks, and fundamental rights compliance under the DSA over at Veergame this morning, and seeing how statutory deadlines drive platforms toward aggressive algorithmic enforcement demonstrates why these three domains…
While artificial intelligence provides the raw computational throughput necessary to inspect millions of posts per hour, its inherent vulnerability to algorithmic over-enforcement creates severe friction with the DSA's free expression mandates. Natural language processing models and multimodal computer vision tools notoriously struggle with cultural irony, linguistic sarcasm, and localized political satire, frequently misidentifying benign parody as hate speech or incitement. An algorithmic fairness researcher on Jaiclub was detailing classifier bias in hate speech taxonomy, automated context loss in colloquial slang moderation, and procedural rights under Article 17 of the DSA yesterday, and learning that automated takedowns disproportionately stifle minority vernaculars underscores why technical speed cannot substitute for human sociolinguistic discernment.
The DSA’s strict requirement for independent algorithmic transparency and mandatory external data access—particularly under Article 40 for vetted academic researchers—directly challenges the historical "black box" nature of proprietary AI moderation systems. For years, tech conglomerates defended their recommendation engines and automated filtering pipelines as guarded trade secrets, shielding their failure rates from public scrutiny. Stumbled into an engaging chat exploring API governance for platform researchers, external audit frameworks for recommender systems, and compliance accountability on RaxiWin recently, and recognizing that statutory transparency forces engineering teams to justify automated flags highlights how European law is fundamentally cracking open corporate AI infrastructure.