« Dedupe in action

Project
Testimony of Beth Simone Noveck: Fake It till They Make It: How Bad Actors Use Astroturfing to Manipulate Regulators, Disenfranchise Consumers, and Subvert the Rulemaking Process
By
U.S. House Committee on Financial Services
Link
https://financialservices.house.gov/uploadedfiles/hhrg-116-ba09-wstate-noveckb-20200206.pdf
Published
February 2020
Tool used
Dedupe.io
Testimony of Beth Simone Noveck: Fake It till They Make It: How Bad Actors Use Astroturfing to Manipulate Regulators, Disenfranchise Consumers, and Subvert the Rulemaking Process

In this submission, which reflects only my personal opinions, I set out the crucial importance of citizen and stakeholder engagement to increase the effectiveness and legitimacy of regulations, and to strengthen democracy and trust in policymakers when both are under severe challenge. I examine some difficultiesattached to public commenting in rulemaking, and how they can be overcome using new tools and technology. Finally, I showcase how jurisdictions around the world are turning to crowdlaw, the use of online public engagement in order to improve the qualityof the law-and rulemaking process, and provide examples that the United States could draw on as it seeks to deepen the foundations of its democracy in uncertain times.

Solution 2: Use Deduplication Software to Address Identical Comments

The large volume of submissions often results from duplicative “postcard” comments. Datamining technology to deduplicate public comments has existed for well over a decade. Deduplicating records in a database is a common process. Dedupe.iois an example of a software service which helps researchers and data scientists to identify and delete similar records in their database.