Work

Keeping online conversation constructive requires moderation, which takes time. We worked with The New York Times and Alphabet’s Jigsaw to build Moderator, a tool that uses machine learning to significantly increase the efficiency of human moderation.

Conversation starter

Staff at The New York Times have manually moderated every comment posted to nytimes.com for years — yet they could only manage comments for approximately 10% of the site's articles. After implementing Moderator, efficiency gains have allowed journalists to open commenting on nearly every major article.

Under the hood

After closely assessing NYT’s needs, we built a new comment scoring tool using various machine learning models developed by Jigsaw and served through Perspective API. These models were trained on the Times’ previous moderation decisions, allowing Moderator to score and organize comments according to their predicted impact on conversation (e.g. likelihood of being inflammatory, off-topic, insubstantial, etc.). Staff could accept or reject Moderator’s findings, providing feedback that improves the API over time.

 

Adam Gold

Comment

Perspective API
Comment is scored

Moderator
Comment is reviewed

Created with Sketch.
Created with Sketch.

The New York Times
Comment is posted

“Machine learning doesn't replace human moderators. It empowers them to work at super-human levels of speed and accuracy. It's a handy sidekick that lets humans focus on nuance, context and empathy.”
Thomas Reynolds, Technical Director, Instrument
More, better dialog

By freeing the staff at The New York Times to quickly approve and publish comments on more articles, Moderator has removed a major bottleneck — time — allowing rich and constructive online discussion to flow unimpeded. And since the award-winning software is now open-source, it is expected to continue making the Internet a more constructive place for conversation across many other publishing platforms.

Read what The New York Times had to say about Moderator.

Engineering

Technical Strategy
Accessibility
Prototyping
Agile Sprints
Systems Architecture
Front-end Development
Back-end Development
Platform Migration
Integration Support
React

Strategy

UX Strategy
User Testing
Market Research

Design

UI Design
UX Design
Product Design

Next
Google Horizon