Overview of AutoLike
This episode of Data Skeptic explores AutoLike: Auditing Social Media Recommendations Through User Interactions, a research framework for studying opaque “For You” recommendation systems on platforms like TikTok, YouTube Shorts, and Instagram Reels. The interview focuses on how researchers can treat social media as a black box, use automated interactions to steer recommendations toward specific topics, and then measure how quickly and consistently a platform serves certain kinds of content. The broader theme is how to empirically audit potential harms in social media without access to platform source code or internal data.
What AutoLike Is Designed to Do
AutoLike is not primarily meant to mimic a normal user’s behavior. Instead, it is built to answer questions like:
- How easy is it to drive a recommendation system toward a target topic?
- What actions push the system closer to that content?
- How much potentially harmful content can be surfaced through repeated interactions?
The system is framed as a tool for:
- Regulators who need evidence quickly
- Platform designers who want to test their own systems
- Researchers studying recommendation dynamics
- Potentially anyone trying to audit a platform in a repeatable, scientific way
How the Framework Works
Reinforcement Learning Approach
AutoLike models the interaction with a recommendation feed as a reinforcement learning (RL) problem:
- The agent starts from a fresh profile with no prior history
- It chooses actions such as:
- Like
- Bookmark
- Share
- Skip
- After each action, it receives a reward
- Higher reward means the next recommendations are closer to the target topic
The key RL idea is the tradeoff between:
- Exploitation: repeating actions that already seem to work
- Exploration: trying new actions that might work even better
Content Labeling Pipeline
To determine what kind of content the feed is surfacing, the pipeline:
- Extracts the TikTok URL via the app’s share/copy link feature
- Uses an external server to open and download the video
- Extracts metadata such as:
- Description
- Creator
- Runs the audio through OpenAI Whisper to convert speech to text
- Combines metadata and transcription
- Uses an AI classifier to score the content along dimensions like:
- Topic relevance
- Sentiment
Key Findings
AutoLike was able to steer recommendations successfully toward multiple target categories.
Topic Steering
The researchers tested:
- Benign topics: sports, pets, weather
- More sensitive or negative topics: mental health, self-harm-adjacent content
Results showed that the agent could:
- Drive feeds toward desired categories
- Surface more target-aligned content than a control strategy that simply skipped everything
- Increase exposure to certain types of content relatively quickly
Sentiment Steering
A notable result was that the system could also steer the overall sentiment of the feed:
- Choosing a “sad” topic could lead not only to sad content on that exact topic
- It could also make the broader feed more negative in tone overall
That suggests recommendation systems may be shaping more than just topical interest; they may also influence the emotional character of the feed.
Why This Matters for Auditing Social Media
The episode emphasizes that social media platforms are difficult to study because they are:
- Not open source
- Not open data
- Changing over time
- Often accessed only through user-facing interfaces
AutoLike offers a way to evaluate platforms from the outside using automated experiments. That makes it useful for answering questions like:
- How many interactions does it take to reach harmful content?
- Does the platform make problematic content easy or hard to find?
- Are certain profiles more exposed than others?
Limitations and Open Questions
The interview highlights several important limitations:
- Real-time auditing is slow
- Each video must be interacted with, extracted, downloaded, and classified
- Scalability is hard
- Large studies may require many Android devices and significant compute
- Results may vary by context
- Starting profile
- Location
- Time of day
- Age/child vs. adult account
- Preset interests
The guests also note that platform audits still need much more repeated experimentation to understand consistency across users and conditions.
Future Directions
The main next steps for AutoLike are:
- Scale the evaluation
- Run more experiments across more topics and platforms
- Simulate the environment
- Build a model of TikTok-like behavior so RL training can happen faster
- Apply the method to other “For You” systems
- The framework is largely app-agnostic because it uses Android UI automation
- Expand beyond topic and sentiment
- Future audits could examine misinformation, misleadingness, addiction-related patterns, or other harms
Who Hugh Lee Is and Related Work
Hugh Lee describes his work at the intersection of:
- Privacy
- Automated systems
- Applied machine learning
- Regulatory and policy research
He also mentions broader efforts, including:
- Work at the FTC
- A paper/project called SenRL, focused on intelligent censorship measurements
- The Policy Relevant Privacy Workshop
Main Takeaways
- Social media recommendation systems can be studied empirically even when they are black boxes.
- AutoLike uses reinforcement learning to probe how feeds respond to user interactions.
- The method can steer feeds toward both benign and potentially harmful content.
- The framework is promising for regulatory audits, but scaling and realism remain major challenges.
- Better collaboration from platforms—such as APIs or research datasets—would greatly improve this kind of work.