Summary of Why intent prediction needs more than an LLM

Overview of Why intent prediction needs more than an LLM

In this episode of the Stack Overflow Podcast, host Ryan Donovan speaks with Frank Portman, CTO at Yobi AI, about why large language models (LLMs) are powerful but insufficient for certain prediction problems—especially intent prediction, recommendation, and decision-making under uncertainty. The conversation explores how Yobi builds a “behavioral AI” foundation model using proprietary, privacy-sensitive behavioral data, combining transformers, graph models, and specialized training methods to forecast future behavior for use cases like ad targeting, personalization, fraud, and marketing.

Key Themes and Takeaways

• LLMs are excellent at language tasks, but not necessarily at decision-making

  • Frank argues that the inductive bias of next-token prediction is great for synthesis, coding, and conversation, but not obviously suited to forecasting intent or choosing the best action under uncertainty.
  • He sees LLMs as strong tools, but not a complete replacement for purpose-built behavioral prediction systems.

• Behavior prediction is a different modeling problem

  • Yobi describes itself as a behavioral AI company, not simply an ad-tech company.
  • Their goal is to build a foundation model for future behavior, not for text, image, or video generation.

• Proprietary behavioral data matters

  • Unlike public language data, Yobi works with sensitive, identifiable behavioral signals from anonymous user sessions, websites, and product interactions.
  • This data is central to building useful predictions while respecting privacy constraints.

• Scale and architecture still matter

  • Yobi uses large-scale transformers plus graph neural networks to model identity and relationships between anonymous identifiers.
  • The team emphasizes inductive models where possible, because user behavior changes frequently and systems need to generalize to new nodes and entities.

Why LLMs Alone Fall Short for Intent Prediction

Language is not the same as behavior

Frank’s core argument is that language models are trained to predict tokens, not to optimize for expected value or make business decisions. While they can mimic reasoning and generate plausible outputs, that does not mean they are inherently good at:

• predicting user intent,
• ranking actions,
• selecting the best ad,
• or deciding which recommendation will perform best.

Context matters more than generic fluency

A chatbot can discuss a publisher or a brand intelligently, but personalization systems need a much more specific kind of context: historical behavioral signals, identity relationships, and real-time signals that drive outcomes like clicks or conversions.

LLMs can help, but usually as part of a system

Frank is not anti-LLM. He argues that LLMs can fit into broader agentic or decision-making systems, especially when paired with the right tools and specialized models. But he does not believe “one giant LLM” is the right answer for everything.

Yobi’s Technical Approach

Foundation model for behavior

Yobi trains a base model on behavioral data and then fine-tunes it for specific outcomes, such as:

• ad campaigns,
• personalization decisions,
• recommendation tasks,
• and potentially fraud/risk use cases.

This lets them amortize the cost of pretraining across many downstream tasks.

Transformers plus graph models

Their stack includes:

• Transformers for large-scale sequence modeling
• Graph models / GNNs for identity and relationship structure
• Specialized embeddings and lookup tables for high-cardinality, discrete entities

Inductive vs. transductive modeling

Frank highlights an important distinction:

• Inductive models can generalize to new users, behaviors, or nodes.
• Transductive models rely more heavily on the exact entities seen during training.

Because user behavior evolves constantly, Yobi puts a lot of effort into models that can adapt to new behavior patterns without retraining everything from scratch.

Agentic Systems and the “Decision Layer”

Frank sees a future where behavioral models can serve as a decision or forecasting layer inside agentic systems.

Where agents are already useful

He identifies two economically valuable agent categories today:

• Coding assistants
• Personal assistant / “LLM in your pocket” tools

What “decision” means here

For Yobi, a decision is not just choosing words in a chat. It can mean:

• whether to show an ad,
• which campaign to select,
• which creative to serve,
• or whether to recommend something at all.

In other words, the system is deciding among actions with measurable expected value.

Privacy and Trust as Core Requirements

Because Yobi works with sensitive behavioral data, privacy is a major part of the product and technical strategy.

Privacy-preserving machine learning

Frank discusses interest in:

• Differential privacy
• K-anonymity concepts
• Homomorphic machine learning

The goal is to build models that remain useful without exposing user-level data unnecessarily.

Security is not enough

He emphasizes that companies handling consumer behavior data need both:

• strong security controls
• and strong algorithmic privacy guarantees

This is central to earning customer trust.

Performance and Scaling Considerations

At the scale Yobi operates—millions of queries per second—latency and cost matter.

Main optimization strategies

• Precompute as much as possible
• Use embedding lookup tables instead of generating everything on the fly
• Apply batching and queuing to increase throughput
• Use fallback heuristics where needed, but aim to solve the actual end problem

Frank notes that the team is still prioritizing correctness and product value over pure margin optimization, as long as the architecture remains viable.

Broader Applications Beyond Ads

Although ads are the first major use case, Frank sees the approach extending to:

• Marketing technology (MarTech)
• Personalized product recommendations
• Fraud and risk
• Other prediction-heavy personalization decisions

Ads are a logical starting point because they are economically meaningful and structurally similar to many other decision problems.

Final Thoughts

The episode argues that while LLMs are transformative, they are not a universal solution for prediction or decision-making. Frank Portman’s core message is that behavior prediction requires different data, different inductive biases, and often different model architectures. For domains like ads, personalization, and risk, the most valuable system may be a specialized behavioral foundation model—not a general-purpose chatbot.

Notable Insight

“Can we just solve the right problem?”

That idea captures the episode’s practical takeaway: instead of forcing every problem into an LLM-shaped solution, teams should choose the model and training strategy that best matches the real business objective.