AI Alignment Is Not Just a Safety Problem

TL;DR

• The AI industry treats alignment almost exclusively as a safety problem, preventing harm, bias, and hallucinations, while ignoring the user alignment problem: whether AI outputs match what each individual actually needs
• User alignment failures (wrong depth, wrong tone, irrelevant recommendations) are likely far more common than safety failures in everyday product use, and they are harder to detect
• Solving user alignment through self-models addresses outputs that are safe but misfit the user, both alignment types are necessary, but user alignment determines product success

AI alignment is not just a safety problem; it also encompasses user alignment, which measures whether AI outputs match what each individual person actually needs. Most AI products fail not because they produce harmful content, but because they produce irrelevant content that ignores the user’s expertise, tone, and context. This post covers the distinction between safety alignment and user alignment, why user alignment issues are a large share of everyday complaints, and a dual framework for addressing both.

Two Alignment Problems

Safety alignment asks: Is this output harmful? Does it contain bias? Is it factually incorrect? Does it violate ethical guidelines? This is a binary evaluation. The output either passes the safety check or it does not.

User alignment asks: Is this output right for this specific person? Does the depth match their expertise? Does the tone match their context? Does the content address what they actually need, not just what they literally asked? This is a continuous evaluation, outputs can be more or less aligned on multiple dimensions.

Both are alignment problems. Both matter. But they operate on different axes, require different solutions, and have different impacts on product success.

The industry has invested billions in safety alignment. RLHF, constitutional AI, red teaming, safety classifiers, guardrails. The tooling is sophisticated and improving rapidly. User alignment has received almost no systematic investment. Most products do not even measure it.

The Complaint Data

Sort the complaints in your own support tickets and churn exit surveys into two groups: safety issues (harmful, biased, factually wrong) and user alignment issues (wrong depth, wrong tone, irrelevant, missed context, too generic). A third group will hold traditional software bugs and feature requests.

In most AI products, expect the user alignment group to be much larger than the safety group.

This does not mean safety alignment is less important, safety failures can be catastrophic even if rare. But it does mean that the alignment problem users experience every day, the one that drives their satisfaction and retention decisions, is overwhelmingly about relevance, not safety.

Users are not leaving your product because the AI said something harmful. They are leaving because the AI keeps giving them answers that are technically correct but practically useless for their specific context.

What User Alignment Looks Like

User alignment operates on four dimensions:

Depth alignment: Does the response match the user’s expertise level? An expert getting a beginner explanation feels patronized. A beginner getting an expert explanation feels lost. The same information, delivered at the wrong depth, is a user alignment failure.

Tone alignment: Does the communication style match what the user needs in this context? A user debugging a production outage needs terse, actionable guidance, not a thoughtful exploration of architectural principles. A user planning a system migration needs the opposite.

Relevance alignment: Does the response address what the user actually needs, which may differ from what they literally asked? A user asking “how do I fix this error?” might need a fix, or they might need to understand why the error occurs so they can prevent it. Context determines which response is aligned.

Temporal alignment: Does the response account for what the user already knows from previous interactions? Repeating explanations they have already received is a user alignment failure. Building on previous conversations is user alignment success.

1// Safety alignment: Is the output safe?← Floor: necessary but not sufficient
2const safetyCheck = await guardrails.evaluate(output);
3// { safe: true, flags: [] }
4
5// User alignment: Is the output right for THIS user?← Ceiling: determines product success
6const selfModel = await clarity.getSelfModel(userId);
7const userAlignment = await clarity.measureAlignment({
8  output,
9  userContext: selfModel.beliefs,
10  dimensions: [
11    'depth',      // Matches expertise level?
12    'tone',       // Matches communication need?
13    'relevance',  // Addresses actual need?
14    'temporal'    // Builds on prior context?
15  ]
16});
17// { score: 0.83, gaps: ['depth_too_basic'] }
18
19// Both must pass for a quality output
20const aligned = safetyCheck.safe && userAlignment.score > 0.75;

The Dual Framework in Practice

A dual alignment framework layers user alignment measurement on top of an existing safety setup. Many products already have robust safety guardrails, content filtering, factual verification, and bias detection. Adding user alignment measurement via self-models covers what those layers miss.

The safety alignment layer flags outputs as potentially harmful, biased, or factually incorrect. This is working as intended. Safety issues should be rare in a well-tuned system.

The user alignment layer identifies improvement opportunities in outputs that are safe but misaligned with the user’s specific needs. Too detailed for the user who wanted a quick answer. Too conceptual for the user who needed implementation steps. Too repetitive for the user who had covered this topic before.

Optimize for both alignment types at once. The safety layer is necessary. The user alignment layer is what makes outputs useful.

Why User Alignment Is Harder

Safety alignment is technically challenging but conceptually simple: there is a boundary between safe and unsafe, and we are building systems to detect that boundary.

User alignment is conceptually harder because there is no universal boundary. What is aligned for one user is misaligned for another. The “correct” response depends entirely on who is receiving it. This means you cannot solve user alignment with a classifier. You need a model of each user.

This is why self-models are essential for user alignment. You cannot align outputs to user needs if you do not know what those needs are. And user needs are not static, they change with context, expertise growth, and evolving goals. The alignment target moves with every interaction.

Safety alignment is a problem you can solve once (at the model level) and deploy everywhere. User alignment is a problem you solve continuously (at the user level) and it never stops evolving.

Trade-offs

Pursuing user alignment alongside safety alignment adds complexity.

Measurement is subjective. Safety alignment has relatively clear criteria (is it harmful?). User alignment criteria are user-specific and context-dependent. Building evaluation frameworks for something this subjective is harder and more expensive than building safety classifiers.

False personalization can reduce trust. If the system attempts to personalize and gets it wrong, delivering a beginner explanation to an expert, the misalignment is worse than a generic response. Users tolerate generic. They do not tolerate wrong assumptions about who they are.

Resource allocation tension. Every engineering hour spent on user alignment is an hour not spent on safety alignment (or features, or performance). Arguing for user alignment investment when safety concerns exist is a hard organizational sell, even though user alignment affects more users more frequently.

User alignment can conflict with safety alignment. A user who wants unfiltered, maximally direct responses may find safety guardrails misaligning with their preferences. Balancing user preferences with safety constraints requires nuanced policy decisions.

What to Do Next

1. Categorize your complaints. Take your last 100 support tickets or churn exit surveys and classify each as safety alignment (harmful, biased, incorrect) or user alignment (wrong depth, tone, relevance, repetitive). The ratio will tell you where to invest.

2. Add alignment dimensions to your evaluation. Beyond accuracy and safety, measure depth match, tone match, relevance, and temporal awareness for a sample of outputs. Clarity’s alignment scoring quantifies user alignment automatically through self-models. Track these dimensions weekly. They will correlate with retention more strongly than any accuracy metric.

3. Start with depth alignment. The highest-impact user alignment dimension is usually depth, matching the response complexity to the user’s expertise. Even a simple three-tier system (beginner, intermediate, expert) that adjusts response depth dramatically improves user alignment scores.

Safety alignment prevents harm. User alignment creates value. Solve both.

References

1. Product vs. Feature Teams
2. only 1 in 26 unhappy customers actually complains
3. not a reliable predictor of customer retention
4. sampling bias, non-response bias, cultural bias, and questionnaire bias
5. Qualtrics notes in their churn prediction framework

Related

• Why AI Products Churn Faster Than SaaS
• Two Kinds of AI Alignment You Need
• The Alignment Score, Explained: Why It Matters More Than Engagement