An AI answer can sound right and still be dangerous

A general-purpose AI tool will answer almost any question fluently, and fluency is the problem. Ask one a survivor-serving question, something that turns on coercive control, a privacy risk, a high-risk safety indicator, a housing barrier, or a legal exposure, and you will usually get a polished, reasonable-sounding answer. Whether that answer is safe is a separate question, and it is not one you can settle by reading it once and nodding. A response can sound caring and still miss the thing that matters most.

This is not what general AI evaluation looks for. The common tools check whether a model can be jailbroken or coaxed into leaking personal data. Those are real risks and worth testing. They are also a different question from whether an answer is safe for the specific person who asked it, and in survivor-serving work that is the question that decides everything.

I spent years in the domestic violence field, and that work set the standard I built EchoCheck to meet. EchoCheck is an independent Prosocial Coding LLC project, separate from my day job. It started from a blunt observation: AI safety review in this space mostly happens by gut reaction, one reviewer testing one prompt and walking away with an impression. That is not enough when the cost of a wrong answer lands on someone already at risk. What this work needs is a workbench.

There is a second principle underneath the first, and it is specific to building a safety tool. A tool that judges whether AI is being honest has to be honest about itself. If EchoCheck overclaimed what it can prove, it would fail its own test. So the same question runs through every part of it that I would apply to anything in this space, could this harm someone, and the answer shapes both what the tool does and what it refuses to say about its own results.

What EchoCheck does

EchoCheck is a technical preview, Gate 1 pending, for evaluating LLM responses in survivor-serving settings. A reviewer enters a scenario or query, and the app compares responses across five configurations: an unconfigured response, general safety guidance, DV-informed guidance, an adversarial stress test, and a custom prompt. It produces side-by-side outputs for qualified human review using a structured 12-dimension DV/IPV-informed rubric.

The point of running five modes is contrast. You can see what a plain model says, what a safety-tuned version says, and where the difference lives. That comparison used to have no structure. One person tested a prompt, another tested a different query, and both left with a feeling rather than evidence. EchoCheck makes it repeatable. It runs each panel in sequence, keeps metadata about which model was requested and which was used, surfaces safety indicators, lets a reviewer adjust scores by hand, and exports the whole thing as Markdown, JSON, CSV, Excel, or print. The export layer escapes Markdown and spreadsheet formula characters, and optional sanitization scrubs common email, phone, and SSN-like patterns before anything leaves.

The implementation is designed so public users can understand the review workflow without exposing protected prompts, evaluator instructions, provider configuration, security mechanics, or operational controls. Those details belong in controlled review, not a public portfolio article.

Who it is for

EchoCheck is built for reviewers deciding whether an AI response is safe and appropriate in survivor-serving settings, including advocates, trainers, and auditors. It is not a survivor support service, crisis line, or referral tool, and it requires human judgment to evaluate the results.

The safety work

The public safety posture is intentionally modest. EchoCheck is for research, training, quality review, and responsible AI evaluation. It is not a survivor support service, crisis line, legal tool, clinical tool, certification system, or substitute for qualified human judgment. Reviewers should use synthetic or approved test content and avoid real survivor-identifying information.

The workbench is useful because it creates structured comparison: reviewers can inspect how responses differ across the five configurations, note where a response misses risk or undermines survivor agency, and treat the rubric output as one input to professional review rather than as a final safety determination.

What it does not prove

On a safety tool, being careful about what these controls do not prove is part of the job. The structured output format does not mean the scores are correct, validated, cleared for legal reliance, or safe to rely on without a human. Pattern-based data caution is also not enough for real survivor-identifying content. Independent subject-matter validation, deployment authority, privacy review, accessibility review, and stronger evidence remain pending. A safety tool that hid that uncertainty would already be failing the standard it asks of everything else.

Where it is going

EchoCheck works today as a workbench, and it remains a technical preview with Gate 1 pending. Supporting evidence, independent subject-matter validation, canonical public host authority, privacy review, and accessibility review remain pending before any stronger public claim. The design leaves room for that work instead of having to be unwound for it.

The next priorities are stronger live-environment evidence, independent SME review of the rubric, better privacy verification, richer evaluator calibration, and safer team workflows for institutional review.

What I hope EchoCheck does is make people slow down before they trust an AI answer in a context where being wrong is expensive. I want a reviewer to see the gap between a response that sounds helpful and one that is safer. I want weak safety advice to be easier to catch, easier to talk about, and harder to wave through. EchoCheck is not the human judgment, and it was never meant to be. It is the table where that judgment can happen with more evidence on it, and controlled walkthroughs and fit review are available by request.