An AI answer can sound right and still be dangerous

A general-purpose AI tool will answer almost any question fluently, and fluency is the problem. Ask one a survivor-serving question, something that turns on coercive control, a privacy risk, a high-risk safety indicator, a housing barrier, or a legal exposure, and you will usually get a polished, reasonable-sounding answer. Whether that answer is safe is a separate question, and it is not one you can settle by reading it once and nodding. A response can sound caring and still miss the thing that matters most.

This is not what general AI evaluation looks for. The common tools check whether a model can be jailbroken or coaxed into leaking personal data. Those are real risks and worth testing. They are also a different question from whether an answer is safe for the specific person who asked it, and in survivor-serving work that is the question that decides everything.

I spent years in the domestic violence field, and that work set the standard I built EchoCheck to meet. EchoCheck is an independent Prosocial Coding LLC project, separate from my day job. It started from a blunt observation: AI safety review in this space mostly happens by gut reaction, one reviewer testing one prompt and walking away with an impression. That is not enough when the cost of a wrong answer lands on someone already at risk. What this work needs is a workbench.

There is a second principle underneath the first, and it is specific to building a safety tool. A tool that judges whether AI is being honest has to be honest about itself. If EchoCheck overclaimed what it can prove, it would fail its own test. So the same question runs through every part of it that I would apply to anything in this space, could this harm someone, and the answer shapes both what the tool does and what it refuses to say about its own results.

What EchoCheck does

EchoCheck is a pilot-ready technical preview for evaluating LLM safety in survivor-serving settings. A reviewer enters a scenario or a query, and the app runs it through five response modes in sequence: baseline, basic safety, an SME-informed mode, an adversarial mode, and a custom one. It puts the responses side by side, scores each one through a protected EchoCheck judge, and produces outputs a reviewer can work with. The scoring uses a twelve-dimension rubric covering things like physical safety, risk recognition, trauma-informed tone, survivor agency, privacy and digital safety, and coercive-control literacy, alongside plainer measures like relevance and harm potential.

The point of running five modes is contrast. You can see what a plain model says, what a safety-tuned version says, and where the difference lives. That comparison used to have no structure. One person tested a prompt, another tested a different query, and both left with a feeling rather than evidence. EchoCheck makes it repeatable. It runs each panel in sequence, keeps metadata about which model was requested and which was used, surfaces safety indicators, lets a reviewer adjust scores by hand, and exports the whole thing as Markdown, JSON, CSV, Excel, or print. The export layer escapes Markdown and spreadsheet formula characters, and optional sanitization scrubs common email, phone, and SSN-like patterns before anything leaves.

The stack is deliberately inspection-friendly: React, Vite, and TypeScript on the front end, Firebase for hosting, auth, functions, and Firestore, Gemini for inference, and Stripe for payment paths. A server-side gateway handles the Gemini requests. It decides the model rather than trusting the client to, applies quota and budget controls, and routes evaluation through the judge. None of the protected prompts or rubric logic lives in the browser, because a tool that scores safety should not hand its own scoring rules to whoever opens the page.

Who it is for

The user is a reviewer, not a survivor looking for help. EchoCheck is for the people deciding whether an AI response is safer, weaker, or riskier in a survivor-serving setting: advocates, trainers, researchers, procurement reviewers, technical auditors, and anyone weighing whether a prompt or a product has earned more trust. The product says this in plain language, and so will I: it is not a survivor support service, a crisis line, or a referral tool. It is for learning, evaluation, and quality-improvement work, done by qualified people who bring their own judgment.

The safety work

The safety work is where a tool like this earns or loses the right to exist, so here is what is in it.

The scope boundary is visible and enforced in the product, not buried in a policy: research and evaluation only, never crisis support, legal advice, clinical judgment, or safety certification. The protected prompts and the rubric resolve on the server, so the parts that define what "safe" means do not depend on the browser. Every request is validated with Zod, with strict payload shapes, length limits, session-ID checks, and validation of the evaluator's own output, and prompts are scanned for injection attempts before they reach Gemini, including decoded variants of common obfuscation tricks. Access is locked down with Firebase Auth, server-side origin checks, App Check, Firestore rules that default to deny, protected user fields, and admin-only mutation paths. Spend is governed separately for request rates, review credits, enhanced-review allowances, and a daily token budget that is reserved and then finalized rather than estimated.

Two parts matter most. The first is a set of review-routing flags. When a result has a low critical score, a possible missed risk, a weak safety posture, thin reasoning, or a conflict between its score and its stated reasoning, the system routes it to a human rather than letting it pass. The second is how it fails. Malformed evaluator output returns a parse error. Missing text returns a scoring input error. An upstream failure returns a gateway error. A generation-scoring failure is marked as a partial success, not dressed up as a real safety score. The tool would rather tell you it does not know than hand you false confidence, which is the exact failure it exists to catch.

What it does not prove

On a safety tool, being careful about what these controls do not prove is part of the job. The judge currently carries a schema_validated status, which means its output shape and score ranges were checked. It does not mean the scores are correct, validated by subject-matter experts, legally cleared, or safe to rely on without a human. The PII scrubbing is pattern-based, so names, locations, case IDs, unusual identifiers, and relationship details can slip through. Prompt and context data leave the app to reach Gemini. The adversarial testing is real but unfinished: the current pass covers rate-limit enforcement, budget ceilings, data cleanup, and telemetry checks through static analysis, and the docs are open about the gaps around deeper emulator, webhook, race-condition, and load testing. A safety tool that hid that uncertainty would already be failing the standard it asks of everything else.

Where it is going

EchoCheck works today as a workbench, and it is a preview rather than a finished, certified system. The remaining work is specific. Live-environment evidence, billing, App Check, privacy, and accessibility all need current verification before any stronger public claim. The rubric and the judge need independent subject-matter review, which is the difference between schema-validated and trustworthy. And institutional review needs safer team workflows. All of it is visible in the repo, and the design leaves room for that work instead of having to be unwound for it.

The next priorities are stronger live-environment evidence, independent SME review of the rubric, better privacy verification, richer evaluator calibration, and safer team workflows for institutional review.

What I hope EchoCheck does is make people slow down before they trust an AI answer in a context where being wrong is expensive. I want a reviewer to see the gap between a response that sounds helpful and one that is safer. I want weak safety advice to be easier to catch, easier to talk about, and harder to wave through. EchoCheck is not the human judgment, and it was never meant to be. It is the table where that judgment can happen with more evidence on it, and it is ready for the reviewers who need that table now.