AI high-risk evaluation

<p>This document summarizes HiringCoachAI&#39;s posture on the high-risk AI evaluation questions assessed during higher-education vendor reviews.</p> <h2>Classification of our AI use</h2> <p>We classify HiringCoachAI&#39;s AI use as moderate risk:</p> <ul><li>No fully automated decisions with legal or similarly significant effects (Art. 22 GDPR).</li><li>No public AI-generated content about identifiable users is published under HiringCoachAI&#39;s name. AI-assisted aggregate insights and AI-assisted product marketing content may be published; neither identifies an individual user or uses personal data.</li><li>No AI-driven hiring decisions: users generate content for themselves, which they then use at their own discretion.</li><li>Human review is output-type specific. Resume, cover-letter, pitch, and similar drafting outputs are intended for user review and editing before use. Other outputs, such as interview-practice scoring, transcription, task breakdowns, and value-proposition suggestions, are informational aids and should not be treated as automated decisions.</li></ul> <p>The following answers map onto HECVAT 4.1.5 High-Risk items.</p> <h2>Data handling</h2> <ul><li><strong>Input data categories:</strong> Resume text, job description, user-authored questions, audio (for transcription), career history, company context.</li><li><strong>Sensitive categories:</strong> None intended. We prohibit PHI, government IDs, payment card data, and child-directed data.</li><li><strong>Residency:</strong> US default. AI providers are primarily US-based.</li><li><strong>Retention at provider:</strong> Per-request controls minimize storage or transcript exposure where supported (<code>store: false</code> on OpenAI, <code>redact=true</code> on Deepgram). No Zero Data Retention amendments are in place with any AI provider; each provider&#39;s then-current standard API retention windows apply. See the <a href="/trust/docs/ai-model-inventory">AI model inventory</a>; DPA register evidence is available on request via <code>[email protected]</code>.</li><li><strong>Training use:</strong> We do not train models on customer data. We rely on provider standard API terms and per-request retention-minimization flags where available; no separate enterprise no-training amendment has been signed.</li><li><strong>Internal retention:</strong> <code>aiCallAudit</code> metadata only, no prompts or completions, for 1 year.</li></ul> <h2>Model lifecycle</h2> <ul><li><strong>Model selection:</strong> Driven by feature needs; reviewed at feature kickoff per the <a href="/trust/docs/sdlc">secure development lifecycle</a>. Primary model strings are documented in the <a href="/trust/docs/ai-model-inventory">AI model inventory</a>.</li><li><strong>Fine-tuning:</strong> None today. If ever adopted, fine-tuning data would be HiringCoachAI-authored, not customer data.</li><li><strong>Versioning:</strong> Model strings are managed through a central registry.</li><li><strong>Evaluation:</strong> A formal bias-evaluation methodology is documented at the <a href="/trust/docs/ai-bias-evaluation">AI bias evaluation page</a>. A baseline run completed on 2026-05-07: positive controls passed and no demographic or name-derived references were detected. The baseline did surface several output-type findings (length or tone variance, strict-format JSON failures, and one candidate-name leak in a fit-score output). A remediation rerun on 2026-05-14 cleared all thresholds across the expanded suite.</li></ul> <h2>Safety controls</h2> <ul><li><strong>Prompt injection:</strong> Regex-based prompt-injection and jailbreak heuristics are applied to AI requests where the safety-check option is enabled.</li><li><strong>Output handling:</strong> Current controls are scoped AI features, user review before reliance, reporting/escalation, and provider/request controls where available. These are the output controls represented for the current service.</li><li><strong>Rate limiting:</strong> Per-user rate limiting is applied to selected high-risk endpoints; remaining coverage is reviewed through the API-validation and security workflows.</li></ul> <p>AI use is disclosed through the first-visit banner, privacy policy, and AI Disclosure page rather than a label on every generated output.</p> <h2>Transparency to users</h2> <ul><li>First-visit disclosure banner names AI use and links to the AI Disclosure page.</li><li>Privacy Policy discloses AI processing, providers, retention, and lawful basis.</li><li>AI Disclosure page at <code>/ai-disclosure</code> enumerates AI features, data sent, and triggers.</li><li>No blocking consent modal. AI use is processed under contract performance when the user invokes or configures a feature that requires the call.</li><li>No in-product AI opt-out toggle, by design. Users avoid AI processing by not using AI-assisted features; account deletion is available.</li></ul> <h2>Accountability</h2> <ul><li><strong>Owner:</strong> Security Officer and Privacy Officer / data-protection contact.</li><li><strong>Incident route:</strong> Same as any other Sev 1 or Sev 2 incident; see the <a href="/trust/docs/incident-response">incident response policy</a>.</li><li><strong>Audit log:</strong> AI call audit and the append-only audit log.</li><li><strong>Bias eval:</strong> methodology documented at the <a href="/trust/docs/ai-bias-evaluation">AI bias evaluation page</a>; a baseline run was completed on 2026-05-07 with follow-up items, and a remediation rerun completed on 2026-05-14 with no thresholds exceeded.</li></ul> <h2>Rights and recourse</h2> <ul><li>Users can avoid AI processing by not using AI-assisted features. Manual workflows do not invoke AI.</li><li>Users can export their data via <code>/account/export</code>.</li><li>Users can delete their account via <code>/account/delete</code>.</li><li>Users can complain to a supervisory authority; see the <a href="/trust/docs/privacy">privacy policy</a>.</li></ul> <h2>Ethical considerations</h2> <ul><li>We do not use AI to make automated decisions about users.</li><li>We do not use AI to profile users for marketing.</li><li>We do not claim AI outputs are authoritative without user verification; generated drafts, scores, transcripts, and suggestions should be reviewed before being relied on.</li><li>We evaluate outputs for bias at least annually. The 2026 baseline cycle completed with a 2026-05-07 baseline run and a 2026-05-14 remediation rerun that cleared all configured thresholds. Findings drive prompt and process updates.</li></ul> <h2>Incident examples and response</h2> <p>Illustrative scenarios, not real incidents:</p> <table><thead><tr><th>Scenario</th><th>Response</th></tr></thead><tbody><tr><td>User reports AI suggested something discriminatory</td><td>Log; analyst review; prompt update plus bias-evaluation rerun; user apology if warranted; disclosure in next bias report</td></tr><tr><td>Model leak of one user&#39;s resume to another</td><td>Immediate Sev 1; forensic review of AI call audit; breach-notification assessment per the <a href="/trust/docs/breach-notification">breach notification policy</a></td></tr><tr><td>Jailbreak used to coerce model into writing fraud content</td><td>Log; update prompt-injection guard patterns; block user if intentional; notify if targeted attack</td></tr></tbody></table> <h2>Data for high-risk HECVAT questions</h2> <ul><li>Providers: OpenAI, Perplexity, ElevenLabs, Deepgram, Google Cloud Text-to-Speech.</li><li>Provider retention: per-request <code>store: false</code> on OpenAI and <code>redact=true</code> on Deepgram minimize storage or transcript exposure where supported. No Zero Data Retention amendments are in place; standard provider retention windows apply.</li><li>Internal AI metadata retention: 365 days (metadata only: no prompts, no completions).</li><li>Disclosure mechanism: first-visit cookie and privacy banner naming AI use, plus the privacy policy and footer-linked AI Disclosure page.</li><li>Opt-out mechanism: none in-product, by design. Users avoid AI by not using AI-assisted features; account deletion is available.</li><li>Human review: output-type specific. Resume, cover-letter, and pitch drafting outputs are intended for user review; scoring, transcription, task breakdowns, and value-proposition suggestions are informational aids.</li><li>Bias eval frequency: annual intended cadence. The 2026 baseline cycle completed with a 2026-05-07 baseline run and a 2026-05-14 remediation rerun that cleared all configured thresholds.</li><li>Audit-log retention: 365 days for AI-call metadata; 2 years for general audit log.</li></ul> <h2>Related</h2> <ul><li><a href="/trust/docs/ai-disclosure">AI use disclosure</a></li><li><a href="/trust/docs/ai-model-inventory">AI model inventory</a></li><li><a href="/trust/docs/ai-bias-evaluation">AI bias evaluation</a></li><li><a href="/trust/docs/privacy">Privacy policy</a></li></ul>