AI high-risk evaluation

Last reviewed 2026-05-16

This document summarizes HiringCoachAI's posture on the high-risk AI evaluation questions assessed during higher-education vendor reviews.

Classification of our AI use

We classify HiringCoachAI's AI use as moderate risk:

No fully automated decisions with legal or similarly significant effects (Art. 22 GDPR).
No public AI-generated content about identifiable users is published under HiringCoachAI's name. AI-assisted aggregate insights and AI-assisted product marketing content may be published; neither identifies an individual user or uses personal data.
No AI-driven hiring decisions: users generate content for themselves, which they then use at their own discretion.
Human review is output-type specific. Resume, cover-letter, pitch, and similar drafting outputs are intended for user review and editing before use. Other outputs, such as interview-practice scoring, transcription, task breakdowns, and value-proposition suggestions, are informational aids and should not be treated as automated decisions.

The following answers map onto HECVAT 4.1.5 High-Risk items.

Input data categories: Resume text, job description, user-authored questions, audio (for transcription), career history, company context.
Sensitive categories: None intended. We prohibit PHI, government IDs, payment card data, and child-directed data.
Residency: US default. AI providers are primarily US-based.
Retention at provider: Per-request controls minimize storage or transcript exposure where supported (store: false on OpenAI, redact=true on Deepgram). No Zero Data Retention amendments are in place with any AI provider; each provider's then-current standard API retention windows apply. See the AI model inventory; DPA register evidence is available on request via [email protected].
Training use: We do not train models on customer data. We rely on provider standard API terms and per-request retention-minimization flags where available; no separate enterprise no-training amendment has been signed.
Internal retention: aiCallAudit metadata only, no prompts or completions, for 1 year.

Model selection: Driven by feature needs; reviewed at feature kickoff per the secure development lifecycle. Primary model strings are documented in the AI model inventory.
Fine-tuning: None today. If ever adopted, fine-tuning data would be HiringCoachAI-authored, not customer data.
Versioning: Model strings are managed through a central registry.
Evaluation: A formal bias-evaluation methodology is documented at the AI bias evaluation page. A baseline run completed on 2026-05-07: positive controls passed and no demographic or name-derived references were detected. The baseline did surface several output-type findings (length or tone variance, strict-format JSON failures, and one candidate-name leak in a fit-score output). A remediation rerun on 2026-05-14 cleared all thresholds across the expanded suite.

Prompt injection: Regex-based prompt-injection and jailbreak heuristics are applied to AI requests where the safety-check option is enabled.
Output handling: Current controls are scoped AI features, user review before reliance, reporting/escalation, and provider/request controls where available. These are the output controls represented for the current service.
Rate limiting: Per-user rate limiting is applied to selected high-risk endpoints; remaining coverage is reviewed through the API-validation and security workflows.

AI use is disclosed through the first-visit banner, privacy policy, and AI Disclosure page rather than a label on every generated output.

First-visit disclosure banner names AI use and links to the AI Disclosure page.
Privacy Policy discloses AI processing, providers, retention, and lawful basis.
AI Disclosure page at /ai-disclosure enumerates AI features, data sent, and triggers.
No blocking consent modal. AI use is processed under contract performance when the user invokes or configures a feature that requires the call.
No in-product AI opt-out toggle, by design. Users avoid AI processing by not using AI-assisted features; account deletion is available.

Owner: Security Officer and Privacy Officer / data-protection contact.
Incident route: Same as any other Sev 1 or Sev 2 incident; see the incident response policy.
Audit log: AI call audit and the append-only audit log.
Bias eval: methodology documented at the AI bias evaluation page; a baseline run was completed on 2026-05-07 with follow-up items, and a remediation rerun completed on 2026-05-14 with no thresholds exceeded.

Users can avoid AI processing by not using AI-assisted features. Manual workflows do not invoke AI.
Users can export their data via /account/export.
Users can delete their account via /account/delete.
Users can complain to a supervisory authority; see the privacy policy.

We do not use AI to make automated decisions about users.
We do not use AI to profile users for marketing.
We do not claim AI outputs are authoritative without user verification; generated drafts, scores, transcripts, and suggestions should be reviewed before being relied on.
We evaluate outputs for bias at least annually. The 2026 baseline cycle completed with a 2026-05-07 baseline run and a 2026-05-14 remediation rerun that cleared all configured thresholds. Findings drive prompt and process updates.

Illustrative scenarios, not real incidents:

Scenario	Response
User reports AI suggested something discriminatory	Log; analyst review; prompt update plus bias-evaluation rerun; user apology if warranted; disclosure in next bias report
Model leak of one user's resume to another	Immediate Sev 1; forensic review of AI call audit; breach-notification assessment per the breach notification policy
Jailbreak used to coerce model into writing fraud content	Log; update prompt-injection guard patterns; block user if intentional; notify if targeted attack

Providers: OpenAI, Perplexity, ElevenLabs, Deepgram, Google Cloud Text-to-Speech.
Provider retention: per-request store: false on OpenAI and redact=true on Deepgram minimize storage or transcript exposure where supported. No Zero Data Retention amendments are in place; standard provider retention windows apply.
Internal AI metadata retention: 365 days (metadata only: no prompts, no completions).
Disclosure mechanism: first-visit cookie and privacy banner naming AI use, plus the privacy policy and footer-linked AI Disclosure page.
Opt-out mechanism: none in-product, by design. Users avoid AI by not using AI-assisted features; account deletion is available.
Human review: output-type specific. Resume, cover-letter, and pitch drafting outputs are intended for user review; scoring, transcription, task breakdowns, and value-proposition suggestions are informational aids.
Bias eval frequency: annual intended cadence. The 2026 baseline cycle completed with a 2026-05-07 baseline run and a 2026-05-14 remediation rerun that cleared all configured thresholds.
Audit-log retention: 365 days for AI-call metadata; 2 years for general audit log.