Prompt Engineering

A Prompt Engineering assessment measures how someone works with an AI tool. The candidate gets a task, writes a prompt, and a model produces output from that prompt. What is scored is the prompting and the iteration, not the model's answer.

Use it where the job involves directing an AI rather than recalling syntax. It sits next to interviews rather than replacing them: no video, no recording, no live interviewer, and the whole thing is text.

What an attempt is

An attempt is one prompt submitted against one task. It is the unit everything else is counted in.

Each task carries its own budget, max_attempts. Submitting a prompt spends one. The attempt is generated and scored on its own, so the candidate sees feedback and can rewrite. When the budget runs out, that task is evaluated as a whole and locked.

That is deliberate. A candidate who lands a good result on attempt four after three bad prompts is telling you something different from one who lands it on attempt one, and the task score is an evaluation of the sequence, not of the best attempt in it.

Two things follow that surprise people:

• Spending every attempt on a task finalizes that task with no further call. Spend them on every task and the session completes on its own.
• A task with zero attempts is not skipped. It is scored zero and averaged in.

The shape of an assessment

An assessment is a pack of tasks. Tasks are grouped into slots, and a slot can hold several interchangeable variants. When you assign the assessment, one variant per slot is drawn at random and locked to that session, so two candidates on the same assessment do not necessarily get the same tasks.

GET /v1/prompt-engineering-assessments/{id} returns one representative task per slot (the lowest task id in that slot), so you can show what the assessment covers. Treat them as examples: the variant a session draws is picked at random, so it is not necessarily the one you were shown. Scoring rubrics and evaluator prompts are never returned by any endpoint.

curl -G https://prep-api.experthire.cloud/v1/prompt-engineering-assessments \
  -H "Authorization: Bearer $EH_SECRET_KEY" \
  --data-urlencode "mode=tech"

{
  "object": "list",
  "data": [
    {
      "object": "prompt_engineering_assessment",
      "id": "3f5c0f7e-2b7a-4a51-9f0d-8c2a1d4e6b90",
      "title": "Prompt Engineering Fundamentals",
      "slug": "prompt-engineering-fundamentals",
      "mode": "tech",
      "difficulty": "intermediate",
      "skill_slug": "prompt-engineering",
      "skill_name": "Prompt Engineering",
      "max_attempts_per_task": 5,
      "time_limit_minutes": 45,
      "created_at": 1786348200
    }
  ],
  "total_count": 1
}

You see the shared Expert Hire catalog plus anything your organization authored. Filter with mode (tech or non_tech) and skill_slug. This list is not paginated and takes no limit.

time_limit_minutes is omitted entirely when the assessment is untimed. Absent means untimed, not zero. Do not default it to 0 and count down from there.

Lifecycle

assign ──> assigned ──> started ──> in_progress ──> completed
              │                                        │
              ├──> cancelled                         report
              └──> timed_out

There is no scoring status on this surface, and the status filter on the list endpoint takes these strings, not the integer codes that interviews use.

timed_out is narrower than it sounds. In practice it means one thing: an assignment nobody opened for seven days, terminated by the expiry sweep. It is not the outcome of an abandoned session that was started. Finalizing writes a zero result for every task with no scored attempts and then completes the session, so an abandoned or swept session normally lands on completed with a low or zero score, not on timed_out. A timed_out session never produces a report.

Assigning

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions \
  -H "Authorization: Bearer $EH_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "candidate": { "email": "[email protected]", "name": "Ada Lovelace" },
    "assessment_id": "3f5c0f7e-2b7a-4a51-9f0d-8c2a1d4e6b90"
  }'

Pass candidate_id instead if you already have one. Exactly one of the two is required.

{
  "object": "prompt_engineering_session",
  "id": "9a4b1c62-7d31-4e08-b6a2-5f0c9e7d3a11",
  "status": "assigned",
  "candidate_id": "7f1c2b90-3a44-4d51-9c0e-1b2f9c4d1a55",
  "assessment_id": "3f5c0f7e-2b7a-4a51-9f0d-8c2a1d4e6b90",
  "assessment_name": "Prompt Engineering Fundamentals",
  "skill_name": "Prompt Engineering",
  "report_available": false,
  "resumed": false,
  "environment": "live",
  "livemode": true,
  "created_at": 1786348200
}

The clock does not start here. started_at is absent until the candidate opens the session, because an assigned row carries only its creation time there and that would read as a timer already running.

Assigning is idempotent by organization, candidate and assessment. If that candidate already holds an assigned, started or in_progress session for the same assessment, you get 200 with resumed: true and nothing is charged. A new assignment is 201. Since Idempotency-Key is optional rather than required, that resumed flag is what makes a blind retry safe here.

It only covers live sessions. A completed or cancelled session does not block a new one, so re-assigning the same assessment to the same candidate after they finish creates a second session and spends a second credit. That is usually what you want, but it is not a retry guard.

Sending the candidate in

Mint a launch link and email it.

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/launch-link \
  -H "Authorization: Bearer $EH_SECRET_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ttl_seconds": 604800, "redirect_url": "https://app.example.com/done"}'

{
  "object": "launch_link",
  "subject_kind": "prompt_engineering",
  "subject_id": "9a4b1c62-7d31-4e08-b6a2-5f0c9e7d3a11",
  "url": "https://room.experthire.io/launch?token=ehp_lt_...",
  "token": "ehp_lt_...",
  "expires_at": 1786953000
}

The body is optional. ttl_seconds runs from 300 to 604800 and defaults to seven days. Read the token from the token field rather than parsing the URL, and do not hardcode the host: it is https://room.experthire.io in production and https://room.sandbox.experthire.io otherwise.

The link is single use, the token is stored hashed, and the response above is the only time you see it. The full launch mechanics, including why the exchange has to happen on a click rather than on page load, are in frontend integration.

What the candidate sees

The hosted room takes it from there. The launch token is exchanged for a session token scoped to that one session, and every call below uses that token instead of your secret key.

GET /v1/session renders the pre-join screen: the assessment title, the candidate name, your organization name and logo, whether it is timed and for how long. It starts nothing, so a candidate who opens the link and closes the tab has spent nothing beyond the credit you already spent assigning.

Then the loop:

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/start \
  -H "Authorization: Bearer $EH_SESSION_TOKEN"

That stamps the clock and returns the tasks this session locked, each with its own max_attempts, plus a progress array and any results so far. Calling it again is a 200 that changes nothing: started_at is never re-stamped, so a reload cannot buy extra time.

Submitting a prompt:

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/attempts \
  -H "Authorization: Bearer $EH_SESSION_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "task_id": "c0de4a11-9f2b-4d77-8a31-6b5e2c9d7f04",
    "prompt_text": "You are a technical writer. Rewrite the release note below for...",
    "input_method": "typed"
  }'

prompt_text is capped at 10000 characters. input_method is typed, voice or paste, and defaults to typed.

{
  "object": "prompt_engineering_attempt_receipt",
  "attempt_id": "b2e9f0a4-1c53-4c8a-9d16-3f7a0b4c2e88",
  "attempt_number": 2,
  "remaining": 3
}

202, not 200. Generation and scoring run asynchronously, so poll the attempt until status is scored or error. Nothing else is terminal. The path is submitted to generating to evaluating to scored, and an attempt still non-terminal after five minutes is swept to error.

curl https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/attempts/$ATTEMPT_ID \
  -H "Authorization: Bearer $EH_SESSION_TOKEN"

A scored attempt carries generated_output, an evaluation_feedback string and a score object with prompt_quality, context_framing, instruction_quality, output_quality, off_topic, overall and feedback. Those four dimensions are the ones a single prompt can show. The other four arrive at task level, where there is a sequence to judge.

An attempt that ends in error does not consume the budget: the server counts active attempts and skips errored ones. remaining on the receipt is derived from the attempt number instead, so after an error it under-reports. Use progress[].attempts_used from GET /v1/prompt-engineering-sessions/{id}, which excludes errors, as the number you show the candidate.

The two counts also disagree at the end of a task. Attempt numbers include errored attempts, and the automatic task evaluation triggers on the number reaching max_attempts, so after an error a task can seal while the budget still looks like it has room left. The next submit then fails with attempt_limit_reached.

GET /v1/prompt-engineering-sessions/{id}/timer is the server clock. Count down locally between polls but never upward, and check timed first: it is false for an untimed assessment, and remaining_seconds means nothing then.

Ending the assessment:

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/finalize \
  -H "Authorization: Bearer $EH_SESSION_TOKEN"

Finalize aborts attempts still in flight, evaluates every task that has scored attempts, and writes a zero result for every task that has none. It returns immediately with tasks_scoring set to how many task evaluations it kicked off, so status is usually still in_progress in the response. It is safe to call more than once.

The report

GET /v1/prompt-engineering-sessions/{id}/report, secret key only. A session token cannot read it.

curl https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/report \
  -H "Authorization: Bearer $EH_SECRET_KEY"

It gates, the same way the interview report does. Before the session is completed you get 404 with code report_not_ready, and the message names the current status. The gate is on completion, not on effort: a session that was finalized with no attempts completes and returns a real report scored zero.

{
  "object": "prompt_engineering_report",
  "session_id": "9a4b1c62-7d31-4e08-b6a2-5f0c9e7d3a11",
  "candidate_id": "7f1c2b90-3a44-4d51-9c0e-1b2f9c4d1a55",
  "assessment_id": "3f5c0f7e-2b7a-4a51-9f0d-8c2a1d4e6b90",
  "assessment_name": "Prompt Engineering Fundamentals",
  "skill_name": "Prompt Engineering",
  "status": "completed",
  "overall_score": 71.5,
  "readiness_band": "job_ready",
  "sub_scores": {
    "prompt_quality": 74,
    "context_framing": 68,
    "instruction_quality": 72,
    "iteration_strategy": 70,
    "efficiency": 66,
    "output_quality": 78,
    "adaptability": 73,
    "professional_usage": 71
  },
  "strengths": ["Framed the audience and constraints before asking"],
  "improvements": ["Specify the output format up front"],
  "completed_at": 1786351800,
  "tasks": [
    {
      "task_id": "00000000-0000-0000-0000-000000000000",
      "title": "Rewrite a release note for a non-technical audience",
      "attempts_used": 3,
      "final_score": 74,
      "prompt_quality_score": 76,
      "context_framing_score": 70,
      "instruction_quality_score": 73,
      "iteration_strategy_score": 71,
      "efficiency_score": 68,
      "output_quality_score": 80,
      "adaptability_score": 75,
      "professional_usage_score": 72,
      "strengths": ["Named the audience in the second prompt"],
      "improvements": ["Ask for a length limit up front"],
      "evaluator_summary": "Recovered well after a vague opening prompt."
    }
  ],
  "environment": "live",
  "livemode": true
}

overall_score is the mean of the per-task final_score, and each entry in sub_scores is the mean of that dimension across tasks. Both are clamped to 0 to 100. Because an unattempted task is scored zero rather than dropped, a candidate who finishes two of three tasks well still lands in the low sixties. Read tasks[].attempts_used before reading the headline.

readiness_band is a fixed cut of overall_score: advanced at 85 and above, job_ready at 65, emerging at 40, beginner below that.

Two shape notes. sub_scores, strengths and improvements are JSON that is null when nothing was written, not an empty object or array. And report tasks are identified by title: the task_id field is serialized here but is always 00000000-0000-0000-0000-000000000000, so do not store it. The real id is on the results in GET /v1/prompt-engineering-sessions/{id}, which in turn omit title. Join the two by position, both are ordered by slot.

The report is the evaluator's judgment. The candidate's prompts and the model output they produced are not in it, and no secret-key endpoint returns them: the attempt endpoints take a session token, which only the candidate's browser holds. If you need the prompts, ask us rather than building against something that is not there.

The webhook

Two events, both covered by the registration and signature flow in webhooks.

assessment.assigned fires only on a newly created session, never on a resumed one. It is shared with the other assessment surfaces, so branch on the payload's object field to know it was this one.

prompt_engineering.completed is named for the session finishing, not for a report existing. It also fires for a session that was abandoned past its time limit and finalized for you, which is scored the same way a candidate-driven finalize is. The payload carries status, which is completed on every path that produces a report. Read it before you fetch:

{
  "id": "whevt_9f2c...",
  "type": "prompt_engineering.completed",
  "created_at": 1786351800,
  "livemode": true,
  "data": {
    "object": {
      "object": "prompt_engineering_session",
      "id": "9a4b1c62-7d31-4e08-b6a2-5f0c9e7d3a11",
      "user_id": "7f1c2b90-3a44-4d51-9c0e-1b2f9c4d1a55",
      "organisation_id": "0f1a0ede-...",
      "assessment_id": "3f5c0f7e-2b7a-4a51-9f0d-8c2a1d4e6b90",
      "status": "completed",
      "overall_score": 71.5,
      "readiness_band": "job_ready"
    }
  }
}

The candidate is user_id on the webhook payload and candidate_id on every REST response. Same value, two names.

Sandbox sessions deliver to endpoints registered with a sandbox key, and live sessions to endpoints registered with a live key. They never cross.

Metering

Prompt Engineering draws on its own credit pool. It is separate from the interview pool: GET /v1/usage reports credits.prompt_engineering and credits.interview separately, alongside modules.prompt_engineering. Boot off that rather than discovering a 403 module_not_enabled when a candidate is already in front of you.

One credit per session, charged at assign. Not per attempt, and not per task, so a candidate who spends every attempt on every task costs the same as one who submits nothing.

A resumed assign spends nothing. Neither does anything the candidate does: start, attempts, timer, events and finalize are all free once the session exists.

Cancelling returns the credit, but only while the session is still assigned:

curl -X POST https://prep-api.experthire.cloud/v1/prompt-engineering-sessions/$SESSION_ID/cancel \
  -H "Authorization: Bearer $EH_SECRET_KEY"

Once the candidate has opened it, cancel is refused with session_already_started and the credit stays spent. Cancelling an already-cancelled session is a 200 that changes nothing, with no second refund. The seven-day expiry does not refund: expiry is not a decision you made, so if you want the credit back, cancel inside the window.

Sandbox is a monthly counter rather than a balance, so sandbox assignments are metered but never refunded. Sandbox sessions are also purged after 30 days.

One ordering trap: the candidate is resolved before the credit is checked. A 402 insufficient_credits on a request that used the inline candidate object has already created that candidate. Reuse the id, do not assume nothing happened.

What is not here yet

Honest gaps, so you do not design around them.

• Idempotency-Key is optional. The assign endpoint is also idempotent by candidate and assessment, which covers the common retry, but nothing else on this surface is. A retried finalize is harmless; a retried assign after the first one completed is a second session and a second credit.
• No proctoring readback. POST /v1/prompt-engineering-sessions/{id}/events stores browser signals, and no endpoint returns them. They are not in the report, not in the session detail, and not in the webhook payload. Posting them today only feeds Expert Hire's own reporting. A request carries 1 to 200 events, and once 200 are stored on a session further batches are dropped: recorded comes back as 0 with a 202 and no error. A completed or timed_out session records nothing the same way.
• No prompt retrieval for partners. Covered above: attempts are readable with a session token only.
• No server-side start. start, attempts, timer, events and finalize all require a session token. You cannot drive an assessment from your backend with a secret key, and there is no way to finalize a stuck session on the candidate's behalf. An abandoned timed session is swept and finalized roughly 30 minutes past its limit; an abandoned untimed one is not swept at all.
• No task-level or attempt-level pagination. Attempts are returned in full for one task_id, which is required on that call.
• No way to choose the variant. Slot variants are drawn at random at assign time, and nothing lets you pin one or read the seed.

Every operation on this surface is listed in the API reference, which is generated from the spec.