e2e-cli-test

E2E CLI Test

Pre-release validation that exercises ./bin/heygen against the live HeyGen API.

Prerequisites

• HEYGEN_API_KEY must be set in the environment
• Working directory must be the heygen-cli repo root

Workflow

Run each phase in order. Report results as you go. If a phase fails, continue to the next phase (do not abort early) so the final report covers everything. Always use ./bin/heygen (the freshly built binary), never a globally installed heygen.

Step 1: Build

make build

If this fails, stop and report the build error. Nothing else can run.

Step 2: Phase 1 -- Auth and account

./bin/heygen auth status
./bin/heygen user me get

• Assert auth status exits 0
• Assert user me get exits 0, stdout is valid JSON, and jq -e '.data.username' succeeds
• If either command fails, stop and report the auth error. Every subsequent phase requires a valid key.

Step 3: Phase 2 -- Read-only list commands

Run each command below. Assert exit 0 and valid JSON stdout for every one. Save the JSON output from each -- Phase 3 will extract IDs from these results.

./bin/heygen video list --limit 1
./bin/heygen template list --limit 1
./bin/heygen ai-clipping list --limit 1
./bin/heygen avatar list --limit 1
./bin/heygen avatar looks list --limit 1
./bin/heygen voice list --limit 1
./bin/heygen audio sounds list --query "calm ambient piano" --limit 1
./bin/heygen video-translate list --limit 1
./bin/heygen video-translate languages list
./bin/heygen video-agent list --limit 1
./bin/heygen video-agent styles list --limit 1
./bin/heygen lipsync list --limit 1
./bin/heygen webhook endpoints list
./bin/heygen webhook event-types list
./bin/heygen webhook events list
./bin/heygen brand kits list --limit 1
./bin/heygen brand glossaries list --limit 1

asset list requires --username (the workspace member whose assets to list, the owner value on asset items) while the endpoint is in beta. Use the .data.username captured in Phase 1 (user me get): ./bin/heygen asset list --username <phase-1-username> --limit 1.

Step 4: Phase 3 -- Read-only get/detail commands

For each command below, extract the required ID from the corresponding Phase 2 list result. If a list returned an empty .data array, skip that detail command and mark it as SKIPPED (not FAIL). If more than half of the detail commands are skipped, mark the phase as WARN and print a warning that the account lacks sufficient data for meaningful get/detail coverage.

./bin/heygen video get <video-id>              # .data[0].id from video list
./bin/heygen video scenes get <video-id>       # .data[0].id from video list; 404 "no editor document" (e.g. a translated video) is SKIPPED, not FAIL
./bin/heygen ai-clipping get <job-id>          # .data[0].id from ai-clipping list
./bin/heygen avatar get <group-id>             # .data[0].id from avatar list
./bin/heygen avatar looks get <look-id>        # .data[0].id from avatar looks list
./bin/heygen video-agent get <session-id>      # .data[0].session_id from video-agent list
./bin/heygen video-agent videos list <session-id>
./bin/heygen video-translate get <id>          # .data[0].id from video-translate list
./bin/heygen lipsync get <id>                  # .data[0].id from lipsync list
./bin/heygen voice get <voice-id>              # .data[0].voice_id from voice list
./bin/heygen template get <template-id>        # .data[0].id from template list
./bin/heygen brand kits get <id>               # .data[0].brand_kit_id from brand kits list
./bin/heygen brand glossaries get <id>         # .data[0].brand_glossary_id from brand glossaries list
./bin/heygen video statuses list --video-ids <video-id>   # a video-id from video list (bulk status lookup)
./bin/heygen lipsync statuses list --lipsync-ids <id>     # .data[0].id from lipsync list
./bin/heygen video-translate statuses list --video-translation-ids <id>  # .data[0].id from video-translate list

asset statuses list --asset-ids <asset-id> needs an asset id from the asset list result above (.data[0].id). Skip only if that list is empty.

<resource> batches get <batch-id> needs a batch id, which has no read-only list source (batches are created via the write path). Run each only if a batch id is available from a prior batch-create call; otherwise skip. The producer is video batches create / lipsync batches create / video-translate batches create, and for assets asset direct-uploads batches create.

For video-agent resources get: first run ./bin/heygen video-agent get <session-id>, then look for a resource_id in messages[*].resource_ids[*]. Only run ./bin/heygen video-agent resources get <session-id> <resource-id> if both values are available; otherwise skip.

Assert exit 0 and valid JSON for each command that runs.

Step 5: Phase 4 -- --human output mode

./bin/heygen video list --limit 1 --human
./bin/heygen template list --limit 1 --human
./bin/heygen avatar list --limit 1 --human
./bin/heygen voice list --limit 1 --human

Assert exit 0. Assert stdout is NOT valid JSON (it should be a formatted table).

Step 6: Phase 5 -- Schema introspection

These do not make API calls.

./bin/heygen video create --request-schema
./bin/heygen video create --response-schema

Assert exit 0 and stdout is valid JSON for each.

Step 7: Phase 6 -- Error handling

# Invalid API key -- expect exit 3 (auth error)
HEYGEN_API_KEY=sk_invalid_key_000 ./bin/heygen user me get

# Missing required flags -- expect exit 2 (usage error)
./bin/heygen video get

Assert the expected exit codes. Temporarily override HEYGEN_API_KEY for the auth test only; restore the real key afterward.

Step 8: Phase 7 -- Write path (costs credits)

This phase must always clean up, even on failure.

# Create a short video. Keep the prompt phrased as a video request -- the agent
# answers a conversational one ("Say hello world") in chat and never generates,
# so the phase burns the full --wait timeout and reports FAIL on a healthy CLI.
./bin/heygen video-agent create --prompt "Create a short 5 second video of an avatar saying: Hello world." --wait

• Accept exit 0 (completed) or exit 4 (poll timeout) as success
• Extract the video id from stdout as .data.id. Under --wait the CLI prints the polled video object, not the create response, in both the completed and the timeout case -- so there is no .data.video_id to read here
• If exit code is anything other than 0 or 4, or the id is missing, fail immediately but still run cleanup below

# Verify the video exists
./bin/heygen video get <video_id>

• If create returned exit 4, continue polling with ./bin/heygen video get <video_id> in a bounded loop (e.g., every 15 seconds for up to 5 minutes) until .data.status reaches completed or failed
• If status reaches failed or never reaches completed, mark the phase as FAIL

# Download the completed video (use a unique temp path to avoid collisions)
DOWNLOAD_PATH="/tmp/e2e-cli-test-$$-$(date +%s).mp4"
./bin/heygen video download <video_id> --output-path "$DOWNLOAD_PATH" --force

• Assert exit 0, stdout is valid JSON with a .path field
• Assert the file exists at $DOWNLOAD_PATH with non-zero size

# Cleanup (unconditional -- always runs if video_id was extracted)
./bin/heygen video delete <video_id> --force
rm -f "$DOWNLOAD_PATH"

Step 9: Report

Print a summary table:

Phase                    Result
-----                    ------
1. Auth and account      PASS
2. List commands         PASS (<passed>/<total>)
3. Get/detail commands   PASS (<passed>/<ran>, <skipped> skipped)   # or WARN if >50% skipped
4. --human output        PASS
5. Schema introspection  PASS
6. Error handling        PASS
7. Write path            PASS

Counts are the actual number of commands executed in each phase (Phase 2/3 grow as new list/get commands are added) — compute them from the run, don't hard-code totals.

If any phase is FAIL, end with a clear message identifying which phase(s) failed and the first failing command with its exit code and stderr.