Skip to main content
GET
cURL
When to use: After starting a crawl job with the POST endpoint, use this endpoint to:
  • Check if the crawl is complete
  • Monitor real-time progress
  • Get the total number of listings found when the crawl completes
Authentication Required: This endpoint requires a valid API key. The API key is verified, but execution IDs are not restricted to specific API keys. Keep your execution IDs secure.Cancel Running Executions: If you need to stop a crawl that is currently running, use DELETE /v1/crawl/{execution_id} to stop the execution.

Request

string
required
Your API key for authentication
string
required
The execution ID returned from POST /v1/crawlFormat: crawl-{hostname}-{uuid}Note: If the execution ID does not exist, the endpoint returns a 404 Not Found error.

Response

string
Current execution statusPossible values:
  • "pending" - Execution has been created but not yet started
  • "running" - Execution is currently processing
  • "completed" - Execution finished successfully
  • "failed" - Execution failed or was aborted
number | null
Total number of product listings discovered during the crawl (only available when status is “completed”)Note: This value is extracted from the execution output and may be null if the crawl is still running or if the count cannot be determined.
cURL

Polling Strategy

For best results when waiting for completion:
  1. Initial Poll: Check status immediately after receiving execution_id
  2. Polling Interval: Wait 10-30 seconds between polls for running executions (crawls can take longer than product processing)
  3. Exponential Backoff: Consider increasing wait time for long-running crawls
  4. Timeout: Set a maximum wait time based on the size of the vendor website
  5. Cancellation: If a crawl is taking too long or you need to stop it, use DELETE /v1/crawl/{execution_id} to stop running executions