​
Base URL

All API requests should be made to: 
https://spideriq.di-atomic.com

​
Authentication

All endpoints require Bearer token authentication. See Authentication for details. 
Authorization: Bearer <client_id>:<api_key>:<api_secret>

​
API Endpoints

​
Job Management

Submit and manage scraping jobs.

Submit Job

POST /api/v1/jobs/submit

SpiderSite Job

POST /api/v1/jobs/spiderSite/submit

SpiderMaps Job

POST /api/v1/jobs/spiderMaps/submit

Get Status

GET /api/v1/jobs/{id}/status

Get Results

GET /api/v1/jobs/{id}/results

List Jobs

GET /api/v1/jobs/list

Cancel Job

DELETE /api/v1/jobs/{id}

​
System

Monitor system health and queue statistics.

Health Check

GET /api/v1/system/health

Queue Stats

GET /api/v1/system/queue-stats

System Info

GET /api/v1/system/info

​
Response Format

​
Success Response

All successful responses follow a consistent structure: 
{
  "success": true,
  "job_id": "uuid",
  "type": "spiderSite" | "spiderMaps",
  "status": "completed",
  "data": {
    // Flat structure with results
  }
}

​
Error Response

Error responses include detailed error information: 
{
  "detail": "Error message describing what went wrong"
}

​
Status Codes

CodeMeaningDescription
200OKRequest successful, results available
201CreatedResource created successfully
202AcceptedRequest accepted, processing (poll for results)
400Bad RequestInvalid request format or parameters
401UnauthorizedMissing or invalid authentication
403ForbiddenClient account inactive
404Not FoundResource not found
410GoneJob failed or was cancelled
422Unprocessable EntityValidation error (internal bug)
429Too Many RequestsRate limit exceeded
500Internal Server ErrorServer error
503Service UnavailableQueue service unavailable

​
Rate Limiting

  • Limit: 100 requests per minute per client
  • Burst: 20 requests allowed in burst
  • Headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
When rate limited, you’ll receive a 429 response with a Retry-After header.

​
Pagination

List endpoints support pagination: 
GET /api/v1/jobs/list?page=1&page_size=50
Parameters:
  • page - Page number (default: 1)
  • page_size - Items per page (default: 50, max: 100)
Response: 
{
  "total": 1234,
  "page": 1,
  "page_size": 50,
  "jobs": [...]
}

​
Filtering

List jobs by status or type: 
GET /api/v1/jobs/list?status_filter=completed&type_filter=spiderSite

​
Versioning

Current API version: v1 The version is included in the URL path: /api/v1/*

​
OpenAPI Specification

Download the complete OpenAPI spec:

OpenAPI JSON

View or download the OpenAPI 3.0 specification

​
SDKs and Libraries

Currently, SpiderIQ does not provide official SDKs. Use standard HTTP libraries:
  • Python: requests, httpx
  • JavaScript: fetch, axios
  • PHP: curl, Guzzle
  • Go: net/http
  • Ruby: faraday, httparty

​
Support

Need help? Contact us:

Get Support

Email our support team