Get Your API Credentials
Before you begin, you’ll need API credentials. Contact us to register a client:
You’ll receive:
- Client ID:
cli_abc123...
- API Key:
sk_def456...
- API Secret:
secret_ghi789...
Store your credentials securely - they are shown only once!
Make Your First Request
1. Check API Health
First, verify the API is accessible:
curl https://spideriq.di-atomic.com/api/v1/system/health
2. Submit a Scraping Job
Let’s scrape a website:
curl -X POST https://spideriq.di-atomic.com/api/v1/jobs/spiderSite/submit \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <client_id>:<api_key>:<api_secret>" \
-d '{
"payload": {
"url": "https://example.com"
},
"priority": 5
}'
{
"job_id": "974ceeda-84fe-4634-bdcd-adc895c6bc75",
"type": "spiderSite",
"status": "queued",
"created_at": "2025-10-27T10:30:00Z",
"from_cache": false,
"message": "Job submitted successfully and queued for processing"
}
3. Poll for Results
Check if your job is complete:
curl https://spideriq.di-atomic.com/api/v1/jobs/<job_id>/results \
-H "Authorization: Bearer <client_id>:<api_key>:<api_secret>"
- 200 OK - Job completed, results available
- 202 Accepted - Job queued/processing, poll again
- 410 Gone - Job failed or was cancelled
Understanding the Response
When your job completes (200 OK), you’ll get a flat structure response:
{
"success": true,
"job_id": "974ceeda-84fe-4634-bdcd-adc895c6bc75",
"type": "spiderSite",
"status": "completed",
"processing_time_seconds": 19.77,
"worker_id": "spider-site-main-1",
"completed_at": "2025-10-27T11:42:20Z",
"data": {
"url": "https://example.com",
"pages_crawled": 10,
"emails": ["contact@example.com"],
"phones": ["+1-555-0123"],
"linkedin": "https://linkedin.com/company/example",
"twitter": null,
"facebook": "https://facebook.com/example",
// ... all 14 social platforms at top level
"markdown_compendium": "# Content summary...",
"metadata": {
"browser_rendering_available": true
}
}
}
All social media platforms (14 total) are flat at the data level, not nested.
This makes integration much easier!
Next Steps
