# SerpPortal SERP API — REST reference for LLMs

> The self-hosted Google SERP API only. Crawls Google and returns the top 100 organic results, the
> AI Overview, and SERP features for a keyword, with geo and device targeting. Job-based: submit a
> search to get a job_id, then poll for the result.

- Base URL: https://serp-api.hoangha.shop
- Interactive explorer (Swagger UI): https://serp-api.hoangha.shop/docs
- OpenAPI schema: https://serp-api.hoangha.shop/openapi.json
- Auth: `X-API-Key: sk_live_...` on every request. Missing/invalid key -> 401. Keys carry a tier and monthly quota; keep them server-side.

## POST /v1/search

Submit a crawl. JSON body.

Core:
- `keyword` (string, required): the search query. 1-400 chars.
- `location` (string): locality to geo-target, e.g. "Hanoi, Vietnam". Resolves gl/hl/UULE. Preferred over gl/hl.
- `device` (string): "desktop" (default), "mobile", or "tablet".
- `async` (boolean): true (default) returns a job_id immediately; false blocks until the crawl finishes or the sync timeout elapses.
- `stop_domain` (string): stop paginating once this domain appears (host-matched). Rank-check shortcut; saves egress.

Geo overrides (prefer `location`):
- `gl` (string): country code, e.g. "vn", "us", "gb".
- `hl` (string): interface language, e.g. "vi", "en".
- `uule` (string): pre-encoded UULE passthrough.
- `google_domain` (string): search host, e.g. "google.co.uk".

Capture: the AI Overview is captured by default on every search (one page-1 load). It fetches no extra result page, but it is not free: the crawler holds the page open for a render wait and reloads the SERP once on a fast-absent AI Overview. capture_aio=false skips both.
- `capture_aio` (boolean): capture the AI Overview. Default true. False skips its render wait for a faster crawl and returns no `ai_overview`. Part of the cache key, so AIO-on and AIO-off results never mix.
- `capture_features` (boolean): parse rich SERP feature blocks. Defaults to whatever `capture_aio` resolves to, reusing the same page-1 load (0 extra requests). Set true explicitly to keep features while `capture_aio` is false.
- `mode` (string): crawl strategy for the organic pages (the AI Overview is always a browser load). null (default) uses the configured strategy ("browserfetch2"). Selectable: "browserfetch2" (default; N-tab pair variant, crawls matching-locale keywords together in one browser), "browserfetch" (in-browser fetch; holds the browser slot for the whole crawl), "wreq" (Rust HTTP engine reusing browser cookies), "cffi" (curl_cffi HTTP client), "primp" (Rust reqwest HTTP client). The three HTTP-client modes are A/B baselines and measure far slower on the current pool. "browser" is an internal escalation fallback only, not user-selectable.

Google query knobs (part of the cache key): `tbs`, `tbm` ("nws"/"isch"/"vid"/"shop"), `safe` ("active"/"off"), `nfpr` (bool), `filter` (bool), `lr` ("lang_en"), `cr` ("countryVN"), `color_scheme` ("light"/"dark"), plus the opaque Google identifiers `ludocid`, `lsig`, `kgmid`, `si`, `ibp`, `uds` (escaped and forwarded verbatim).

Coordinate geo: `lat` + `lon` (both required together) with optional `radius` (metres, 1-1000; omit = the coordinates are exact) are encoded into the coordinate form of the UULE. `location`, `uule` and `lat`/`lon` all answer "where is this search from" and Google honours exactly one, so sending more than one returns 422 rather than silently picking a winner.

`pages` (int 1-10): how many result pages to CRAWL (10 results each). Unset = the configured depth, normally the full ~100. The one knob that trades coverage for latency and deliberately an explicit opt-in: `pages=1` is a single navigation and by far the fastest search, since full depth costs ~10x the page requests. Part of the cache key — a shallow crawl is never served to a full-depth caller, though a cached full crawl WILL serve a shallow request.

`no_cache` (bool): crawl fresh instead of serving a recent identical result. The fresh result is still cached for later callers, so this costs one crawl, not every future one.

Response: `{ "job_id": "...", "status": "queued", "cached": false }`. Cached identical search -> terminal status immediately, no quota consumed.

```
curl -X POST https://serp-api.hoangha.shop/v1/search \
  -H "X-API-Key: sk_live_..." \
  -H "Content-Type: application/json" \
  -d '{"keyword":"best running shoes","location":"Hanoi, Vietnam","device":"desktop"}'
```

Caching: keyed on keyword + location + device + mode + UULE + query knobs, for a short window. A hit returns `cached: true` and costs no quota.

## GET /v1/search/{job_id}

Fetch a job. Query param `output`:
- `html` (default): metadata + presigned URL to the raw assembled SERP HTML.
- `json`: native schema (search_metadata, search_parameters, organic_results, ai_overview, serp_features).
- `serpapi`: SerpApi-compatible field names (organic_results[].link, answer_box, related_questions, ai_overview.references). Not byte-identical.

Query param `num` (integer): maximum organic results to return, applied after `start`. Windows the stored crawl, never sent to Google. To crawl fewer pages in the first place, use `pages` on POST /v1/search.

Query param `start` (integer, default 0): result offset — 0 returns from result 1, 10 from result 11. Crawls are always full depth, so every offset is served from the same stored SERP: paging costs nothing and never triggers a second crawl. Positions are NOT renumbered (at start=10 the first row is still position 11), and `completeness` keeps describing the crawl rather than the window.

Statuses: queued, running (non-terminal); done, partial, failed (terminal). While polling json/serpapi, a non-terminal job returns HTTP 202 `{ "job_id": "...", "status": "running" }`; poll every 2-3s until HTTP 200. Polls consume no quota but DO count against the per-minute rate limit (60 by default), so a tighter loop on a ~40s crawl will 429 on its own polling.

Completeness (native) = collected / min(target, total_results). A short keyword can score 1.0; a low ratio on a partial means it was blocked.

## SerpApi-compatible surface (drop-in)

serpapi.com's own paths, parameter names, auth style and response envelope are served at the root, so an existing SerpApi client only changes its host:

- `serpapi.HTTPClient.BASE_DOMAIN = "https://serp-api.hoangha.shop"` (serpapi-python)
- `GoogleSearch.BACKEND = "https://serp-api.hoangha.shop"` (google-search-results)

Endpoints: `GET /search` (and `/search.json`), `GET /searches/{id}` (also `{id}.json` / `{id}.html`), `GET /account` (and `/account.json`), `GET /locations.json`. Auth is `?api_key=` (an `X-API-Key` header also works). Errors are a bare `{"error": "..."}`.

Parameters use SerpApi's names: `q`, `location`, `uule`, `lat`, `lon`, `radius`, `google_domain`, `gl`, `hl`, `cr`, `lr`, `tbs`, `tbm`, `safe`, `nfpr`, `filter`, `ludocid`, `lsig`, `kgmid`, `si`, `ibp`, `uds`, `color_scheme`, `start`, `num`, `device`, `no_cache`, `async`, `output`. Unrecognised params are ignored, not rejected. `engine` must be `google`.

Response blocks: `search_metadata` (id, status, json_endpoint, created_at, processed_at, google_url, raw_html_file, total_time_taken), `search_parameters`, `search_information`, `organic_results`, `ai_overview`, `pagination`, `serpapi_pagination`, plus feature blocks. Status is SerpApi's vocabulary: Success | Processing | Error (our `partial` maps to Success — the caller got results; the native `/v1` output still reports the real status).

Extensions beyond SerpApi's set: `pages` (1-10, crawl depth, overrides the depth derived from num/start) and `capture_aio` (AI Overview inline, default ON — richer than serpapi.com, which defers it to a second billable call via `page_token`; set false to skip the render wait). `num=10&capture_aio=false` is the lowest-latency search.

Four differences: (1) we crawl Google live, so a fresh search takes seconds; a blocking `/search` that misses the sync timeout returns SerpApi's async shape (`status: "Processing"` + `json_endpoint`) to poll, and the crawl still lands. (2) `num`/`start` decide how many pages we CRAWL — matching serpapi.com, where one call is one page and `num` defaults to 10 — then window the results; neither is sent to Google as a URL parameter, `position` is never renumbered, and a small `num` is served free from a cached full-depth crawl when one exists. (3) `location`/`uule`/`lat`+`lon` are mutually exclusive (400, never a silent pick). (4) `json_restrictor` is accepted and ignored (you get the full payload), `/account` plan-price fields report 0, and `/locations.json` omits Google's internal ids and `reach` rather than faking them.

## GET /v1/usage

`{ "tier": "pro", "monthly_quota": 10000, "used_this_month": 1342, "remaining": 8658, "rate_limit_per_min": 120 }`
- monthly_quota 0 = unlimited. Each accepted POST that starts a crawl = 1 unit. Cached results and polling do not consume quota.

## Datasets API (serpupdate.com-compatible)

An async batch surface, separate from `/v1/search`, for callers that submit an exact keyword list (each with its own country/language/device/uule) and fetch results as one snapshot. Request/response shapes and URL paths mirror serpupdate.com exactly, so an existing serpupdate.com client can point at this base URL with no code changes. Fully asynchronous: the snapshot row persists BEFORE crawling starts; you poll or subscribe to a webhook for results.

### POST /datasets/v1/trigger

Submit up to 200 keywords. Persists the snapshot immediately, enqueues crawl jobs, and returns the `snapshot_id` for polling.

Request body: `{ "id": "...", "input": [ item, ... ] }`

Per-item fields:
- `keyword` (string, required): the search query.
- `country` (string): country code, maps to Google `gl`, e.g. "VN".
- `language` (string): interface language, maps to Google `hl`, e.g. "vi".
- `brd_mobile` (string): "desktop" (default) or "mobile".
- `url` (string): a Google host, e.g. "https://www.google.co.uk/", sets the search domain. Defaults to google.com.
- `uule` (string): pre-encoded UULE (overrides country).
- `start_page`, `end_page` (int): slice the `organic` results to this page range (default 1-10). We always crawl full ~100 depth; these only slice the response.
- `tbm` (string): Google vertical: "" (web, default), "nws" (news), "isch" (images), "vid" (videos), "shop" (shopping).
- `capture_aio` (boolean): capture the AI Overview (native output only, on by default).
- `id` (string, optional): your own id for this keyword, echoed in output=html markers.

Top-level `id` (next to `input`): your own id for the snapshot, echoed on all responses and the webhook, and used as the output=html filename. Both ids accept only `[A-Za-z0-9._-]`, max 64 chars (they become filenames; anything else is a 400). Not unique, never a lookup key.

Response: `{ "snapshot_id": "sd_...", "id": "..." }` (id echoed only if sent). Quota: 1 unit per keyword, charged only on a valid request. 429 if at in-flight ceiling or over monthly quota.

Example:
```
curl -X POST https://serp-api.hoangha.shop/datasets/v1/trigger \
  -H "X-API-Key: sk_live_..." -H "Content-Type: application/json" \
  -d '{
    "id": "batch-1",
    "input": [
      {"keyword": "sàn nhựa", "country": "VN", "language": "vi"},
      {"keyword": "web design", "country": "US", "language": "en", "brd_mobile": "mobile"}
    ]
  }'
```

### GET /datasets/v1/snapshot/{snapshot_id}

Poll for results. Append query params: `?output=serpupdate` (default), `?output=native`, or `?output=html`.

While crawling (status = processing): returns `{ "snapshot_id", "status": "processing", "created_at" }`.

When complete (status = completed):
- **output=serpupdate** (default): bare array, one element per input keyword, positionally aligned. Each: `{ keyword, general: {search_engine, language, location, search_type, page_title, datetime, query}, language, country, organic: [ {url, rank, link, title, description}, ... ], page_html }`. The `organic` array is sliced to start_page..end_page window; `page_html` is the raw HTML of the first page in that window.
- **output=native**: serp-api's full parse (includes AI Overview, SERP features, People Also Ask, related searches).
- **output=html**: raw crawled pages as one gzipped HTML file. Content-Type: application/gzip (not Content-Encoding, so clients save it). Filename: `<your id>.html.gz`, falling back to a single item's id, then snapshot_id. Pages delimited by `<!--===SERP_PAGE:N url="..." ===-->` (page 1 = start=0, page 2 = start=10, etc.). Multi-keyword snapshots prefix each keyword with `<!--===SERP_KEYWORD:I id="..." keyword="..." ===-->`. The AI Overview render follows page 1 under `<!--===SERP_AI_OVERVIEW url="..." ===-->` (own marker, not a page number: the paginated pages use udm=14 which strips the AIO, so it exists only in that default-SERP render) — it sits between `SERP_PAGE:1` and `SERP_PAGE:2`, so slicing page 1 as an extension capture is sliced keeps the AIO with it; absent when the crawl opted out of AIO. Retained for 30 days; JSON results are not affected.

All outputs return the same status envelope until completion. `id` appears only when you sent one at trigger time.

### PUT /datasets/v1/pingback

Register a completion webhook, once per API key. Body: `{ "pingback_url": "https://your-host/hook" }` or `null` to clear.

When a snapshot completes, we POST `{ "snapshot_id", "status": "ready", "id" }` (id only if you sent one) to your URL. Best-effort delivery, SSRF-guarded; polling is the fallback. Your endpoint must accept POST and respond with 2xx.

## AI Overview & SERP features

AI Overview captured on the page-1 load only. Native `ai_overview`: `summary` (string[], back-compat), `markdown` (string), `text_blocks` (ordered `{type: paragraph|heading|list|table, ...}`), `citations` (`{rank, title, url, domain, snippet}`). serpapi maps to `text_blocks` (snippet/list) + `references` (`{title, link, source, snippet, index}`). Job metadata: `aio_status` (present|absent|failed|skipped), `aio_citations` (int).
SERP features (serpapi alias): featured_snippet (answer_box), people_also_ask (related_questions), related_searches, local_pack (local_results), knowledge_panel (knowledge_graph), inline_images, inline_videos, top_stories, shopping_results, sitelinks. Absent blocks are omitted.
Per-result enrichment (native JSON): domain_info, classification, favicon.

## Errors

200 OK; 202 Accepted (poll again); 400 Bad Request (invalid input); 401 Unauthorized (bad key); 403 Forbidden (locale not enabled); 404 Not Found (no job); 422 Unprocessable (stored HTML unavailable); 429 Too Many Requests (quota/rate limit, back off); 500 Server Error (retry with backoff). A `partial` job is not an error.