> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nativeport.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nativeport.ai/_mcp/server.

# Crawlbase

Fronts the Crawling API of [Crawlbase](https://crawlbase.com); the gateway fills in the secret `token` parameter on the query string:

```
GET|POST /crawlbase      → https://api.crawlbase.com/?token=<normal>&<your query>
GET|POST /crawlbase/js   → https://api.crawlbase.com/?token=<javascript>&<your query>
```

Crawlbase hands out **two** tokens, and **which one gets injected follows from the route**:

| Route           | Token      | Behavior                                  |
| --------------- | ---------- | ----------------------------------------- |
| `/crawlbase`    | Normal     | Fetches static HTML                       |
| `/crawlbase/js` | JavaScript | Loads the page in a real headless browser |

Everything is driven by the **query string**: the target `url` plus any parameter Crawlbase defines (`country`, `format`, `page_wait`, …). What you put there reaches Crawlbase exactly as written.

## Credential handling

* **`token` goes in ahead of everything you send**, so a target URL with incomplete encoding can't swallow it. Which token is chosen depends on which route you called — `/crawlbase` or `/crawlbase/js`.
* **A `token` supplied by the client in the query is discarded** (any encoding), so nothing can displace the injected one.
* **`Authorization` is removed** before the request leaves the gateway; Crawlbase does nothing with your gateway token.

## Examples

#### Static HTML

```bash
curl "$GATEWAY/crawlbase?url=https://example.com" \
  -H "Authorization: Bearer $TOKEN"
```

#### JavaScript (headless browser)

```bash
curl "$GATEWAY/crawlbase/js?url=https://example.com" \
  -H "Authorization: Bearer $TOKEN"
```

Crawlbase's response comes back untouched. Every supported parameter is listed in [Crawlbase's documentation](https://crawlbase.com/docs/crawling-api).