# Web scrape

> Read one web page as markdown or plain text, with its title.

`web_scrape` reads one web page and returns its content as clean markdown or plain text, with the page title. Use it to read a URL you already have, such as a search result, an article or a documentation page. JavaScript-rendered pages are supported by the providers that render them. To get typed fields from a page instead of its text, use [`web_extract`](https://clonvo.com/docs/web-extract.md); to read many pages of a site, use [`web_crawl`](https://clonvo.com/docs/web-crawl.md).

Call it with `POST /v1/tools/web_scrape`, or as `web_scrape` over [MCP](https://clonvo.com/docs/mcp.md).

## Arguments

-   `url` string Required
    
    The http(s) URL of the page to read.
    
-   `format` string
    
    Output format: "markdown" (default) or "text".
    
    One of `markdown` `text`
    
-   `only_main_content` boolean
    
    Return only the main content, without navigation, headers, footers and ads (default true).
    
-   `max_chars` integer
    
    The most characters of content to return (default 50,000); longer pages are cut and marked truncated.
    

## Example

```bash
curl https://api.clonvo.com/v1/tools/web_scrape \
  -H "Authorization: Bearer $CLONVO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "arguments": { "url": "https://www.postgresql.org/about/" } }'
```

```json
{
  "id": "gen-01M69T9C2B6D8F0H2K4M6P8R0T",
  "tool": "web_scrape",
  "provider": "clonvo",
  "result": {
    "url": "https://www.postgresql.org/about/",
    "title": "PostgreSQL: About",
    "content": "# About\n\nPostgreSQL is a powerful, open source…",
    "format": "markdown",
    "truncated": false,
    "status_code": 200,
    "published_at": null
  },
  "usage": { "cost": 0.001, "charged": 0.00102, "is_byok": false }
}
```

## Result

| Field | Description |
| --- | --- |
| `url` | The page that was read, after redirects when the provider reports them. |
| `title` | The page title, or `null`. |
| `content` | The page content in `format`, up to `max_chars`. |
| `format` | `markdown` or `text`, as you asked. |
| `truncated` | `true` when the page was longer than `max_chars` and was cut. |
| `status_code` | The HTTP status the provider got from the page, when it reports one. |
| `published_at` | The page’s publication date, when known. |
| `notes` | Present when the provider handled an argument differently. |

The fields around `result` (`id`, `tool`, `provider`, `usage`) are the same for every tool; see [Calling tools](https://clonvo.com/docs/calling-tools.md#response).

## Notes

-   Results can differ slightly between providers. Some always extract the main content, so `only_main_content: false` can be ignored, and `format: "text"` can be derived from the page markdown by stripping markup. `notes` says when an argument was handled differently.
-   Some providers may return a recent indexed copy of the page instead of fetching it live.
-   On Clonvo credits, a page read costs at most $0.02; see [Tool billing](https://clonvo.com/docs/tool-billing.md#on-clonvo-credits).