parent
b61bf7b155
commit
4e7b25d825
@ -0,0 +1,348 @@ |
||||
# `http` |
||||
|
||||
The `http` module is an HTTP/HTTPS client for calling APIs, scraping pages, and |
||||
submitting forms. TLS is compiled into the binary (rustls with the `ring` |
||||
provider), so HTTPS works with no system library. There are two ways in: plain |
||||
functions for one-off requests, and a **session** object that carries a cookie |
||||
jar across requests for stateful flows like logging in. |
||||
|
||||
```lua |
||||
local resp = http.get("https://example.com") |
||||
print(resp.status, resp.ok) --> 200 true |
||||
print(resp.body) --> "<!doctype html>…" |
||||
|
||||
-- JSON in one step |
||||
local data = http.getJSON("https://api.example.com/users") |
||||
for _, user in ipairs(data) do |
||||
print(user.name) |
||||
end |
||||
``` |
||||
|
||||
## Making requests |
||||
|
||||
Every helper takes a URL and an optional `opts` table (see [Request |
||||
options](#request-options)) and returns a [response](#the-response) table. |
||||
|
||||
```lua |
||||
http.get(url [, opts]) |
||||
http.post(url [, opts]) |
||||
http.put(url [, opts]) |
||||
http.patch(url [, opts]) |
||||
http.delete(url [, opts]) |
||||
http.head(url [, opts]) |
||||
``` |
||||
|
||||
These are thin shorthands over the one primitive: |
||||
|
||||
### `http.request(method, url [, opts])` |
||||
|
||||
Send a request with an explicit method (any verb, e.g. `"GET"`, `"OPTIONS"`) and |
||||
return the response. The shorthands above are just `http.request` with the method |
||||
filled in. |
||||
|
||||
```lua |
||||
local resp = http.request("DELETE", "https://api.example.com/items/42") |
||||
``` |
||||
|
||||
## The response |
||||
|
||||
Every request returns a table describing the response: |
||||
|
||||
| Field | Type | Description | |
||||
|-----------|-----------|-------------------------------------------------------------------| |
||||
| `status` | integer | HTTP status code, e.g. `200`, `404`. | |
||||
| `ok` | boolean | `true` when `status` is in the 2xx range. | |
||||
| `headers` | table | Response headers, keyed by **lowercase** name. | |
||||
| `body` | string | The raw response body (Lua strings are byte sequences). | |
||||
| `json` | function | `resp.json()` parses `body` as JSON. See [JSON](#json). | |
||||
|
||||
Header names are lowercased so you can look them up without guessing the server's |
||||
capitalization. If a header appears more than once, the first value wins. |
||||
|
||||
```lua |
||||
local resp = http.get("https://example.com") |
||||
print(resp.headers["content-type"]) --> "text/html; charset=utf-8" |
||||
``` |
||||
|
||||
A non-2xx status is **not** an error — `resp.status` and `resp.ok` simply report |
||||
it. Only a failure to get a response at all (DNS, connection, TLS, timeout) raises |
||||
a Lua error. See [Errors](#errors). |
||||
|
||||
## Request options |
||||
|
||||
The optional `opts` table accepts these fields, all optional: |
||||
|
||||
| Field | Type | Behaviour | |
||||
|-----------|--------|--------------------------------------------------------------------------| |
||||
| `headers` | table | Extra request headers, `{["X-Foo"] = "bar"}`. | |
||||
| `body` | string | Raw request body; set `Content-Type` yourself via `headers`. | |
||||
| `json` | any | Serialized to JSON; sets `Content-Type: application/json`. | |
||||
| `form` | table | URL-encoded; sets `Content-Type: application/x-www-form-urlencoded`. | |
||||
| `cookies` | table | Cookies for this request, `{session = "abc"}` → `Cookie:` header. | |
||||
| `timeout` | number | Per-request timeout in seconds (default `30`). | |
||||
| `auth` | table | Basic or digest credentials. See [Authentication](#authentication). | |
||||
|
||||
### Bodies |
||||
|
||||
`json`, `form`, and `body` are three ways to set the request body; if more than |
||||
one is given, the first present in that order wins. |
||||
|
||||
```lua |
||||
-- JSON body (table serialized to an object) |
||||
http.post("https://api.example.com/items", { json = { name = "test", count = 5 } }) |
||||
|
||||
-- Form submission (application/x-www-form-urlencoded) |
||||
http.post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||
|
||||
-- Raw body with an explicit content type |
||||
http.post("https://example.com/ingest", { |
||||
body = "id,name\n1,alice\n", |
||||
headers = { ["Content-Type"] = "text/csv" }, |
||||
}) |
||||
``` |
||||
|
||||
`json` serializes the same way as `utils.toJSON`: a table with |
||||
sequential integer keys becomes a JSON array, otherwise an object, and |
||||
`utils.NULL` becomes JSON `null` (a literal Lua `nil` cannot live in |
||||
a table). |
||||
|
||||
### Headers and cookies |
||||
|
||||
```lua |
||||
local resp = http.get("https://example.com", { |
||||
headers = { ["Accept"] = "application/json", ["X-Token"] = "xyz" }, |
||||
cookies = { session = "abc123" }, |
||||
timeout = 10, |
||||
}) |
||||
``` |
||||
|
||||
Per-request `cookies` are sent as a `Cookie` header. With a [session](#sessions), |
||||
they are merged with the jar and take precedence on a name collision. |
||||
|
||||
## JSON |
||||
|
||||
### `resp.json()` |
||||
|
||||
Parse the response body as JSON and return the resulting Lua value. It is the |
||||
counterpart of `utils.fromJSON`: JSON objects become tables, arrays |
||||
become array-tables, and `null` becomes `utils.NULL`. Calling it on a |
||||
body that is not valid JSON raises an error. |
||||
|
||||
```lua |
||||
local resp = http.post("https://api.example.com/echo", { json = { hello = "world" } }) |
||||
print(resp.json().hello) --> "world" |
||||
``` |
||||
|
||||
### `http.getJSON(url [, opts])` |
||||
|
||||
Shorthand for a GET that parses the body. Returns **two** values: the parsed body |
||||
and the full response. |
||||
|
||||
```lua |
||||
local data, resp = http.getJSON("https://api.example.com/users") |
||||
print(resp.status, #data) |
||||
``` |
||||
|
||||
### `http.postJSON(url, body [, opts])` |
||||
|
||||
Shorthand for a POST with a JSON body — equivalent to setting `opts.json = body`. |
||||
Returns the response. |
||||
|
||||
```lua |
||||
local resp = http.postJSON("https://api.example.com/items", { name = "test", count = 5 }) |
||||
if resp.ok then print(resp.json().id) end |
||||
``` |
||||
|
||||
## Authentication |
||||
|
||||
Pass credentials in `opts.auth`. The `scheme` is `"basic"` (the default) or |
||||
`"digest"`. |
||||
|
||||
| Field | Type | Description | |
||||
|------------|--------|----------------------------------------------| |
||||
| `username` | string | Required. | |
||||
| `password` | string | Required. | |
||||
| `scheme` | string | `"basic"` (default) or `"digest"`. | |
||||
|
||||
```lua |
||||
-- HTTP Basic |
||||
local resp = http.get("https://api.example.com/private", { |
||||
auth = { username = "alice", password = "secret" }, |
||||
}) |
||||
|
||||
-- HTTP Digest — the 401 challenge is answered automatically |
||||
local resp = http.get("https://api.example.com/private", { |
||||
auth = { username = "alice", password = "secret", scheme = "digest" }, |
||||
}) |
||||
``` |
||||
|
||||
For basic auth the `Authorization` header is sent with the request. For digest the |
||||
client sends the request, reads the server's `401` challenge, computes the |
||||
response, and retries once; you only see the final response. Auth works the same |
||||
way on [sessions](#sessions). |
||||
|
||||
## Sessions |
||||
|
||||
A **session** wraps its own cookie jar. Cookies from `Set-Cookie` responses are |
||||
stored automatically and sent back on later requests to matching hosts — which is |
||||
what makes login-then-fetch flows work. A session also has the same request |
||||
methods as the plain module. |
||||
|
||||
### `http.session([path])` |
||||
|
||||
Create a session. With a `path`, the cookie jar is preloaded from that file (see |
||||
[Persisting the jar](#persisting-the-jar)). |
||||
|
||||
```lua |
||||
local s = http.session() -- fresh, empty jar |
||||
local s = http.session("cookies.jsonl") -- jar loaded from disk |
||||
``` |
||||
|
||||
### Requests |
||||
|
||||
A session has every method the plain module has — `request`, `get`, `post`, |
||||
`put`, `patch`, `delete`, `head`, `getJSON`, `postJSON` — called with `:` syntax |
||||
and accepting the same `opts`: |
||||
|
||||
```lua |
||||
local s = http.session() |
||||
|
||||
-- Log in; the response's Set-Cookie is captured into the jar |
||||
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||
|
||||
-- The session cookie is sent automatically |
||||
local page = s:get("https://example.com/dashboard") |
||||
``` |
||||
|
||||
### Cookie behaviour |
||||
|
||||
A stored cookie is sent on a request when the request host **equals** the cookie's |
||||
domain or is a subdomain of it. The jar reads the `name=value` pair and the |
||||
`Domain` attribute from each `Set-Cookie` header; other attributes (`Path`, |
||||
`Expires`, `Secure`, `HttpOnly`) are ignored. When no `Domain` is given, the |
||||
request host is used. |
||||
|
||||
`Set-Cookie` headers set on redirect responses are captured too, so a login that |
||||
`302`-redirects to a dashboard still records its cookie. |
||||
|
||||
### Inspecting and clearing |
||||
|
||||
#### `s:cookies()` |
||||
|
||||
Return the jar as a nested table, `{domain = {name = value}}`, for inspection. |
||||
|
||||
```lua |
||||
local jar = s:cookies() |
||||
for domain, names in pairs(jar) do |
||||
for name, value in pairs(names) do |
||||
print(domain, name, value) |
||||
end |
||||
end |
||||
``` |
||||
|
||||
#### `s:clearCookies()` |
||||
|
||||
Empty the in-memory jar. |
||||
|
||||
### Persisting the jar |
||||
|
||||
Cookies live in memory for the session's lifetime. Save them to reuse a logged-in |
||||
session across script runs. |
||||
|
||||
#### `s:save(path)` |
||||
|
||||
Write the jar to `path` as JSONL — one JSON object per line, one cookie per line: |
||||
|
||||
```jsonl |
||||
{"domain":"example.com","name":"session","value":"abc123"} |
||||
{"domain":"api.example.com","name":"token","value":"xyz789"} |
||||
``` |
||||
|
||||
#### `s:load(path)` |
||||
|
||||
Merge cookies from a JSONL file into the current jar (existing cookies are kept, |
||||
matching names overwritten). `http.session(path)` is the same as creating a |
||||
session and calling `:load(path)`. |
||||
|
||||
```lua |
||||
-- First run: log in and persist |
||||
local s = http.session() |
||||
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||
s:save("session.jsonl") |
||||
|
||||
-- Later run: restore and continue without logging in again |
||||
local s = http.session("session.jsonl") |
||||
local page = s:get("https://example.com/dashboard") |
||||
``` |
||||
|
||||
## Redirects |
||||
|
||||
Redirects are followed automatically, up to 10 hops; you receive the final |
||||
response. A `303`, and a `301`/`302` in response to a `POST`, are followed as a |
||||
bodyless `GET`, matching browser behaviour. For a session, cookies set along the |
||||
way are captured at each hop. |
||||
|
||||
## Errors |
||||
|
||||
Failing to obtain a response raises a Lua error: DNS failure, connection refused, |
||||
a TLS problem, or a timeout. An HTTP error *status* (4xx/5xx) does not — it is |
||||
reported through `resp.status`/`resp.ok`. Parsing a non-JSON body with |
||||
`resp.json()` also raises. |
||||
|
||||
Wrap calls in `pcall` or `utils.try` where you want to handle failure |
||||
rather than abort: |
||||
|
||||
```lua |
||||
local resp, err = utils.try(function() |
||||
return http.get("https://does-not-exist.invalid", { timeout = 5 }) |
||||
end) |
||||
if not resp then |
||||
log.error("request failed: " .. tostring(err)) |
||||
end |
||||
``` |
||||
|
||||
## Notes |
||||
|
||||
- **HTTPS needs no setup.** The TLS stack (rustls + `ring`) is compiled in; trust |
||||
roots come from the system certificate store. |
||||
- **A default `User-Agent` is sent** (`a/<version>`) because some servers reject |
||||
requests without one. Override it with a `User-Agent` entry in `opts.headers`. |
||||
- **Requests don't block the event loop.** Network I/O runs on the async core, so |
||||
a slow request does not stall other async work (timers, `os.sleep`, SQLite) in |
||||
the same script. |
||||
- **The cookie jar is per session.** Plain `http.get`/`http.post` calls do not |
||||
retain cookies between calls; use a session for that. |
||||
|
||||
## Full example |
||||
|
||||
```lua |
||||
-- Talk to a JSON API with a bearer token, then drive a stateful session. |
||||
|
||||
-- One-off authenticated JSON call |
||||
local items, resp = http.getJSON("https://api.example.com/items", { |
||||
headers = { ["Authorization"] = "Bearer " .. token }, |
||||
timeout = 15, |
||||
}) |
||||
if not resp.ok then |
||||
error("list failed: HTTP " .. resp.status) |
||||
end |
||||
for _, item in ipairs(items) do |
||||
print(item.id, item.name) |
||||
end |
||||
|
||||
-- Create one |
||||
local created = http.postJSON("https://api.example.com/items", { name = "widget" }, { |
||||
headers = { ["Authorization"] = "Bearer " .. token }, |
||||
}) |
||||
print("created id:", created.json().id) |
||||
|
||||
-- A login session that persists across runs |
||||
local s = http.session("session.jsonl") -- restore if present |
||||
local home = s:get("https://example.com/dashboard") |
||||
if home.status == 401 then -- session expired; log in again |
||||
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||
home = s:get("https://example.com/dashboard") |
||||
s:save("session.jsonl") |
||||
end |
||||
print(home.ok and "logged in" or "login failed") |
||||
``` |
||||
Loading…
Reference in new issue