commit
6e5a6dcc92
@ -0,0 +1,175 @@ |
|||||||
|
# Concurrency |
||||||
|
|
||||||
|
`a` runs every script on an async core ([tokio](https://tokio.rs/)). The async |
||||||
|
stdlib calls — `os.sleep`, the `http` client, `sqlite` queries — don't block the |
||||||
|
OS thread while they wait: they suspend and let other work run. This page covers |
||||||
|
how that interacts with Lua coroutines and how to run several pieces of work at |
||||||
|
once with `task.join`. |
||||||
|
|
||||||
|
Plain Lua coroutines behave exactly as in stock Lua. The moment a coroutine needs |
||||||
|
to wait on something async — a sleep, a request, a query — run it through |
||||||
|
`task.join` rather than driving it yourself, for the reason in |
||||||
|
[Self-driven coroutines](#self-driven-coroutines). |
||||||
|
|
||||||
|
```lua |
||||||
|
-- Three requests that would take ~3s back-to-back finish in ~1s. |
||||||
|
local a, b, c = task.join( |
||||||
|
function() return http.get("https://httpbingo.org/delay/1").status end, |
||||||
|
function() return http.get("https://httpbingo.org/delay/1").status end, |
||||||
|
function() return http.get("https://httpbingo.org/delay/1").status end |
||||||
|
) |
||||||
|
print(a, b, c) --> 200 200 200 |
||||||
|
``` |
||||||
|
|
||||||
|
## Plain coroutines |
||||||
|
|
||||||
|
Stock Lua coroutines are pure VM machinery and behave normally: |
||||||
|
|
||||||
|
```lua |
||||||
|
local function squares(n) |
||||||
|
for i = 1, n do |
||||||
|
coroutine.yield(i * i) |
||||||
|
end |
||||||
|
end |
||||||
|
|
||||||
|
local gen = coroutine.wrap(squares) |
||||||
|
print(gen(3), gen(), gen()) --> 1 4 9 |
||||||
|
``` |
||||||
|
|
||||||
|
This is ordinary cooperative scheduling: nothing runs concurrently, and the tokio |
||||||
|
runtime is never involved. |
||||||
|
|
||||||
|
## Async calls suspend, they don't block |
||||||
|
|
||||||
|
An async stdlib call — at the top level of your script, or inside a `task.join` |
||||||
|
coroutine — suspends until it is ready without tying up the thread: |
||||||
|
|
||||||
|
```lua |
||||||
|
local t0 = os.microtime() |
||||||
|
os.sleep(0.10) |
||||||
|
print(string.format("waited %.3fs", os.microtime() - t0)) --> waited ~0.100s |
||||||
|
``` |
||||||
|
|
||||||
|
While that sleep is pending, any sibling tasks keep making progress. |
||||||
|
|
||||||
|
## Running work concurrently |
||||||
|
|
||||||
|
### `task.join(fn1, fn2, ...)` |
||||||
|
|
||||||
|
Run each function as its own coroutine, drive them **concurrently** on the |
||||||
|
runtime, and return each one's first result positionally once all have finished. |
||||||
|
If a task raises an error, `task.join` re-raises the first one. |
||||||
|
|
||||||
|
Because async calls suspend instead of blocking, the tasks overlap — the |
||||||
|
wall-clock time is the *longest* task, not the sum: |
||||||
|
|
||||||
|
```lua |
||||||
|
local function worker(name, secs) |
||||||
|
return function() |
||||||
|
os.sleep(secs) -- suspends; siblings run meanwhile |
||||||
|
return name |
||||||
|
end |
||||||
|
end |
||||||
|
|
||||||
|
local t0 = os.microtime() |
||||||
|
local a, b, c = task.join( |
||||||
|
worker("slow", 0.30), |
||||||
|
worker("med", 0.20), |
||||||
|
worker("fast", 0.10) |
||||||
|
) |
||||||
|
print(a, b, c) --> slow med fast |
||||||
|
print(string.format("%.3fs", os.microtime() - t0)) --> ~0.300s, not 0.600s |
||||||
|
``` |
||||||
|
|
||||||
|
### Returning results |
||||||
|
|
||||||
|
Each task's first return value lands in the matching slot, so several results come |
||||||
|
back together: |
||||||
|
|
||||||
|
```lua |
||||||
|
local me, repos = task.join( |
||||||
|
function() return http.getJSON("https://api.github.com/users/torvalds") end, |
||||||
|
function() return http.getJSON("https://api.github.com/users/torvalds/repos") end |
||||||
|
) |
||||||
|
print(me.name, #repos) -- both fetched concurrently |
||||||
|
``` |
||||||
|
|
||||||
|
### Nesting |
||||||
|
|
||||||
|
`task.join` suspends like any other async call, so a task may itself call |
||||||
|
`task.join`: |
||||||
|
|
||||||
|
```lua |
||||||
|
local total = task.join( |
||||||
|
function() |
||||||
|
local x, y = task.join( |
||||||
|
function() os.sleep(0.05); return 21 end, |
||||||
|
function() os.sleep(0.05); return 21 end |
||||||
|
) |
||||||
|
return x + y -- 42 |
||||||
|
end, |
||||||
|
function() os.sleep(0.10); return "sibling" end |
||||||
|
) |
||||||
|
print(total) --> 42 |
||||||
|
``` |
||||||
|
|
||||||
|
## Self-driven coroutines |
||||||
|
|
||||||
|
Async stdlib functions suspend correctly only when the runtime is driving the |
||||||
|
coroutine — at the top level of your script, or inside a `task.join` coroutine. If |
||||||
|
you drive a coroutine **yourself** with `coroutine.resume` or `coroutine.wrap`, an |
||||||
|
async call does *not* wait: |
||||||
|
|
||||||
|
```lua |
||||||
|
-- DON'T do this: |
||||||
|
local co = coroutine.wrap(function() |
||||||
|
os.sleep(0.50) |
||||||
|
return "awoke" |
||||||
|
end) |
||||||
|
|
||||||
|
local v = co() -- returns immediately with an opaque value; the 0.5s wait never |
||||||
|
-- happens, because coroutine.wrap can't drive the runtime. |
||||||
|
``` |
||||||
|
|
||||||
|
Under the hood an async call yields a private marker that only the runtime's |
||||||
|
scheduler understands; a hand-written `resume`/`wrap` loop just receives that |
||||||
|
marker and moves on. |
||||||
|
|
||||||
|
**Rule of thumb:** use plain `coroutine.*` for pure-Lua generators; the moment a |
||||||
|
coroutine needs to `os.sleep`, hit the network, or touch the database, run it |
||||||
|
through `task.join` instead. |
||||||
|
|
||||||
|
## How it works |
||||||
|
|
||||||
|
`task.join` wraps each function in a Lua coroutine and polls them all on the tokio |
||||||
|
runtime. The concurrency comes from the **reactor** — timers and I/O yielding |
||||||
|
control while they wait — not from extra threads. |
||||||
|
|
||||||
|
- **Concurrency, not parallelism.** Everything runs on one thread; there are |
||||||
|
simply many operations in flight at once. |
||||||
|
- **I/O-bound work overlaps.** Sleeps, HTTP requests, and database queries all |
||||||
|
wait at the same time. |
||||||
|
- **CPU-bound work does not yield.** A tight compute loop with no async calls |
||||||
|
blocks its siblings until it finishes or reaches an async call. |
||||||
|
|
||||||
|
## Full example |
||||||
|
|
||||||
|
```lua |
||||||
|
-- Fetch several resources at once, then combine them. |
||||||
|
local function get(url) |
||||||
|
return function() return http.getJSON(url) end |
||||||
|
end |
||||||
|
|
||||||
|
local user, repos = task.join( |
||||||
|
get("https://api.github.com/users/torvalds"), |
||||||
|
get("https://api.github.com/users/torvalds/repos") |
||||||
|
) |
||||||
|
|
||||||
|
print(user.name) |
||||||
|
print(#repos .. " public repos") |
||||||
|
-- Both requests ran concurrently: ~1 round-trip of latency, not 2. |
||||||
|
``` |
||||||
|
|
||||||
|
A runnable version of these patterns ships in |
||||||
|
[`lua/coroutines-demo.lua`](../lua/coroutines-demo.lua) — run it with |
||||||
|
`a lua/coroutines-demo.lua`. |
||||||
@ -0,0 +1,348 @@ |
|||||||
|
# `http` |
||||||
|
|
||||||
|
The `http` module is an HTTP/HTTPS client for calling APIs, scraping pages, and |
||||||
|
submitting forms. TLS is compiled into the binary (rustls with the `ring` |
||||||
|
provider), so HTTPS works with no system library. There are two ways in: plain |
||||||
|
functions for one-off requests, and a **session** object that carries a cookie |
||||||
|
jar across requests for stateful flows like logging in. |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.get("https://example.com") |
||||||
|
print(resp.status, resp.ok) --> 200 true |
||||||
|
print(resp.body) --> "<!doctype html>…" |
||||||
|
|
||||||
|
-- JSON in one step |
||||||
|
local data = http.getJSON("https://api.example.com/users") |
||||||
|
for _, user in ipairs(data) do |
||||||
|
print(user.name) |
||||||
|
end |
||||||
|
``` |
||||||
|
|
||||||
|
## Making requests |
||||||
|
|
||||||
|
Every helper takes a URL and an optional `opts` table (see [Request |
||||||
|
options](#request-options)) and returns a [response](#the-response) table. |
||||||
|
|
||||||
|
```lua |
||||||
|
http.get(url [, opts]) |
||||||
|
http.post(url [, opts]) |
||||||
|
http.put(url [, opts]) |
||||||
|
http.patch(url [, opts]) |
||||||
|
http.delete(url [, opts]) |
||||||
|
http.head(url [, opts]) |
||||||
|
``` |
||||||
|
|
||||||
|
These are thin shorthands over the one primitive: |
||||||
|
|
||||||
|
### `http.request(method, url [, opts])` |
||||||
|
|
||||||
|
Send a request with an explicit method (any verb, e.g. `"GET"`, `"OPTIONS"`) and |
||||||
|
return the response. The shorthands above are just `http.request` with the method |
||||||
|
filled in. |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.request("DELETE", "https://api.example.com/items/42") |
||||||
|
``` |
||||||
|
|
||||||
|
## The response |
||||||
|
|
||||||
|
Every request returns a table describing the response: |
||||||
|
|
||||||
|
| Field | Type | Description | |
||||||
|
|-----------|-----------|-------------------------------------------------------------------| |
||||||
|
| `status` | integer | HTTP status code, e.g. `200`, `404`. | |
||||||
|
| `ok` | boolean | `true` when `status` is in the 2xx range. | |
||||||
|
| `headers` | table | Response headers, keyed by **lowercase** name. | |
||||||
|
| `body` | string | The raw response body (Lua strings are byte sequences). | |
||||||
|
| `json` | function | `resp.json()` parses `body` as JSON. See [JSON](#json). | |
||||||
|
|
||||||
|
Header names are lowercased so you can look them up without guessing the server's |
||||||
|
capitalization. If a header appears more than once, the first value wins. |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.get("https://example.com") |
||||||
|
print(resp.headers["content-type"]) --> "text/html; charset=utf-8" |
||||||
|
``` |
||||||
|
|
||||||
|
A non-2xx status is **not** an error — `resp.status` and `resp.ok` simply report |
||||||
|
it. Only a failure to get a response at all (DNS, connection, TLS, timeout) raises |
||||||
|
a Lua error. See [Errors](#errors). |
||||||
|
|
||||||
|
## Request options |
||||||
|
|
||||||
|
The optional `opts` table accepts these fields, all optional: |
||||||
|
|
||||||
|
| Field | Type | Behaviour | |
||||||
|
|-----------|--------|--------------------------------------------------------------------------| |
||||||
|
| `headers` | table | Extra request headers, `{["X-Foo"] = "bar"}`. | |
||||||
|
| `body` | string | Raw request body; set `Content-Type` yourself via `headers`. | |
||||||
|
| `json` | any | Serialized to JSON; sets `Content-Type: application/json`. | |
||||||
|
| `form` | table | URL-encoded; sets `Content-Type: application/x-www-form-urlencoded`. | |
||||||
|
| `cookies` | table | Cookies for this request, `{session = "abc"}` → `Cookie:` header. | |
||||||
|
| `timeout` | number | Per-request timeout in seconds (default `30`). | |
||||||
|
| `auth` | table | Basic or digest credentials. See [Authentication](#authentication). | |
||||||
|
|
||||||
|
### Bodies |
||||||
|
|
||||||
|
`json`, `form`, and `body` are three ways to set the request body; if more than |
||||||
|
one is given, the first present in that order wins. |
||||||
|
|
||||||
|
```lua |
||||||
|
-- JSON body (table serialized to an object) |
||||||
|
http.post("https://api.example.com/items", { json = { name = "test", count = 5 } }) |
||||||
|
|
||||||
|
-- Form submission (application/x-www-form-urlencoded) |
||||||
|
http.post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||||
|
|
||||||
|
-- Raw body with an explicit content type |
||||||
|
http.post("https://example.com/ingest", { |
||||||
|
body = "id,name\n1,alice\n", |
||||||
|
headers = { ["Content-Type"] = "text/csv" }, |
||||||
|
}) |
||||||
|
``` |
||||||
|
|
||||||
|
`json` serializes the same way as `utils.toJSON`: a table with |
||||||
|
sequential integer keys becomes a JSON array, otherwise an object, and |
||||||
|
`utils.NULL` becomes JSON `null` (a literal Lua `nil` cannot live in |
||||||
|
a table). |
||||||
|
|
||||||
|
### Headers and cookies |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.get("https://example.com", { |
||||||
|
headers = { ["Accept"] = "application/json", ["X-Token"] = "xyz" }, |
||||||
|
cookies = { session = "abc123" }, |
||||||
|
timeout = 10, |
||||||
|
}) |
||||||
|
``` |
||||||
|
|
||||||
|
Per-request `cookies` are sent as a `Cookie` header. With a [session](#sessions), |
||||||
|
they are merged with the jar and take precedence on a name collision. |
||||||
|
|
||||||
|
## JSON |
||||||
|
|
||||||
|
### `resp.json()` |
||||||
|
|
||||||
|
Parse the response body as JSON and return the resulting Lua value. It is the |
||||||
|
counterpart of `utils.fromJSON`: JSON objects become tables, arrays |
||||||
|
become array-tables, and `null` becomes `utils.NULL`. Calling it on a |
||||||
|
body that is not valid JSON raises an error. |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.post("https://api.example.com/echo", { json = { hello = "world" } }) |
||||||
|
print(resp.json().hello) --> "world" |
||||||
|
``` |
||||||
|
|
||||||
|
### `http.getJSON(url [, opts])` |
||||||
|
|
||||||
|
Shorthand for a GET that parses the body. Returns **two** values: the parsed body |
||||||
|
and the full response. |
||||||
|
|
||||||
|
```lua |
||||||
|
local data, resp = http.getJSON("https://api.example.com/users") |
||||||
|
print(resp.status, #data) |
||||||
|
``` |
||||||
|
|
||||||
|
### `http.postJSON(url, body [, opts])` |
||||||
|
|
||||||
|
Shorthand for a POST with a JSON body — equivalent to setting `opts.json = body`. |
||||||
|
Returns the response. |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp = http.postJSON("https://api.example.com/items", { name = "test", count = 5 }) |
||||||
|
if resp.ok then print(resp.json().id) end |
||||||
|
``` |
||||||
|
|
||||||
|
## Authentication |
||||||
|
|
||||||
|
Pass credentials in `opts.auth`. The `scheme` is `"basic"` (the default) or |
||||||
|
`"digest"`. |
||||||
|
|
||||||
|
| Field | Type | Description | |
||||||
|
|------------|--------|----------------------------------------------| |
||||||
|
| `username` | string | Required. | |
||||||
|
| `password` | string | Required. | |
||||||
|
| `scheme` | string | `"basic"` (default) or `"digest"`. | |
||||||
|
|
||||||
|
```lua |
||||||
|
-- HTTP Basic |
||||||
|
local resp = http.get("https://api.example.com/private", { |
||||||
|
auth = { username = "alice", password = "secret" }, |
||||||
|
}) |
||||||
|
|
||||||
|
-- HTTP Digest — the 401 challenge is answered automatically |
||||||
|
local resp = http.get("https://api.example.com/private", { |
||||||
|
auth = { username = "alice", password = "secret", scheme = "digest" }, |
||||||
|
}) |
||||||
|
``` |
||||||
|
|
||||||
|
For basic auth the `Authorization` header is sent with the request. For digest the |
||||||
|
client sends the request, reads the server's `401` challenge, computes the |
||||||
|
response, and retries once; you only see the final response. Auth works the same |
||||||
|
way on [sessions](#sessions). |
||||||
|
|
||||||
|
## Sessions |
||||||
|
|
||||||
|
A **session** wraps its own cookie jar. Cookies from `Set-Cookie` responses are |
||||||
|
stored automatically and sent back on later requests to matching hosts — which is |
||||||
|
what makes login-then-fetch flows work. A session also has the same request |
||||||
|
methods as the plain module. |
||||||
|
|
||||||
|
### `http.session([path])` |
||||||
|
|
||||||
|
Create a session. With a `path`, the cookie jar is preloaded from that file (see |
||||||
|
[Persisting the jar](#persisting-the-jar)). |
||||||
|
|
||||||
|
```lua |
||||||
|
local s = http.session() -- fresh, empty jar |
||||||
|
local s = http.session("cookies.jsonl") -- jar loaded from disk |
||||||
|
``` |
||||||
|
|
||||||
|
### Requests |
||||||
|
|
||||||
|
A session has every method the plain module has — `request`, `get`, `post`, |
||||||
|
`put`, `patch`, `delete`, `head`, `getJSON`, `postJSON` — called with `:` syntax |
||||||
|
and accepting the same `opts`: |
||||||
|
|
||||||
|
```lua |
||||||
|
local s = http.session() |
||||||
|
|
||||||
|
-- Log in; the response's Set-Cookie is captured into the jar |
||||||
|
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||||
|
|
||||||
|
-- The session cookie is sent automatically |
||||||
|
local page = s:get("https://example.com/dashboard") |
||||||
|
``` |
||||||
|
|
||||||
|
### Cookie behaviour |
||||||
|
|
||||||
|
A stored cookie is sent on a request when the request host **equals** the cookie's |
||||||
|
domain or is a subdomain of it. The jar reads the `name=value` pair and the |
||||||
|
`Domain` attribute from each `Set-Cookie` header; other attributes (`Path`, |
||||||
|
`Expires`, `Secure`, `HttpOnly`) are ignored. When no `Domain` is given, the |
||||||
|
request host is used. |
||||||
|
|
||||||
|
`Set-Cookie` headers set on redirect responses are captured too, so a login that |
||||||
|
`302`-redirects to a dashboard still records its cookie. |
||||||
|
|
||||||
|
### Inspecting and clearing |
||||||
|
|
||||||
|
#### `s:cookies()` |
||||||
|
|
||||||
|
Return the jar as a nested table, `{domain = {name = value}}`, for inspection. |
||||||
|
|
||||||
|
```lua |
||||||
|
local jar = s:cookies() |
||||||
|
for domain, names in pairs(jar) do |
||||||
|
for name, value in pairs(names) do |
||||||
|
print(domain, name, value) |
||||||
|
end |
||||||
|
end |
||||||
|
``` |
||||||
|
|
||||||
|
#### `s:clearCookies()` |
||||||
|
|
||||||
|
Empty the in-memory jar. |
||||||
|
|
||||||
|
### Persisting the jar |
||||||
|
|
||||||
|
Cookies live in memory for the session's lifetime. Save them to reuse a logged-in |
||||||
|
session across script runs. |
||||||
|
|
||||||
|
#### `s:save(path)` |
||||||
|
|
||||||
|
Write the jar to `path` as JSONL — one JSON object per line, one cookie per line: |
||||||
|
|
||||||
|
```jsonl |
||||||
|
{"domain":"example.com","name":"session","value":"abc123"} |
||||||
|
{"domain":"api.example.com","name":"token","value":"xyz789"} |
||||||
|
``` |
||||||
|
|
||||||
|
#### `s:load(path)` |
||||||
|
|
||||||
|
Merge cookies from a JSONL file into the current jar (existing cookies are kept, |
||||||
|
matching names overwritten). `http.session(path)` is the same as creating a |
||||||
|
session and calling `:load(path)`. |
||||||
|
|
||||||
|
```lua |
||||||
|
-- First run: log in and persist |
||||||
|
local s = http.session() |
||||||
|
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||||
|
s:save("session.jsonl") |
||||||
|
|
||||||
|
-- Later run: restore and continue without logging in again |
||||||
|
local s = http.session("session.jsonl") |
||||||
|
local page = s:get("https://example.com/dashboard") |
||||||
|
``` |
||||||
|
|
||||||
|
## Redirects |
||||||
|
|
||||||
|
Redirects are followed automatically, up to 10 hops; you receive the final |
||||||
|
response. A `303`, and a `301`/`302` in response to a `POST`, are followed as a |
||||||
|
bodyless `GET`, matching browser behaviour. For a session, cookies set along the |
||||||
|
way are captured at each hop. |
||||||
|
|
||||||
|
## Errors |
||||||
|
|
||||||
|
Failing to obtain a response raises a Lua error: DNS failure, connection refused, |
||||||
|
a TLS problem, or a timeout. An HTTP error *status* (4xx/5xx) does not — it is |
||||||
|
reported through `resp.status`/`resp.ok`. Parsing a non-JSON body with |
||||||
|
`resp.json()` also raises. |
||||||
|
|
||||||
|
Wrap calls in `pcall` or `utils.try` where you want to handle failure |
||||||
|
rather than abort: |
||||||
|
|
||||||
|
```lua |
||||||
|
local resp, err = utils.try(function() |
||||||
|
return http.get("https://does-not-exist.invalid", { timeout = 5 }) |
||||||
|
end) |
||||||
|
if not resp then |
||||||
|
log.error("request failed: " .. tostring(err)) |
||||||
|
end |
||||||
|
``` |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- **HTTPS needs no setup.** The TLS stack (rustls + `ring`) is compiled in; trust |
||||||
|
roots come from the system certificate store. |
||||||
|
- **A default `User-Agent` is sent** (`a/<version>`) because some servers reject |
||||||
|
requests without one. Override it with a `User-Agent` entry in `opts.headers`. |
||||||
|
- **Requests don't block the event loop.** Network I/O runs on the async core, so |
||||||
|
a slow request does not stall other async work (timers, `os.sleep`, SQLite) in |
||||||
|
the same script. |
||||||
|
- **The cookie jar is per session.** Plain `http.get`/`http.post` calls do not |
||||||
|
retain cookies between calls; use a session for that. |
||||||
|
|
||||||
|
## Full example |
||||||
|
|
||||||
|
```lua |
||||||
|
-- Talk to a JSON API with a bearer token, then drive a stateful session. |
||||||
|
|
||||||
|
-- One-off authenticated JSON call |
||||||
|
local items, resp = http.getJSON("https://api.example.com/items", { |
||||||
|
headers = { ["Authorization"] = "Bearer " .. token }, |
||||||
|
timeout = 15, |
||||||
|
}) |
||||||
|
if not resp.ok then |
||||||
|
error("list failed: HTTP " .. resp.status) |
||||||
|
end |
||||||
|
for _, item in ipairs(items) do |
||||||
|
print(item.id, item.name) |
||||||
|
end |
||||||
|
|
||||||
|
-- Create one |
||||||
|
local created = http.postJSON("https://api.example.com/items", { name = "widget" }, { |
||||||
|
headers = { ["Authorization"] = "Bearer " .. token }, |
||||||
|
}) |
||||||
|
print("created id:", created.json().id) |
||||||
|
|
||||||
|
-- A login session that persists across runs |
||||||
|
local s = http.session("session.jsonl") -- restore if present |
||||||
|
local home = s:get("https://example.com/dashboard") |
||||||
|
if home.status == 401 then -- session expired; log in again |
||||||
|
s:post("https://example.com/login", { form = { user = "alice", pass = "secret" } }) |
||||||
|
home = s:get("https://example.com/dashboard") |
||||||
|
s:save("session.jsonl") |
||||||
|
end |
||||||
|
print(home.ok and "logged in" or "login failed") |
||||||
|
``` |
||||||
@ -0,0 +1,99 @@ |
|||||||
|
-- Proof-of-concept: how Lua coroutines interact with the tokio runtime, and |
||||||
|
-- how the async os.sleep behaves in each setting. |
||||||
|
-- |
||||||
|
-- Run with: cargo run -- lua/coroutines-demo.lua |
||||||
|
|
||||||
|
local function banner(s) print("\n=== " .. s .. " ===") end |
||||||
|
|
||||||
|
---------------------------------------------------------------------- |
||||||
|
banner("1. Pure Lua coroutines (no async involved)") |
||||||
|
---------------------------------------------------------------------- |
||||||
|
-- Plain cooperative coroutines work exactly as in stock Lua: this is pure VM |
||||||
|
-- machinery and never touches tokio. |
||||||
|
local function counter(n) |
||||||
|
for i = 1, n do |
||||||
|
coroutine.yield(i * i) |
||||||
|
end |
||||||
|
return "done" |
||||||
|
end |
||||||
|
|
||||||
|
local co = coroutine.wrap(counter) |
||||||
|
print("squares:", co(3), co(3), co(3)) -- 1 4 9 |
||||||
|
|
||||||
|
local raw = coroutine.create(counter) |
||||||
|
print("status fresh:", coroutine.status(raw)) -- suspended |
||||||
|
coroutine.resume(raw, 1) |
||||||
|
coroutine.resume(raw, 1) -- runs to `return` |
||||||
|
print("status after return:", coroutine.status(raw)) -- dead |
||||||
|
|
||||||
|
---------------------------------------------------------------------- |
||||||
|
banner("2. Async os.sleep at the top level") |
||||||
|
---------------------------------------------------------------------- |
||||||
|
-- The main chunk is itself run by mlua's async executor (exec_async), so an |
||||||
|
-- async call here is driven correctly and really suspends on the tokio timer. |
||||||
|
local t0 = os.microtime() |
||||||
|
os.sleep(0.10) |
||||||
|
print(string.format("slept ~0.10s, measured %.3fs", os.microtime() - t0)) |
||||||
|
|
||||||
|
---------------------------------------------------------------------- |
||||||
|
banner("3. GOTCHA: async os.sleep inside a *manually resumed* coroutine") |
||||||
|
---------------------------------------------------------------------- |
||||||
|
-- mlua implements async functions by yielding a private sentinel to whatever is |
||||||
|
-- driving the coroutine. mlua's own executor understands it; a plain |
||||||
|
-- coroutine.resume / coroutine.wrap does NOT. So driving an async call yourself |
||||||
|
-- does not actually wait -- the sleep is not performed by your resume loop. |
||||||
|
local sleeper = coroutine.wrap(function() |
||||||
|
os.sleep(0.50) |
||||||
|
return "awoke" |
||||||
|
end) |
||||||
|
|
||||||
|
local t1 = os.microtime() |
||||||
|
local first = sleeper() -- returns immediately with the sentinel |
||||||
|
print(string.format("first resume returned %s after only %.3fs (did NOT wait 0.5s)", |
||||||
|
tostring(first), os.microtime() - t1)) |
||||||
|
print("--> lesson: don't hand-drive coroutines that call async stdlib functions.") |
||||||
|
|
||||||
|
---------------------------------------------------------------------- |
||||||
|
banner("4. Real concurrency: task.join drives coroutines on tokio") |
||||||
|
---------------------------------------------------------------------- |
||||||
|
-- task.join runs each function as its own coroutine and lets the tokio reactor |
||||||
|
-- interleave them. Three tasks that each sleep concurrently finish in ~the |
||||||
|
-- longest single sleep, not the sum -- proof the sleeps overlap on one thread. |
||||||
|
local function worker(name, secs) |
||||||
|
return function() |
||||||
|
print(string.format(" [%s] start", name)) |
||||||
|
os.sleep(secs) |
||||||
|
print(string.format(" [%s] woke after %.2fs", name, secs)) |
||||||
|
return name .. ":" .. secs |
||||||
|
end |
||||||
|
end |
||||||
|
|
||||||
|
local t2 = os.microtime() |
||||||
|
local a, b, c = task.join( |
||||||
|
worker("slow", 0.30), |
||||||
|
worker("med", 0.20), |
||||||
|
worker("fast", 0.10) |
||||||
|
) |
||||||
|
local elapsed = os.microtime() - t2 |
||||||
|
print(string.format("results: %s, %s, %s", a, b, c)) |
||||||
|
print(string.format("wall time: %.3fs", elapsed)) |
||||||
|
print(string.format("sequential would have been ~0.60s; concurrent ~0.30s => %s", |
||||||
|
elapsed < 0.45 and "CONCURRENT (overlapped on tokio)" or "serialized?!")) |
||||||
|
|
||||||
|
---------------------------------------------------------------------- |
||||||
|
banner("5. Nested + return values") |
||||||
|
---------------------------------------------------------------------- |
||||||
|
-- task.join itself yields, so it composes: a joined task can join again. |
||||||
|
local outer = task.join( |
||||||
|
function() |
||||||
|
local x, y = task.join( |
||||||
|
function() os.sleep(0.05); return 21 end, |
||||||
|
function() os.sleep(0.05); return 21 end |
||||||
|
) |
||||||
|
return x + y |
||||||
|
end, |
||||||
|
function() os.sleep(0.10); return "sibling" end |
||||||
|
) |
||||||
|
print("nested join result:", outer) -- 42 |
||||||
|
|
||||||
|
print("\nAll demos finished.") |
||||||
@ -0,0 +1,66 @@ |
|||||||
|
-- Lua-side of the http module. |
||||||
|
-- http.request (stateless) and http.session (constructor) are provided by Rust |
||||||
|
-- before this runs. This layer adds resp.json(), method shorthands, the |
||||||
|
-- JSON/postJSON helpers, and a metatable for session objects. |
||||||
|
|
||||||
|
-- Attach resp.json() to a response table: parses resp.body via utils.fromJSON. |
||||||
|
local function wrap_resp(resp) |
||||||
|
resp.json = function() return utils.fromJSON(resp.body) end |
||||||
|
return resp |
||||||
|
end |
||||||
|
|
||||||
|
-- Wrap the stateless http.request so responses carry resp.json(). |
||||||
|
local _request = http.request |
||||||
|
http.request = function(method, url, opts) |
||||||
|
return wrap_resp(_request(method, url, opts)) |
||||||
|
end |
||||||
|
|
||||||
|
-- Method shorthands for the stateless module. |
||||||
|
for _, m in ipairs({ "get", "post", "put", "patch", "delete", "head" }) do |
||||||
|
http[m] = function(url, opts) return http.request(m:upper(), url, opts) end |
||||||
|
end |
||||||
|
|
||||||
|
function http.getJSON(url, opts) |
||||||
|
local resp = http.get(url, opts) |
||||||
|
return resp.json(), resp |
||||||
|
end |
||||||
|
|
||||||
|
function http.postJSON(url, body, opts) |
||||||
|
opts = opts or {} |
||||||
|
opts.json = body |
||||||
|
return http.post(url, opts) |
||||||
|
end |
||||||
|
|
||||||
|
-- Session metatable. http.session() returns a raw Rust table whose methods are |
||||||
|
-- _request, save, load, clearCookies and cookies. The metatable adds the |
||||||
|
-- request wrapper (for resp.json()) and the method shorthands on top. |
||||||
|
local session_mt = {} |
||||||
|
session_mt.__index = session_mt |
||||||
|
|
||||||
|
function session_mt:request(method, url, opts) |
||||||
|
return wrap_resp(self:_request(method, url, opts)) |
||||||
|
end |
||||||
|
|
||||||
|
function session_mt:get(url, opts) return self:request("GET", url, opts) end |
||||||
|
function session_mt:post(url, opts) return self:request("POST", url, opts) end |
||||||
|
function session_mt:put(url, opts) return self:request("PUT", url, opts) end |
||||||
|
function session_mt:patch(url, opts) return self:request("PATCH", url, opts) end |
||||||
|
function session_mt:delete(url, opts) return self:request("DELETE", url, opts) end |
||||||
|
function session_mt:head(url, opts) return self:request("HEAD", url, opts) end |
||||||
|
|
||||||
|
function session_mt:getJSON(url, opts) |
||||||
|
local resp = self:get(url, opts) |
||||||
|
return resp.json(), resp |
||||||
|
end |
||||||
|
|
||||||
|
function session_mt:postJSON(url, body, opts) |
||||||
|
opts = opts or {} |
||||||
|
opts.json = body |
||||||
|
return self:post(url, opts) |
||||||
|
end |
||||||
|
|
||||||
|
-- Wrap the Rust session constructor to install the metatable. |
||||||
|
local _session = http.session |
||||||
|
http.session = function(path) |
||||||
|
return setmetatable(_session(path), session_mt) |
||||||
|
end |
||||||
@ -0,0 +1,529 @@ |
|||||||
|
use std::collections::HashMap; |
||||||
|
use std::sync::{Arc, Mutex}; |
||||||
|
use std::time::Duration; |
||||||
|
|
||||||
|
use mlua::prelude::LuaResult; |
||||||
|
use mlua::{Lua, Table as LuaTable, Value as LuaValue}; |
||||||
|
|
||||||
|
use crate::stdlib::utils::lua_to_json; |
||||||
|
|
||||||
|
const HTTP_LUA: &str = include_str!("../../lua/stdlib/http.lua"); |
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Cookie jar
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// A minimal in-memory cookie jar: bare domain (no leading dot) → name → value.
|
||||||
|
/// Deliberately simple — it covers the >95% case of `Set-Cookie` flows without
|
||||||
|
/// pulling in the `cookie_store` crate, and serializes cleanly to JSONL.
|
||||||
|
#[derive(Default)] |
||||||
|
struct CookieJar { |
||||||
|
cookies: HashMap<String, HashMap<String, String>>, |
||||||
|
} |
||||||
|
|
||||||
|
impl CookieJar { |
||||||
|
/// All cookies (name, value) whose stored domain matches `host`: either an
|
||||||
|
/// exact match or `host` being a subdomain of the stored domain.
|
||||||
|
fn cookies_for(&self, host: &str) -> Vec<(String, String)> { |
||||||
|
let mut out = Vec::new(); |
||||||
|
for (domain, names) in &self.cookies { |
||||||
|
let suffix = format!(".{domain}"); |
||||||
|
if host == domain || host.ends_with(&suffix) { |
||||||
|
for (name, value) in names { |
||||||
|
out.push((name.clone(), value.clone())); |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
out |
||||||
|
} |
||||||
|
|
||||||
|
/// Parse a single `Set-Cookie` header value and store the cookie. Extracts
|
||||||
|
/// `name=value` (first segment) and an optional `Domain=` attribute, falling
|
||||||
|
/// back to the request host. Malformed headers are ignored.
|
||||||
|
fn set_from_header(&mut self, host: &str, header: &str) { |
||||||
|
let mut segments = header.split(';'); |
||||||
|
let first = match segments.next() { |
||||||
|
Some(s) => s.trim(), |
||||||
|
None => return, |
||||||
|
}; |
||||||
|
let (name, value) = match first.split_once('=') { |
||||||
|
Some((n, v)) => (n.trim(), v.trim()), |
||||||
|
None => return, |
||||||
|
}; |
||||||
|
if name.is_empty() { |
||||||
|
return; |
||||||
|
} |
||||||
|
|
||||||
|
let mut domain = host.to_ascii_lowercase(); |
||||||
|
for seg in segments { |
||||||
|
if let Some((k, v)) = seg.split_once('=') |
||||||
|
&& k.trim().eq_ignore_ascii_case("domain") |
||||||
|
{ |
||||||
|
let d = v.trim().trim_start_matches('.').to_ascii_lowercase(); |
||||||
|
if !d.is_empty() { |
||||||
|
domain = d; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
self.cookies |
||||||
|
.entry(domain) |
||||||
|
.or_default() |
||||||
|
.insert(name.to_string(), value.to_string()); |
||||||
|
} |
||||||
|
|
||||||
|
/// Serialize as JSONL — one `{"domain","name","value"}` object per line.
|
||||||
|
fn to_jsonl(&self) -> String { |
||||||
|
let mut out = String::new(); |
||||||
|
for (domain, names) in &self.cookies { |
||||||
|
for (name, value) in names { |
||||||
|
let obj = serde_json::json!({ |
||||||
|
"domain": domain, |
||||||
|
"name": name, |
||||||
|
"value": value, |
||||||
|
}); |
||||||
|
out.push_str(&obj.to_string()); |
||||||
|
out.push('\n'); |
||||||
|
} |
||||||
|
} |
||||||
|
out |
||||||
|
} |
||||||
|
|
||||||
|
/// Merge cookies from JSONL produced by `to_jsonl`. Bad lines are skipped.
|
||||||
|
fn merge_jsonl(&mut self, src: &str) { |
||||||
|
for line in src.lines() { |
||||||
|
let line = line.trim(); |
||||||
|
if line.is_empty() { |
||||||
|
continue; |
||||||
|
} |
||||||
|
let Ok(v) = serde_json::from_str::<serde_json::Value>(line) else { |
||||||
|
continue; |
||||||
|
}; |
||||||
|
let domain = v.get("domain").and_then(|x| x.as_str()); |
||||||
|
let name = v.get("name").and_then(|x| x.as_str()); |
||||||
|
let value = v.get("value").and_then(|x| x.as_str()); |
||||||
|
if let (Some(d), Some(n), Some(val)) = (domain, name, value) { |
||||||
|
self.cookies |
||||||
|
.entry(d.to_string()) |
||||||
|
.or_default() |
||||||
|
.insert(n.to_string(), val.to_string()); |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Request execution
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// `application/x-www-form-urlencoded` body from key/value pairs. Implemented
|
||||||
|
/// inline so the build needs no optional reqwest features.
|
||||||
|
fn form_urlencode(pairs: &[(String, String)]) -> String { |
||||||
|
fn encode(s: &str) -> String { |
||||||
|
let mut out = String::new(); |
||||||
|
for b in s.bytes() { |
||||||
|
match b { |
||||||
|
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => { |
||||||
|
out.push(b as char) |
||||||
|
} |
||||||
|
b' ' => out.push('+'), |
||||||
|
_ => out.push_str(&format!("%{b:02X}")), |
||||||
|
} |
||||||
|
} |
||||||
|
out |
||||||
|
} |
||||||
|
pairs |
||||||
|
.iter() |
||||||
|
.map(|(k, v)| format!("{}={}", encode(k), encode(v))) |
||||||
|
.collect::<Vec<_>>() |
||||||
|
.join("&") |
||||||
|
} |
||||||
|
|
||||||
|
const MAX_REDIRECTS: u32 = 10; |
||||||
|
|
||||||
|
/// Shared by the stateless `http.request` and a session's `:request`. When `jar`
|
||||||
|
/// is `Some`, the matching jar cookies are sent and any `Set-Cookie` responses
|
||||||
|
/// are stored back.
|
||||||
|
///
|
||||||
|
/// Redirects are followed manually (the client is built with
|
||||||
|
/// `redirect::Policy::none`) so that `Set-Cookie` headers on 30x responses —
|
||||||
|
/// the common login → redirect → dashboard pattern — are captured into the jar,
|
||||||
|
/// which reqwest's transparent redirect following would otherwise hide.
|
||||||
|
async fn execute_request( |
||||||
|
lua: Lua, |
||||||
|
client: reqwest::Client, |
||||||
|
jar: Option<Arc<Mutex<CookieJar>>>, |
||||||
|
method: String, |
||||||
|
url: String, |
||||||
|
opts: Option<LuaTable>, |
||||||
|
) -> LuaResult<LuaTable> { |
||||||
|
let mut method = reqwest::Method::from_bytes(method.to_ascii_uppercase().as_bytes()) |
||||||
|
.map_err(|e| mlua::Error::external(format!("http: invalid method '{method}': {e}")))?; |
||||||
|
|
||||||
|
// Parse opts once into owned pieces so each redirect hop can rebuild the
|
||||||
|
// request (a reqwest RequestBuilder is single-use).
|
||||||
|
let mut timeout: Option<Duration> = None; |
||||||
|
let mut custom_headers: Vec<(String, String)> = Vec::new(); |
||||||
|
let mut opts_cookies: Vec<(String, String)> = Vec::new(); |
||||||
|
// Body, with the Content-Type it implies (None for a raw body).
|
||||||
|
let mut body: Option<(Vec<u8>, Option<&'static str>)> = None; |
||||||
|
// Auth: (is_digest, username, password).
|
||||||
|
let mut auth: Option<(bool, String, String)> = None; |
||||||
|
|
||||||
|
if let Some(opts) = opts.as_ref() { |
||||||
|
if let Some(t) = opts.get::<Option<f64>>("timeout")? { |
||||||
|
timeout = Some(Duration::try_from_secs_f64(t).map_err(|_| { |
||||||
|
mlua::Error::external( |
||||||
|
"http: timeout must be a non-negative finite number of seconds", |
||||||
|
) |
||||||
|
})?); |
||||||
|
} |
||||||
|
if let Some(headers) = opts.get::<Option<LuaTable>>("headers")? { |
||||||
|
for pair in headers.pairs::<String, String>() { |
||||||
|
custom_headers.push(pair?); |
||||||
|
} |
||||||
|
} |
||||||
|
if let Some(cookies) = opts.get::<Option<LuaTable>>("cookies")? { |
||||||
|
for pair in cookies.pairs::<String, String>() { |
||||||
|
opts_cookies.push(pair?); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Body: json > form > raw body (first one present wins).
|
||||||
|
if let Some(json_val) = opts.get::<Option<LuaValue>>("json")? { |
||||||
|
let json = lua_to_json(json_val, 0)?; |
||||||
|
let s = serde_json::to_string(&json).map_err(mlua::Error::external)?; |
||||||
|
body = Some((s.into_bytes(), Some("application/json"))); |
||||||
|
} else if let Some(form) = opts.get::<Option<LuaTable>>("form")? { |
||||||
|
let mut pairs = Vec::new(); |
||||||
|
for pair in form.pairs::<String, String>() { |
||||||
|
pairs.push(pair?); |
||||||
|
} |
||||||
|
body = Some(( |
||||||
|
form_urlencode(&pairs).into_bytes(), |
||||||
|
Some("application/x-www-form-urlencoded"), |
||||||
|
)); |
||||||
|
} else if let Some(b) = opts.get::<Option<mlua::String>>("body")? { |
||||||
|
body = Some((b.as_bytes().to_vec(), None)); |
||||||
|
} |
||||||
|
|
||||||
|
// Auth: { username, password, scheme = "basic" (default) | "digest" }.
|
||||||
|
if let Some(auth_tbl) = opts.get::<Option<LuaTable>>("auth")? { |
||||||
|
let username = auth_tbl |
||||||
|
.get::<Option<String>>("username")? |
||||||
|
.ok_or_else(|| mlua::Error::external("http: auth.username is required"))?; |
||||||
|
let password = auth_tbl |
||||||
|
.get::<Option<String>>("password")? |
||||||
|
.ok_or_else(|| mlua::Error::external("http: auth.password is required"))?; |
||||||
|
let is_digest = match auth_tbl.get::<Option<String>>("scheme")?.as_deref() { |
||||||
|
None | Some("basic") => false, |
||||||
|
Some("digest") => true, |
||||||
|
Some(other) => { |
||||||
|
return Err(mlua::Error::external(format!( |
||||||
|
"http: auth.scheme must be \"basic\" or \"digest\", got \"{other}\"" |
||||||
|
))); |
||||||
|
} |
||||||
|
}; |
||||||
|
auth = Some((is_digest, username, password)); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
let is_digest = matches!(auth.as_ref(), Some((true, _, _))); |
||||||
|
let mut digest_header: Option<String> = None; |
||||||
|
let mut digest_tried = false; |
||||||
|
|
||||||
|
let mut url = url; |
||||||
|
let mut redirects_left = MAX_REDIRECTS; |
||||||
|
|
||||||
|
let resp = loop { |
||||||
|
// Host used for cookie matching and as the Set-Cookie domain fallback;
|
||||||
|
// recomputed each hop since a redirect may cross hosts.
|
||||||
|
let host = reqwest::Url::parse(&url) |
||||||
|
.ok() |
||||||
|
.and_then(|u| u.host_str().map(|h| h.to_ascii_lowercase())); |
||||||
|
|
||||||
|
let mut req = client.request(method.clone(), &url); |
||||||
|
|
||||||
|
if let Some(t) = timeout { |
||||||
|
req = req.timeout(t); |
||||||
|
} |
||||||
|
for (k, v) in &custom_headers { |
||||||
|
req = req.header(k, v); |
||||||
|
} |
||||||
|
|
||||||
|
// Cookie header: jar cookies for this host, then per-request cookies
|
||||||
|
// which override on name collision.
|
||||||
|
let mut cookie_map: HashMap<String, String> = HashMap::new(); |
||||||
|
if let (Some(jar), Some(host)) = (jar.as_ref(), host.as_ref()) { |
||||||
|
for (n, v) in jar.lock().unwrap().cookies_for(host) { |
||||||
|
cookie_map.insert(n, v); |
||||||
|
} |
||||||
|
} |
||||||
|
for (k, v) in &opts_cookies { |
||||||
|
cookie_map.insert(k.clone(), v.clone()); |
||||||
|
} |
||||||
|
if !cookie_map.is_empty() { |
||||||
|
let header = cookie_map |
||||||
|
.iter() |
||||||
|
.map(|(k, v)| format!("{k}={v}")) |
||||||
|
.collect::<Vec<_>>() |
||||||
|
.join("; "); |
||||||
|
req = req.header(reqwest::header::COOKIE, header); |
||||||
|
} |
||||||
|
|
||||||
|
if let Some((bytes, ct)) = body.as_ref() { |
||||||
|
if let Some(ct) = ct { |
||||||
|
req = req.header(reqwest::header::CONTENT_TYPE, *ct); |
||||||
|
} |
||||||
|
req = req.body(bytes.clone()); |
||||||
|
} |
||||||
|
|
||||||
|
// Auth. Basic goes out on every hop; digest's Authorization is set only
|
||||||
|
// after the 401 challenge below has been answered (digest_header).
|
||||||
|
if let Some((digest, username, password)) = auth.as_ref() { |
||||||
|
if *digest { |
||||||
|
if let Some(h) = digest_header.as_ref() { |
||||||
|
req = req.header(reqwest::header::AUTHORIZATION, h); |
||||||
|
} |
||||||
|
} else { |
||||||
|
req = req.basic_auth(username, Some(password)); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
let resp = req |
||||||
|
.send() |
||||||
|
.await |
||||||
|
.map_err(|e| mlua::Error::external(format!("http: {e}")))?; |
||||||
|
let status = resp.status(); |
||||||
|
|
||||||
|
// Store Set-Cookie from this hop into the jar.
|
||||||
|
if let (Some(jar), Some(host)) = (jar.as_ref(), host.as_ref()) { |
||||||
|
let mut j = jar.lock().unwrap(); |
||||||
|
for value in resp.headers().get_all(reqwest::header::SET_COOKIE).iter() { |
||||||
|
if let Ok(s) = value.to_str() { |
||||||
|
j.set_from_header(host, s); |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Digest auth: answer a 401 challenge once, then retry the same request
|
||||||
|
// with the computed Authorization header.
|
||||||
|
if is_digest && !digest_tried && status == reqwest::StatusCode::UNAUTHORIZED { |
||||||
|
let challenge = resp |
||||||
|
.headers() |
||||||
|
.get(reqwest::header::WWW_AUTHENTICATE) |
||||||
|
.and_then(|v| v.to_str().ok()) |
||||||
|
.map(str::to_string); |
||||||
|
if let Some((_, username, password)) = auth.as_ref() |
||||||
|
&& let Some(challenge) = challenge |
||||||
|
&& let Ok(mut prompt) = digest_auth::parse(&challenge) |
||||||
|
{ |
||||||
|
let uri = reqwest::Url::parse(&url) |
||||||
|
.map(|u| match u.query() { |
||||||
|
Some(q) => format!("{}?{}", u.path(), q), |
||||||
|
None => u.path().to_string(), |
||||||
|
}) |
||||||
|
.unwrap_or_else(|_| url.clone()); |
||||||
|
let ctx = digest_auth::AuthContext::new_with_method( |
||||||
|
username.as_str(), |
||||||
|
password.as_str(), |
||||||
|
uri, |
||||||
|
body.as_ref().map(|(b, _)| b.as_slice()), |
||||||
|
digest_auth::HttpMethod::from(method.as_str()), |
||||||
|
); |
||||||
|
if let Ok(answer) = prompt.respond(&ctx) { |
||||||
|
digest_header = Some(answer.to_header_string()); |
||||||
|
digest_tried = true; |
||||||
|
continue; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Follow a redirect if there is one to follow.
|
||||||
|
if status.is_redirection() |
||||||
|
&& redirects_left > 0 |
||||||
|
&& let Some(next) = resp |
||||||
|
.headers() |
||||||
|
.get(reqwest::header::LOCATION) |
||||||
|
.and_then(|l| l.to_str().ok()) |
||||||
|
.and_then(|loc| reqwest::Url::parse(&url).and_then(|base| base.join(loc)).ok()) |
||||||
|
{ |
||||||
|
redirects_left -= 1; |
||||||
|
// 303, and 301/302 on a POST, degrade to a bodyless GET — the
|
||||||
|
// behaviour browsers and reqwest's own redirect policy apply.
|
||||||
|
let code = status.as_u16(); |
||||||
|
if code == 303 || ((code == 301 || code == 302) && method == reqwest::Method::POST) { |
||||||
|
method = reqwest::Method::GET; |
||||||
|
body = None; |
||||||
|
} |
||||||
|
url = next.to_string(); |
||||||
|
continue; |
||||||
|
} |
||||||
|
|
||||||
|
break resp; |
||||||
|
}; |
||||||
|
|
||||||
|
let status = resp.status().as_u16(); |
||||||
|
|
||||||
|
// Response headers: lowercase names, first value wins.
|
||||||
|
let headers_tbl = lua.create_table()?; |
||||||
|
for (name, value) in resp.headers().iter() { |
||||||
|
let lname = name.as_str().to_ascii_lowercase(); |
||||||
|
if !headers_tbl.contains_key(lname.as_str())? { |
||||||
|
headers_tbl.raw_set(lname.as_str(), lua.create_string(value.as_bytes())?)?; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
let bytes = resp |
||||||
|
.bytes() |
||||||
|
.await |
||||||
|
.map_err(|e| mlua::Error::external(format!("http: {e}")))?; |
||||||
|
|
||||||
|
let out = lua.create_table()?; |
||||||
|
out.raw_set("status", status)?; |
||||||
|
out.raw_set("ok", (200..=299).contains(&status))?; |
||||||
|
out.raw_set("headers", headers_tbl)?; |
||||||
|
out.raw_set("body", lua.create_string(&bytes)?)?; |
||||||
|
Ok(out) |
||||||
|
} |
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Session
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/// Build the plain Lua table that backs a session. State (the cookie jar) lives
|
||||||
|
/// in the closures; the Lua layer applies a metatable for the method shorthands.
|
||||||
|
/// Methods are called as `s:method(...)`, so each closure receives the session
|
||||||
|
/// table as a leading `_this` argument that it ignores.
|
||||||
|
fn make_session( |
||||||
|
lua: &Lua, |
||||||
|
client: reqwest::Client, |
||||||
|
jar: Arc<Mutex<CookieJar>>, |
||||||
|
) -> LuaResult<LuaTable> { |
||||||
|
let tbl = lua.create_table()?; |
||||||
|
|
||||||
|
tbl.raw_set( |
||||||
|
"_request", |
||||||
|
lua.create_async_function({ |
||||||
|
let client = client.clone(); |
||||||
|
let jar = jar.clone(); |
||||||
|
move |lua, (_this, method, url, opts): (LuaTable, String, String, Option<LuaTable>)| { |
||||||
|
let client = client.clone(); |
||||||
|
let jar = jar.clone(); |
||||||
|
async move { |
||||||
|
execute_request(lua, client, Some(jar), method, url, opts).await |
||||||
|
} |
||||||
|
} |
||||||
|
})?, |
||||||
|
)?; |
||||||
|
|
||||||
|
tbl.raw_set("save", { |
||||||
|
let jar = jar.clone(); |
||||||
|
lua.create_function(move |_, (_this, path): (LuaTable, String)| { |
||||||
|
let jsonl = jar.lock().unwrap().to_jsonl(); |
||||||
|
std::fs::write(&path, jsonl) |
||||||
|
.map_err(|e| mlua::Error::external(format!("session:save: {e}"))) |
||||||
|
})? |
||||||
|
})?; |
||||||
|
|
||||||
|
tbl.raw_set("load", { |
||||||
|
let jar = jar.clone(); |
||||||
|
lua.create_function(move |_, (_this, path): (LuaTable, String)| { |
||||||
|
let src = std::fs::read_to_string(&path) |
||||||
|
.map_err(|e| mlua::Error::external(format!("session:load: {e}")))?; |
||||||
|
jar.lock().unwrap().merge_jsonl(&src); |
||||||
|
Ok(()) |
||||||
|
})? |
||||||
|
})?; |
||||||
|
|
||||||
|
tbl.raw_set("clearCookies", { |
||||||
|
let jar = jar.clone(); |
||||||
|
lua.create_function(move |_, _this: LuaTable| { |
||||||
|
jar.lock().unwrap().cookies.clear(); |
||||||
|
Ok(()) |
||||||
|
})? |
||||||
|
})?; |
||||||
|
|
||||||
|
tbl.raw_set("cookies", { |
||||||
|
let jar = jar.clone(); |
||||||
|
lua.create_function(move |lua, _this: LuaTable| { |
||||||
|
let outer = lua.create_table()?; |
||||||
|
let j = jar.lock().unwrap(); |
||||||
|
for (domain, names) in j.cookies.iter() { |
||||||
|
let inner = lua.create_table()?; |
||||||
|
for (name, value) in names.iter() { |
||||||
|
inner.raw_set(name.as_str(), value.as_str())?; |
||||||
|
} |
||||||
|
outer.raw_set(domain.as_str(), inner)?; |
||||||
|
} |
||||||
|
Ok(outer) |
||||||
|
})? |
||||||
|
})?; |
||||||
|
|
||||||
|
Ok(tbl) |
||||||
|
} |
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Installation
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
pub(super) fn install(lua: &Lua) -> LuaResult<()> { |
||||||
|
// reqwest is built with `rustls-no-provider`, so it has no crypto provider
|
||||||
|
// of its own and panics ("No provider set") unless one is installed as the
|
||||||
|
// process default before the client is built. `ring` is self-contained
|
||||||
|
// (compiled in, no system OpenSSL). Idempotent across Lua states — only the
|
||||||
|
// first install in the process wins, and they would all install ring.
|
||||||
|
let _ = rustls::crypto::ring::default_provider().install_default(); |
||||||
|
|
||||||
|
let client = reqwest::Client::builder() |
||||||
|
.timeout(Duration::from_secs(30)) |
||||||
|
// reqwest sends no User-Agent by default; some edges/CDNs reject
|
||||||
|
// empty-UA requests outright. A per-request `headers` entry overrides it.
|
||||||
|
.user_agent(concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION"))) |
||||||
|
// Redirects are followed manually in execute_request so Set-Cookie
|
||||||
|
// headers on 30x responses can be captured into the cookie jar.
|
||||||
|
.redirect(reqwest::redirect::Policy::none()) |
||||||
|
.build() |
||||||
|
.map_err(mlua::Error::external)?; |
||||||
|
|
||||||
|
let http = lua.create_table()?; |
||||||
|
|
||||||
|
// Stateless http.request — no cookie jar.
|
||||||
|
http.raw_set( |
||||||
|
"request", |
||||||
|
lua.create_async_function({ |
||||||
|
let client = client.clone(); |
||||||
|
move |lua, (method, url, opts): (String, String, Option<LuaTable>)| { |
||||||
|
let client = client.clone(); |
||||||
|
async move { execute_request(lua, client, None, method, url, opts).await } |
||||||
|
} |
||||||
|
})?, |
||||||
|
)?; |
||||||
|
|
||||||
|
// http.session(path?) — constructor returning the raw session table.
|
||||||
|
http.raw_set( |
||||||
|
"session", |
||||||
|
lua.create_async_function({ |
||||||
|
let client = client.clone(); |
||||||
|
move |lua, path: Option<String>| { |
||||||
|
let client = client.clone(); |
||||||
|
async move { |
||||||
|
let mut jar = CookieJar::default(); |
||||||
|
if let Some(p) = path { |
||||||
|
let src = tokio::fs::read_to_string(&p) |
||||||
|
.await |
||||||
|
.map_err(|e| mlua::Error::external(format!("http.session: {e}")))?; |
||||||
|
jar.merge_jsonl(&src); |
||||||
|
} |
||||||
|
make_session(&lua, client, Arc::new(Mutex::new(jar))) |
||||||
|
} |
||||||
|
} |
||||||
|
})?, |
||||||
|
)?; |
||||||
|
|
||||||
|
lua.globals().raw_set("http", http)?; |
||||||
|
|
||||||
|
// Lua side adds: resp.json(), method shorthands, getJSON/postJSON, and the
|
||||||
|
// session metatable wrapping http.session().
|
||||||
|
lua.load(HTTP_LUA).set_name("@[stdlib/http]").exec() |
||||||
|
} |
||||||
@ -0,0 +1,46 @@ |
|||||||
|
use futures::future::join_all; |
||||||
|
use mlua::prelude::{LuaResult, LuaValue}; |
||||||
|
use mlua::{Function, Lua, MultiValue, Variadic}; |
||||||
|
|
||||||
|
/// Proof-of-concept concurrency primitive built on Lua coroutines + tokio.
|
||||||
|
///
|
||||||
|
/// `task.join(f1, f2, ...)` runs each function as its own Lua coroutine and
|
||||||
|
/// drives them *concurrently* on the tokio runtime, returning each one's first
|
||||||
|
/// result positionally once all have finished. Because async stdlib calls like
|
||||||
|
/// `os.sleep` yield to the tokio reactor rather than blocking the OS thread,
|
||||||
|
/// sibling coroutines make progress while one is sleeping — so N tasks that each
|
||||||
|
/// sleep T seconds finish in ~T, not ~N*T.
|
||||||
|
///
|
||||||
|
/// Note: the coroutines are NOT `tokio::spawn`ed (the Lua state is `!Send`);
|
||||||
|
/// they are polled cooperatively on the current thread via `join_all`. The
|
||||||
|
/// concurrency comes from the reactor, not from extra threads.
|
||||||
|
pub(super) fn install(lua: &Lua) -> LuaResult<()> { |
||||||
|
let task = lua.create_table()?; |
||||||
|
|
||||||
|
task.raw_set( |
||||||
|
"join", |
||||||
|
lua.create_async_function(|lua, funcs: Variadic<Function>| async move { |
||||||
|
// Wrap each function in its own coroutine and turn it into a future.
|
||||||
|
let mut threads = Vec::with_capacity(funcs.len()); |
||||||
|
for f in funcs.iter() { |
||||||
|
let thread = lua.create_thread(f.clone())?; |
||||||
|
threads.push(thread.into_async::<LuaValue>(())); |
||||||
|
} |
||||||
|
let threads: Vec<_> = threads.into_iter().collect::<LuaResult<_>>()?; |
||||||
|
|
||||||
|
// Poll them all concurrently. The await point is where the tokio
|
||||||
|
// reactor gets to interleave the sleeping coroutines.
|
||||||
|
let results = join_all(threads).await; |
||||||
|
|
||||||
|
// Collect first-return-values positionally; propagate the first error.
|
||||||
|
let mut out = Vec::with_capacity(results.len()); |
||||||
|
for r in results { |
||||||
|
out.push(r?); |
||||||
|
} |
||||||
|
Ok(MultiValue::from_vec(out)) |
||||||
|
})?, |
||||||
|
)?; |
||||||
|
|
||||||
|
lua.globals().raw_set("task", task)?; |
||||||
|
Ok(()) |
||||||
|
} |
||||||
Loading…
Reference in new issue