velt/net/http#

fetch, Request, Response and Headers are global, as in Node: no import. They follow the WHATWG Fetch standard, which TypeScript's lib.dom.d.ts types. fetch speaks HTTP/2 to https: origins that offer it and HTTP/1.1 otherwise; the API is the same either way.

The client is written in Velt on velt/net: HTTP/2 for HTTPS origins that offer it, else HTTP/1.1 with keep-alive connections pooled per origin, both shared by every task, TLS from the runtime for HTTPS, bodies decoded by velt/compress, and redirects followed as the standard says. A pooled connection idle for more than a second is checked before it is reused, and one idle for a minute is closed. A request sent on a reused pooled connection that the server had closed (it fails before any response) is sent once more, on another connection, when its method is idempotent (GET, HEAD, OPTIONS, TRACE, PUT, DELETE), as Go's client does; POST and PATCH are never sent twice, and neither is a request on a fresh connection (that server got the request and failed). Node fails such a request. A server's handlers (serve in the HTTP server package) take and return the same Request and Response, as in Deno and Bun. The option types of their signatures (RequestInit, RequestRedirect, ResponseInit, HeadersInit, BodyInit) are global too, as in TypeScript, so an init object can be built before the call: const init: RequestInit = { method: "POST" }; if (body != null) { init.body = body; }. velt/net/http exports the same names and BodyStream, a Velt body stream of u8[] chunks.

type User = { id: number; name: string };

async function users(base: string): Promise<User[]> {
  const res = await fetch(`${base}/users`, { headers: { accept: "application/json" } });
  if (!res.ok) {
    throw new Error(`GET /users: ${res.status} ${res.statusText}`);
  }
  return await res.json<User[]>();
}

fetch(input, init?)#

fetch(input: string | URL | Request, init: RequestInit = {}): Promise<Response, IoError | AbortError> sends a request and resolves once the response's status and headers have arrived; the body is received when you read it. An HTTP error status (404, 500) is a response, not an error: check res.ok. fetch rejects when there is no response:

• IoError (from velt/io) when the request fails: code is "ECONNREFUSED", "ETIMEDOUT" (connecting takes more than 10 s), "ECONNRESET", … and the message starts with fetch failed, as Node's does. An invalid URL, header name or value, a GET or HEAD request with a body, and a scheme other than http: / https: fail with "EINVAL" / "ENOTSUP" before anything is sent. A redirect loop (more than 20) or a redirect with redirect: "error" fails with fetch failed: ….
• The signal's AbortError (from velt/task) when init.signal is aborted: a TimeoutError, which extends AbortError, for AbortSignal.timeout(ms). Aborting stops the request where it waits (connecting, sending, waiting for the response or receiving its body) and closes its connection.

RequestInit has the standard's fields that make sense outside a browser:

Requests send accept: */*, user-agent: velt and, like Node, accept-encoding: gzip, deflate (br, gzip, deflate over HTTPS, chosen again for each redirect hop) unless you set them. Following a redirect works as in browsers and Node: only 301, 302, 303, 307 and 308 are followed (a 300 or 304 is returned as it is), a 303 (and a 301 or 302 after a POST) becomes a GET without a body, and authorization, cookie and a host you set are not sent to another origin.

The body's framing is the client's own, as in Node: content-length is written from the body (one you set must agree with it, else IoError "EINVAL" Request body length does not match content-length header), and setting transfer-encoding fails with IoError "EINVAL" (invalid transfer-encoding header), so a header cannot make the server read another body than the one sent. Responses are read as strictly as Node reads them: a status below 100, a header name that is not a token, a control character in a header value or the reason phrase, both transfer-encoding and content-length, or a chunked trailer larger than a head may be (400 KiB) fail with IoError "EPROTO". A connection taken from the pool is checked first, so a request does not go out on one whose close has arrived. A request that may not be sent twice (a POST) takes only one idle for at least 5 ms, by when a close the server sent shortly after its last response (at once, on its next turn or a few milliseconds later) has usually arrived; it gets a new connection otherwise. This narrows the race every HTTP/1.1 client has, but does not rule it out: a server that closes an idle connection later than that, or one slowed by load, can still close it just as a POST goes out, and the POST then fails with IoError "ECONNRESET" (other side closed) after it reached the server. fetch never sends a POST again, as Node does not (Node waits two event-loop turns before it reuses a connection, and has the same residual race): retry a POST yourself only if it is idempotent for your server.

HTTPS trusts Mozilla's root certificates (compiled in, so the result does not depend on the machine) plus ca. An untrusted certificate fails with IoError.

Response#

• status: number, ok: bool (200–299), statusText (the server's reason phrase, else the standard one), headers: Headers, url (the final URL, after redirects), redirected, type ("basic" for a fetched response, "default" for one you made, "error"), bodyUsed.
• Reading the body: await res.text() (invalid UTF-8 becomes U+FFFD and a leading byte order mark is dropped, as in JS), await res.json<T>(), await res.bytes(): Uint8Array and await res.arrayBuffer(): ArrayBuffer (typed arrays). A body is read once: a second read throws IoError (Body is unusable: Body has already been read); a response without a body can be read any number of times, as in JS. A large body is received into one buffer sized from content-length, so bytes() copies it once.
• json<T>() decodes the body as T and checks it (JsonError names what doesn't match), so there is no separate schema step. Velt has no any: name the type (res.json<User[]>()) or give the variable one (const users: User[] = await res.json()); an untyped res.json() is a compile error that says so.
• res.body is the body as it arrives, a ReadableStream of Uint8Arrays: res.body.getReader() or for await (const chunk of res.body), so a large download need not be held in memory: each chunk is at most 64 KiB, however much a compressed body expands. It is null for a status without a body (101, 103, 204, 205, 304), for a HEAD request's response and for a response made without one. Reading it uses the body up, as text() does; getting it does not. A ReadableStream of Uint8Array chunks is a body too, as in Node: new Response(ReadableStream.from(gen())) (a chunk of another type fails the read with IoError "EINVAL", Received non-Uint8Array chunk), and so is a BodyStream.
• A body the server compressed (content-encoding: gzip, deflate or br, or several of them, such as deflate, gzip: up to five) is decoded as it arrives, whichever way you read it; the headers stay as received. A list naming another coding leaves the body as sent, and a response without a body (a HEAD request's, a 204 or 304) has nothing to decode; an empty body is "" in any coding, as in Node. A body that does not decode fails the read with IoError (fetch failed: invalid compressed body: …).
• The status text, URL and headers stay readable after the body was read; the headers are kept as received and made into strings the first time you read them, so a response whose headers you never look at costs nothing for them. A small body that arrives with the head gives its connection back to the pool at once, whether or not you read it. headers is read-only, as in JS (a getter): change the Headers it returns, not the property. Dropping a response whose body was not read closes its connection.
• new Response(body?: string | Uint8Array | ArrayBuffer | DataView | u8[] | URLSearchParams | BodyStream | null, init?: ResponseInit { status?; statusText?; headers? }), Response.json(data, init?) (content-type: application/json), Response.error() and Response.redirect(url, status = 302) build responses: what a server handler returns, or a stand-in in tests. A string body implies content-type: text/plain;charset=UTF-8 and form fields application/x-www-form-urlencoded;charset=UTF-8, unless headers set one; a constructed response makes no Headers until you read headers. A Blob body implies its type, and a FormData body multipart/form-data; boundary=….
• await res.blob() and await res.formData() read the body as a Blob (its type the content-type) or a FormData (a multipart/form-data or application/x-www-form-urlencoded body; any other fails with IoError "EINVAL").
• res.clone() is a copy whose body can be read on its own, also at the same time as the original's: a streamed or fetched body is teed, each chunk going to both, and a chunk one of them has not read waits in memory for it, as in Node. Cloning a response whose body was read throws IoError "EINVAL" (Response.clone: Body has already been consumed.).

Request#

new Request(input: string | URL | Request, init?: RequestInit) holds what fetch takes: url, method, headers, redirect, signal (the init.signal, else one that is never aborted, as in JS), bodyUsed, the body readers text(), json<T>(), bytes(), arrayBuffer(), blob(), formData() and body (a ReadableStream of Uint8Arrays, null without a body), and clone() (as Response.clone(); a received request's body is teed too). fetch(request) sends it (its body counts as read afterwards); fetch(request, init) overrides fields of it. A body that is a stream is read whole before the request is sent (Planned: streamed uploads).

A server hands its handler a Request for what it received: its url is absolute, its headers are immutable, and its body is received while it is read.

Blob, File and FormData#

Global, as in Node. new Blob(parts?: (string | Uint8Array | ArrayBuffer | DataView | u8[] | Blob)[], { type? }) holds the bytes of its parts (text as UTF-8) with a media type (lowercased; "" if it is not printable ASCII): size, type, slice(start?, end?, contentType?) (negative indices count from the end), and the readers text(), bytes(): Uint8Array, arrayBuffer() and stream() (a ReadableStream of Uint8Arrays). new File(parts, name, { type?, lastModified? }) is a Blob with a name and a lastModified time (milliseconds; default now).

FormData holds named fields, text or files: append(name, value: string | Blob, filename?) (a blob is held as a File named filename, its own name, or "blob"), set, get(name): string | File | null, getAll, has, delete, forEach, keys(), values(), entries() and for...of. As a body it is sent as multipart/form-data, the way undici writes it (line breaks in text values as CRLF, a file without a type as application/octet-stream).

async function upload(url: string, report: string): Promise<number> {
  const form = new FormData();
  form.append("title", "Weekly report");
  form.append("file", new Blob([report], { type: "text/csv" }), "report.csv");
  const res = await fetch(url, { method: "POST", body: form });
  return res.status;
}

BodyStream#

Velt's body stream of u8[] chunks (a ReadableStream of Uint8Arrays is a body too, as in JS): BodyStream.from(chunks: AsyncIterable<u8[], E>) makes one from an async iterable, such as an async generator, for new Response(stream): a server sends each chunk as it is made, and for await (const chunk of stream) reads it. A stream is read once. An error the iterable throws reaches the reader as an IoError ("EIO") carrying its text.

import { BodyStream } from "velt/net/http";
import { utf8Encode } from "velt/encoding";

async function* lines(): AsyncGenerator<u8[]> {
  yield utf8Encode("one ");
  yield utf8Encode("two");
}

const res = new Response(BodyStream.from(lines()));
console.log(await res.text()); // one two

Headers#

Case-insensitive, in insertion order, with repeated names: new Headers(init?: Headers | Record<string, string> | [string, string][]) (pairs keep repeated names: new Headers([["accept", "text/html"], ["accept", "application/json"]])), append(name, value), set, delete, get(name): string | null (repeated values joined with ", "), has, getSetCookie(): string[], forEach((value, name) => …), and entries(), keys(), values() and for...of, which yield the names lowercased and sorted, with repeats joined (each set-cookie separately), as in JS. Values have surrounding HTTP whitespace (tab, LF, CR, space) removed. Unlike JS, an invalid name or value is not rejected when it is added: fetch rejects it with IoError "EINVAL".

const h = new Headers({ "Content-Type": "text/plain" });
h.append("Set-Cookie", "a=1");
h.append("set-cookie", "b=2");
for (const [name, value] of h) {
  console.log(name, value); // content-type text/plain, then set-cookie a=1, set-cookie b=2
}

console.log prints a Response, Request, Headers, AbortSignal or AbortController the way Node does, with its public state (never the runtime handle behind it):

const res = new Response("hi", { status: 201 });
console.log(res.headers); // Headers { 'content-type': 'text/plain;charset=UTF-8' }
console.log(res.status, new Headers({ a: "1" })); // 201 Headers { a: '1' }

Timeouts and cancellation#

import { AbortError, TimeoutError } from "velt/task";

async function status(url: string): Promise<string> {
  try {
    const res = await fetch(url, { signal: AbortSignal.timeout(2000) });
    return `${res.status}`;
  } catch (e) {
    if (e instanceof TimeoutError) {
      return "timed out";
    }
    if (e instanceof AbortError) {
      return "cancelled";
    }
    return `failed: ${e.message}`; // IoError
  }
}

AbortController, AbortSignal (with AbortSignal.timeout and AbortSignal.any) are global too; their error classes AbortError and TimeoutError come from velt/task.

HTTP/2#

A new connection to an https: origin offers h2 and http/1.1 with ALPN (as browsers, Deno and Bun do; Node's fetch uses HTTP/1.1 unless undici's allowH2 is set). When the server chooses HTTP/2, that one connection carries every later request to the origin from every task, at once (as many as its SETTINGS_MAX_CONCURRENT_STREAMS allows; more wait in order), until it has been idle for 4 s, the server sends GOAWAY or it fails. Requests that start while it is being made wait for it rather than open connections of their own. A server that chooses HTTP/1.1 is served the HTTP/1.1 way, and later requests take its idle pooled connections first. Plain http: is always HTTP/1.1 (no h2c), as in browsers and undici.

The API does not change: the same Response, headers (lowercase, as HTTP/2 sends them), redirects, content-encoding decoding and signal. Bodies have flow control: the server may send 1 MiB of a response body ahead of what the program read (4 MiB for the connection), and a request body goes out as the server's windows allow. An abort (before or after the response's head), or a body given up (res.body cancelled), resets just that stream and frees its place on the shared connection. A request the server did not process (a GOAWAY's later streams, REFUSED_STREAM, a connection that closed before it went out) is sent again on a new connection, whatever its method, since the server never saw it (RFC 9113 §8.7). One the server may have processed (the connection failed after it went out, before the response's head) is sent again only if its method is idempotent (GET, HEAD, OPTIONS, TRACE, PUT, DELETE), as over HTTP/1.1: a POST then fails with IoError "ECONNRESET". Server push is turned off; response trailers are read and dropped (the Fetch standard has no API for them).

HTTP/2's wire format, which the client and HTTP/2 servers share, is velt/net/http/h2.

For servers#

velt/net/http is the client and the types; serving HTTP is the HTTP server package's job (serve((req: Request) => Response), HTTP/1.1 and HTTP/2, TLS, WebSocket upgrades). What a server written in Velt needs of these classes is in velt/net/http too, so a server is a library like any other (that package is one): it accepts connections with velt/net, speaks HTTP/2 with velt/net/http/h2, WebSockets with velt/net/websocket, and hands its handlers the same Request and Response fetch uses:

• RequestSource, an interface the server implements: what a received request's Request reads (method(), url(), header(name), hasHeader(name), headers(), hasBody(), read(), body()), each called when the handler first asks for that part, so the server can keep the request where it read it until then. serverRequest(source): Request makes the Request a handler reads; requestSource(req) gives the source back (null for a request made with new Request), for what is the server's own (a connection to upgrade, the client's address).
• responseParts(res): ResponseParts: a response's status, headers ([name, value, …], lowercase names), the type its body implies (written unless headers names a content-type), and its body as the response holds it (kind: BodyKind: None, Text (text), Bytes (bytes) or Stream (stream)), with no copy. A streamed body is used up, as sending it does in JS (IoError "EINVAL" if it was read already); a fetched body arrives decoded, so its content-encoding is not among the headers.
• switchingProtocols(headers?): Response: a 101 Switching Protocols for an upgrade (the Response constructor takes 200-599).
• statusText(status): string: the reason phrase, as Node's http.STATUS_CODES has it (Not Found for 404), Unknown for a status it does not list.

A server writes a handler's response from its parts:

import { BodyKind, responseParts, statusText } from "velt/net/http";

const res = Response.json({ ok: true }, { status: 201 });
const parts = responseParts(res);
let head = `HTTP/1.1 ${parts.status} ${statusText(res.status)}\r\n`;
if (parts.type != null) {
  head += `content-type: ${parts.type}\r\n`;
}
for (let i = 0; i < parts.headers.length; i += 2) {
  head += `${parts.headers[i]}: ${parts.headers[i + 1]}\r\n`;
}
console.log(head);
console.log(parts.kind == BodyKind.Text ? parts.text : "(not text)"); // {"ok":true}

Differences from Node#

• Errors are typed: an IoError with a code where Node throws TypeError: fetch failed with a cause, and the signal's AbortError / TimeoutError where Node throws a DOMException (or the abort reason).
• res.json() needs the type of the data (above).
• A Request given a signal has that signal as its signal; Node gives it one that follows it.
• console.log shows a File as a Blob (File { size, type }), without its name and time.
• res.body and req.body have no pipeTo(), pipeThrough(), tee() or BYOB reader (streams).
• Mistakes JS reports with a catchable RangeError or TypeError stop the program instead, like an index out of bounds: new Response(body, { status }) outside 200–599, a body with a status that has none (204, 205, 304), Response.redirect(url, status) with a status other than 301, 302, 303, 307 or 308, and changing the (immutable) headers of a fetched response or a received request.
• A compressed body cut off before its stream ends fails the read with IoError; Node returns the part that decoded. Silently truncated data is a bug source Velt does not copy.
• Response heads are accepted where Node is stricter in two ways: lines may end with a bare LF, and a head may be up to 400 KiB (Node: 16 KiB).
• WebAssembly programs have no network: fetch rejects with IoError ENOTSUP.
• console.log shows header names in lowercase, where Node keeps the case a name was first added with (Headers { 'content-type': … } for "Content-Type"). A printed Request shows Node's values for the options Velt does not have (mode: 'cors', cache: 'default', …), and its url as it was given (Node normalizes it: http://a.com/ for http://a.com).

async function download(url: string): Promise<number> {
  const res = await fetch(url);
  let size = 0;
  const body = res.body;
  if (body != null) {
    for await (const chunk of body) {
      size += chunk.length; // or write it to a file
    }
  }
  return size;
}