Function streaming to be framework-agnostic on Vercel

Copy link to headingWhat changes when streaming responses?

When your Vercel Functions stream responses instead of buffering, you can immediately return results to your visitors—either an API function response or a server-side template.

Copy link to headingResponses on runtime errors

When streaming, response headers are sent immediately with the initial stream. If there is a runtime error after the initial headers have been sent, the function closes the stream and logs an error to Vercel. For example:

1
import { setTimeout } from "node:timers/promises";
2
export default async function (req, res) {
3
  res.writeHead(200, { "Content-Type": "text/plain" });
4
  res.write("Chunk 1\\n");
5
  await setTimeout(1000);
6
  throw new Error("Oh no!");
7
}

When buffering a response and not streaming, this function would not respond for several seconds. Since the function throws an error before completion, it would respond with a 500 error:

1
HTTP/2 500
2
cache-control: public, max-age=0, must-revalidate
3
content-type: text/plain; charset=utf-8
4
date: Mon, 24 Jun 2024 10:26:35 GMT
5
server: Vercel
6
strict-transport-security: max-age=63072000; includeSubDomains; preload
7
x-robots-tag: noindex
8
x-vercel-error: FUNCTION_INVOCATION_FAILED
9
x-vercel-id: cdg1::hgz6d-1719224792404-c4031128df3b
10
content-length: 56
11
A server error has occurred
12
FUNCTION_INVOCATION_FAILED

When streaming is enabled, the response status code and part of the body will have already been sent to the client. The stream will send the status code and part of the body before ending. For example, you would then see Oh no! error in the Vercel logs.

1
HTTP/2 200
2
age: 0
3
cache-control: public, max-age=0, must-revalidate
4
content-type: text/plain
5
date: Mon, 24 Jun 2024 10:29:58 GMT
6
server: Vercel
7
strict-transport-security: max-age=63072000; includeSubDomains; preload
8
x-robots-tag: noindex
9
x-vercel-cache: MISS
10
x-vercel-execution-region: iad1
11
x-vercel-id: cdg1::iad1::zclv9-1719224997952-452256b220b0
12

13
Chunk 1

Copy link to headingEnhanced runtime logs experience

Without streaming, Functions need to finish executing before a response can be sent to the client. Similarly, logs are only sent from the function and visible in the Vercel logs UI until after the function finishes responding.

When viewing runtime logs in the Logs tab, you would see a single line for each function invocation. The log line contains all context for that invocation up to 4KB size limit. For example:

1
import { setTimeout } from "node:timers/promises";
2
export default async function (req, res) {
3
  console.log("Before setting a response");
4
  res.writeHead(200, { "Content-Type": "text/plain" });
5
  res.write("Chunk 1\\n");
6
  await setTimeout(1000);
7
  console.warn("Before writing a second chunk");
8
  res.write("Chunk 2\\n");
9
  await setTimeout(1000);
10
  console.error("Before ending the response");
11
  res.end("Done!");
12
}

When this function is called three times in a row, we can see the following in Vercel Logs tab:

We see three log lines above, each appearing only after the function invocation had finished. Logs for each invocation are collapsed into a single line. Now, let's explore how this changes when we enable streaming.

With streaming, each log statement gets its own line with a precise timestamp. Logs appear in near real-time, streaming in as they happen. You do not need to wait for the function invocation to complete to see logs.

You will notice the individual log lines do not display the status code. This is because logs are independent of the response and can occur before a status code is determined. With streaming being enabled by default, this brings consistency to the logs experience when using Vercel Functions.

These changes in frequency and format of Function logs will also affect Vercel Log Drains. If you are using Log Drains, ensure that your provider supports streaming responses and this increase frequency for logs.

Copy link to headingEnabling streaming functions today

Streaming will be enabled by default for Hobby accounts starting July 8th, 2024; and for Pro and Enterprise accounts starting October 1st, 2024.

To enable streaming as the default today for all your Vercel Functions, set the VERCEL_FORCE_NODEJS_STREAMING environment variable in your project to true. Streaming will be enabled on your next deployment.