Keeps a Function running until its OTLP telemetry is exported
effect-vercel/PgClient
@effect/sql-pg pool that doesn't leave connections open on frozen instances
effect-vercel/RuntimeCache
for Effect
Install
pnpm add effect-vercel effectpnpm add @effect/ai-anthropic # only for effect-vercel/AiGatewaypnpm add @vercel/functions # only for effect-vercel/Functions and RuntimeCachepnpm add @effect/sql-pg # only for effect-vercel/PgClient
AI Gateway
The gateway serves the Anthropic Messages API and routes to any model it lists, named
provider/model. AiGateway points Effect AI's Anthropic provider at the gateway, handles
authentication per request, and smooths over the small differences between the gateway's
dialect and what the provider's schemas expect.
Usage
import * as AiGateway from "effect-vercel/AiGateway"import { AnthropicLanguageModel } from "@effect/ai-anthropic"import { Effect, Layer } from "effect"import { LanguageModel } from "effect/ai"import { FetchHttpClient } from "effect/http"const Model = AnthropicLanguageModel.layer({ model: "google/gemini-2.5-flash" }).pipe( Layer.provide(AiGateway.layer), Layer.provide(FetchHttpClient.layer),)const program = LanguageModel.generateText({ prompt: "Say hi" }).pipe( Effect.map((r) => r.text),
AiGateway.layer provides AnthropicClient and requires an HttpClient. It authenticates
with AI_GATEWAY_API_KEY if set, else the Vercel OIDC token, the same order as Vercel's own
SDK. So it works locally with an API key and on Vercel with no setup.
Any Effect AI feature that works with the Anthropic provider (tools, structured output,
streaming, thinking) works through the gateway, for every model the gateway offers.
Credentials
To pick the source yourself, use AiGateway.layerWithoutCredentials and provide an
AiGatewayCredentials layer:
import * as AiGatewayCredentials from "effect-vercel/AiGatewayCredentials"AnthropicLanguageModel.layer({ model: "google/gemini-2.5-flash" }).pipe( Layer.provide(AiGateway.layerWithoutCredentials), Layer.provide([AiGatewayCredentials.layerFromApiKey(key), FetchHttpClient.layer]),)
Layer
Source
AiGatewayCredentials.layer
The default: layerFromEnv, then layerFromOidc.
AiGatewayCredentials.layerFromEnv
AI_GATEWAY_API_KEY. Local dev, CI, anywhere off Vercel.
The credential is resolved on every request, so a rotating token is always current.
Each request carries x-api-key and ai-gateway-auth-method (api-key or oidc). A
missing credential fails the request with an AiError wrapping an AiGatewayCredentialsError that
names the source it tried and how to fix it.
Environment variables are read through Effect's Config, so a ConfigProvider can redirect
them.
Dialect fixes
Two adjustments are applied to traffic with the gateway:
Requests: "cache_control": null is removed from content blocks. The Effect provider emits it,
Anthropic accepts it, the gateway rejects it.
Responses: keys Anthropic always sends but the gateway omits are filled in with Anthropic's
"nothing to report" values, so the provider's schemas decode. These are the cache and
service-tier usage fields, signature on thinking blocks from non-Anthropic models, and
type/request_id on error envelopes (unknown error.type values map to api_error).
Streaming responses pass through untouched.
OIDC
Vercel issues each deployment an OIDC token. Use it for the AI Gateway, or exchange it for
cloud credentials (AWS AssumeRoleWithWebIdentity, GCP workload identity federation).
import * as Oidc from "effect-vercel/Oidc"import { Effect, Redacted } from "effect"const program = Effect.gen(function* () { const token = yield* Oidc.ambientToken // Redacted.value(token) is the web identity token})
Oidc.ambientToken reads the x-vercel-oidc-token header of the current request
(Functions), else VERCEL_OIDC_TOKEN (builds, vercel env pull). Empty values count as
unset. It fails with an OidcError when neither is set.
It reads the token on every run, since it rotates per request. It does not refresh an
expired local token; re-run vercel env pull.
Functions
Functions.waitUntil(fiber) keeps the Function running until fiber ends, as waitUntil
from @vercel/functions does for a promise. Use it to finish work after the response is sent.
import * as Functions from "effect-vercel/Functions"import { Effect } from "effect"const handler = Effect.gen(function* () { yield* Effect.forkDetach(sendReceipt).pipe(Effect.tap(Functions.waitUntil)) return new Response("OK")})
Fork with Effect.forkDetach, or into a scope that outlives the request. A child fiber is
interrupted when the request fiber ends, whatever waitUntil says.
The fiber keeps the request's services. Acquire resources it needs inside its own
Effect.scoped, since the request's resources may be released first.
Failures are not reported. Handle or log them in the forked effect.
Vercel stops the work when the Function reaches its maximum duration.
Off Vercel, waitUntil does nothing and the fiber runs in the background.
OTLP
Effect's OTLP exporters buffer spans, logs and metrics and export them on an interval. Vercel can
suspend the instance once the response is sent, so the interval may never fire and the telemetry
is lost. effect-vercel/Otlp flushes the exporters when each request ends and keeps the Function
running until the flush is done.
For an HttpRouter app, add the flushAfterRequest middleware next to your routes. Provide the
OTLP layer with Layer.provideMerge, so its tracer and logger reach each request:
import * as VercelOtlp from "effect-vercel/Otlp"import { Layer } from "effect"import { FetchHttpClient, HttpRouter } from "effect/http"import { Otlp } from "effect/observability"const { handler } = HttpRouter.toWebHandler( Layer.merge(Routes, VercelOtlp.flushAfterRequest).pipe( Layer.provideMerge(Otlp.layerJson({ baseUrl, resource })), Layer.provide(FetchHttpClient.layer), ),)
For another entry point, such as a queue consumer, run VercelOtlp.flushOnClose at the start of
the request in a scope that closes when the request ends:
// `services` holds the app's services, including the OTLP layer's Flusher.const runSync = Effect.runSyncWith(services)const scope = runSync(Scope.make())runSync(VercelOtlp.flushOnClose.pipe(Scope.provide(scope)))// When the request ends:runSync(Scope.close(scope, Exit.void))
The flush starts when the request's scope closes. For an HttpRouter app, that is after the
response body is sent, so telemetry recorded while a body streams is included.
Every request flushes, sampled or not, since logs and metrics are exported either way. Spans and
logs are sent only when some are buffered. Metrics send a snapshot on every flush.
The Function waits for the flush for at most 2 seconds.
Both need the OtlpExporter.Flusher that the OTLP layers provide, so leaving them out fails to
compile.
They register the wait with Vercel when the request starts, while its context is current. Run
flushOnClose first in its scope, so finalizers added later, such as ending a span, run before
the flush.
flush can't wait for an export the interval already started, and skips exporters disabled
after a failure.
Off Vercel, or outside a request, they do nothing and the exporters' interval sends the
telemetry.
Postgres
PgClient.layer is PgClient.layer from @effect/sql-pg that keeps the Function instance up
until the pool has closed its connections.
Vercel freezes an instance once no request or waitUntil promise is left. A frozen instance
keeps its sockets open, so a pooler like PgBouncer still counts them against max_client_conn.
During a deploy many instances freeze at once, which can use up the limit. attachDatabasePool
from @vercel/functions handles this for node-postgres pools by waiting out the idle timeout
after each release. It doesn't support the Effect pool, which also closes an idle connection up
to twice its idleTimeout after its last use.
import * as PgClient from "effect-vercel/PgClient"import { Redacted } from "effect"const SqlLive = PgClient.layer({ url: Redacted.make(process.env.DATABASE_URL!), maxConnections: 3, idleTimeout: "5 seconds",})
It takes the same config as @effect/sql-pg, except stream: it opens each socket itself
and counts it until it closes. While a socket is open, one waitUntil promise keeps the
instance up.
Each time a request takes a connection from the pool, the promise moves to that request, so a
busy instance never keeps a finished request open. The wait also ends a second before the
request's deadline.
It moves the promise from onAcquire. An onAcquire in your config still runs, after.
Keep idleTimeout short. The instance stays up until the pool closes its last idle
connection.
Off Vercel, it behaves like the plain layer.
Runtime Cache
RuntimeCache.layer provides Effect's Persistence, stored in the Vercel Runtime Cache.
Use it with PersistedCache to share results across Function instances in a region.
import * as RuntimeCache from "effect-vercel/RuntimeCache"import { Effect, Schema } from "effect"import { Persistable, PersistedCache } from "effect/persistence"class GetUser extends Persistable.Class<{ payload: { id: number } }>()("GetUser", { primaryKey: (req) => `GetUser:${req.id}`, success: User, error: Schema.Never,}) {}const program = Effect.gen(function* () { const users = yield* PersistedCache.make(fetchUser, { storeId:
Off Vercel, @vercel/functions falls back to an in-memory cache, so the layer works in
local dev and tests.
Entries are keyed by [storeId, primaryKey], hashed with SHA-256. Only the hash reaches
Vercel. getCache's default hash is 32-bit and can collide, so
RuntimeCache.makeBacking(cache) expects a cache with a stronger keyHashFunction if
you build your own.
Each entry is tagged effect-persistence:<sha256 of storeId>. clear expires that tag.
TTLs round up to whole seconds.
Each call runs in a RuntimeCache.get, set, remove or clear client span with a
cache.store_id attribute. get spans also set cache.hit. Keys stay out of span attributes and error messages.
The Runtime Cache swallows its own network errors and timeouts, which then read as misses
and silently skipped writes, removes and clears. getCache has no option to surface them.