package
Effect integrations for Vercel: AI Gateway, OIDC, OTLP flushing, Postgres pools, Runtime Cache, waitUntil
npm install effect-vercelGitHub-flavored package documentation.
Across all versions.
Download trend unavailable.
Runtime metadata.
Tagged topics.
Estimated unpacked size across 1 packages.
116.4 kB
Includes dependencies.
Effect (v4) integrations for Vercel. Each module is its own entry point, imported as a namespace like Effect's own modules:
import * as AiGateway from "effect-vercel/AiGateway"| Module | What |
|---|---|
effect-vercel/AiGateway | Vercel AI Gateway for Effect AI |
effect-vercel/AiGatewayCredentials | How AI Gateway requests authenticate |
effect-vercel/Functions | Bindings for @vercel/functions: waitUntil |
effect-vercel/Oidc | The OIDC token Vercel issues to a deployment |
effect-vercel/Otlp | Keeps a Function running until its OTLP telemetry is exported |
effect-vercel/PgClient | @effect/sql-pg pool that doesn't leave connections open on frozen instances |
effect-vercel/RuntimeCache | for Effect |
pnpm add effect-vercel effect
pnpm add @effect/ai-anthropic # only for effect-vercel/AiGateway
pnpm add @vercel/functions # only for effect-vercel/Functions and RuntimeCache
pnpm add @effect/sql-pg # only for effect-vercel/PgClientThe gateway serves the Anthropic Messages API and routes to any model it lists, named
provider/model. AiGateway points Effect AI's Anthropic provider at the gateway, handles
authentication per request, and smooths over the small differences between the gateway's
dialect and what the provider's schemas expect.
import * as AiGateway from "effect-vercel/AiGateway"
import { AnthropicLanguageModel } from "@effect/ai-anthropic"
import { Effect, Layer } from "effect"
import { LanguageModel } from "effect/ai"
import { FetchHttpClient } from "effect/http"
const Model = AnthropicLanguageModel.layer({ model: "google/gemini-2.5-flash" }).pipe(
Layer.provide(AiGateway.layer),
Layer.provide(FetchHttpClient.layer),
)
const program = LanguageModel.generateText({ prompt: "Say hi" }).pipe(
Effect.map((r) => r.text),
AiGateway.layer provides AnthropicClient and requires an HttpClient. It authenticates
with AI_GATEWAY_API_KEY if set, else the Vercel OIDC token, the same order as Vercel's own
SDK. So it works locally with an API key and on Vercel with no setup.
Any Effect AI feature that works with the Anthropic provider (tools, structured output, streaming, thinking) works through the gateway, for every model the gateway offers.
To pick the source yourself, use AiGateway.layerWithoutCredentials and provide an
AiGatewayCredentials layer:
import * as AiGatewayCredentials from "effect-vercel/AiGatewayCredentials"
AnthropicLanguageModel.layer({ model: "google/gemini-2.5-flash" }).pipe(
Layer.provide(AiGateway.layerWithoutCredentials),
Layer.provide([AiGatewayCredentials.layerFromApiKey(key), FetchHttpClient.layer]),
)| Layer | Source |
|---|---|
AiGatewayCredentials.layer | The default: layerFromEnv, then layerFromOidc. |
AiGatewayCredentials.layerFromEnv | AI_GATEWAY_API_KEY. Local dev, CI, anywhere off Vercel. |
AiGatewayCredentials.layerFromOidc | The Vercel OIDC token (see OIDC). |
AiGatewayCredentials.layerFromApiKey(key) | A fixed key. Tests, custom wiring. |
The credential is resolved on every request, so a rotating token is always current.
Each request carries x-api-key and ai-gateway-auth-method (api-key or oidc). A
missing credential fails the request with an AiError wrapping an AiGatewayCredentialsError that
names the source it tried and how to fix it.
Environment variables are read through Effect's Config, so a ConfigProvider can redirect
them.
Two adjustments are applied to traffic with the gateway:
"cache_control": null is removed from content blocks. The Effect provider emits it,
Anthropic accepts it, the gateway rejects it.signature on thinking blocks from non-Anthropic models, and
type/request_id on error envelopes (unknown error.type values map to api_error).Streaming responses pass through untouched.
Vercel issues each deployment an OIDC token. Use it for the AI Gateway, or exchange it for
cloud credentials (AWS AssumeRoleWithWebIdentity, GCP workload identity federation).
import * as Oidc from "effect-vercel/Oidc"
import { Effect, Redacted } from "effect"
const program = Effect.gen(function* () {
const token = yield* Oidc.ambientToken
// Redacted.value(token) is the web identity token
})Oidc.ambientToken reads the x-vercel-oidc-token header of the current request
(Functions), else VERCEL_OIDC_TOKEN (builds, vercel env pull). Empty values count as
unset. It fails with an OidcError when neither is set.
It reads the token on every run, since it rotates per request. It does not refresh an
expired local token; re-run vercel env pull.
Functions.waitUntil(fiber) keeps the Function running until fiber ends, as waitUntil
from @vercel/functions does for a promise. Use it to finish work after the response is sent.
import * as Functions from "effect-vercel/Functions"
import { Effect } from "effect"
const handler = Effect.gen(function* () {
yield* Effect.forkDetach(sendReceipt).pipe(Effect.tap(Functions.waitUntil))
return new Response("OK")
})Effect.forkDetach, or into a scope that outlives the request. A child fiber is
interrupted when the request fiber ends, whatever waitUntil says.Effect.scoped, since the request's resources may be released first.waitUntil does nothing and the fiber runs in the background.Effect's OTLP exporters buffer spans, logs and metrics and export them on an interval. Vercel can
suspend the instance once the response is sent, so the interval may never fire and the telemetry
is lost. effect-vercel/Otlp flushes the exporters when each request ends and keeps the Function
running until the flush is done.
For an HttpRouter app, add the flushAfterRequest middleware next to your routes. Provide the
OTLP layer with Layer.provideMerge, so its tracer and logger reach each request:
import * as VercelOtlp from "effect-vercel/Otlp"
import { Layer } from "effect"
import { FetchHttpClient, HttpRouter } from "effect/http"
import { Otlp } from "effect/observability"
const { handler } = HttpRouter.toWebHandler(
Layer.merge(Routes, VercelOtlp.flushAfterRequest).pipe(
Layer.provideMerge(Otlp.layerJson({ baseUrl, resource })),
Layer.provide(FetchHttpClient.layer),
),
)For another entry point, such as a queue consumer, run VercelOtlp.flushOnClose at the start of
the request in a scope that closes when the request ends:
// `services` holds the app's services, including the OTLP layer's Flusher.
const runSync = Effect.runSyncWith(services)
const scope = runSync(Scope.make())
runSync(VercelOtlp.flushOnClose.pipe(Scope.provide(scope)))
// When the request ends:
runSync(Scope.close(scope, Exit.void))HttpRouter app, that is after the
response body is sent, so telemetry recorded while a body streams is included.OtlpExporter.Flusher that the OTLP layers provide, so leaving them out fails to
compile.flushOnClose first in its scope, so finalizers added later, such as ending a span, run before
the flush.flush can't wait for an export the interval already started, and skips exporters disabled
after a failure.PgClient.layer is PgClient.layer from @effect/sql-pg that keeps the Function instance up
until the pool has closed its connections.
Vercel freezes an instance once no request or waitUntil promise is left. A frozen instance
keeps its sockets open, so a pooler like PgBouncer still counts them against max_client_conn.
During a deploy many instances freeze at once, which can use up the limit. attachDatabasePool
from @vercel/functions handles this for node-postgres pools by waiting out the idle timeout
after each release. It doesn't support the Effect pool, which also closes an idle connection up
to twice its idleTimeout after its last use.
import * as PgClient from "effect-vercel/PgClient"
import { Redacted } from "effect"
const SqlLive = PgClient.layer({
url: Redacted.make(process.env.DATABASE_URL!),
maxConnections: 3,
idleTimeout: "5 seconds",
})@effect/sql-pg, except stream: it opens each socket itself
and counts it until it closes. While a socket is open, one waitUntil promise keeps the
instance up.onAcquire. An onAcquire in your config still runs, after.idleTimeout short. The instance stays up until the pool closes its last idle
connection.RuntimeCache.layer provides Effect's Persistence, stored in the Vercel Runtime Cache.
Use it with PersistedCache to share results across Function instances in a region.
import * as RuntimeCache from "effect-vercel/RuntimeCache"
import { Effect, Schema } from "effect"
import { Persistable, PersistedCache } from "effect/persistence"
class GetUser extends Persistable.Class<{ payload: { id: number } }>()("GetUser", {
primaryKey: (req) => `GetUser:${req.id}`,
success: User,
error: Schema.Never,
}) {}
const program = Effect.gen(function* () {
const users = yield* PersistedCache.make(fetchUser, {
storeId:
@vercel/functions falls back to an in-memory cache, so the layer works in
local dev and tests.[storeId, primaryKey], hashed with SHA-256. Only the hash reaches
Vercel. getCache's default hash is 32-bit and can collide, so
RuntimeCache.makeBacking(cache) expects a cache with a stronger keyHashFunction if
you build your own.effect-persistence:<sha256 of storeId>. clear expires that tag.RuntimeCache.get, set, remove or clear client span with a
cache.store_id attribute. get spans also set cache.hit. Keys stay out of span attributes and error messages.getCache has no option to surface them.pnpm install
pnpm test
pnpm check # tsc
pnpm lint # oxlint
pnpm fmt # oxfmt
pnpm build # tsdown -> dist/MIT
Persistence