Skip to content
Building Real-time AI Agents with Astro and Cloudflare
Artificial Intelligence Last updated on 🎯 Intermediate Astro 6.x, Cloudflare Workers AI Verified

Building Real-time AI Agents with Astro and Cloudflare

Learn how to architect a performance-first AI agent using Astro's type-safe Actions and Cloudflare's global edge network.

Key takeaways
  1. → Astro Actions provide a type-safe interface for invoking LLMs from the client.
  2. → Cloudflare Workers AI allows running models like Llama 3 at the edge with zero latency.
  3. → Streaming responses via ReadableStream is essential for a responsive AI UX.
  4. → Using edge databases like Cloudflare D1 enables real-time state management for multi-hop agent execution.

Bottom Line Up Front: Building real-time AI agents requires moving inference to the edge. By combining Astro Actions for type-safe server logic with Cloudflare Workers AI, you can deploy low-latency, streaming AI experiences that scale globally without the overhead of traditional server management.

What is Edge AI Inference?

Edge AI Inference is the process of executing machine learning models (such as LLMs, embedding engines, or image generators) on physical servers situated close to the end user, rather than routing traffic to a centralized database center (e.g., US-East-1). Running model inference on Cloudflare’s global network (spanning 300+ cities) minimizes round-trip time (RTT), resulting in a highly responsive user experience.

Why the Edge is the Future of AI Agents

Traditional AI architectures often involve sending data to a centralized server, which then calls an LLM provider (like OpenAI), waits for a response, and sends it back. This “hub-and-spoke” model introduces significant latency.

main
src/ index.plaintext
--:-- 
CENTRALIZED PATTERN:
User ---> Application Server (US-East) ---> OpenAI API (West) ---> App Server ---> User
Total RTT: 800ms - 1500ms

EDGE INFRASTRUCTURE PATTERN:
User ---> Cloudflare Edge Server (Closest Data Center) ---> Workers AI (Same Center GPU) ---> User
Total RTT: 150ms - 300ms

Deploying on Cloudflare allows your “agent” logic to run in a data center closest to your user. When combined with Astro’s performance-first architecture, you eliminate the “middleman” server entirely. This shift is critical because why AI agents matter in 2026 depends heavily on their ability to execute sequential tool-calling loops without causing frustrating user-facing delays.

Latency Metrics Comparison

| Model | Provider | Location | TTFT (Average) | Tokens/Sec (Average) | | :----------------------- | :-------------------- | :-------------- | :------------- | :------------------- | | Llama-3-8B-Instruct | Cloudflare Workers AI | Edge (Anywhere) | 180ms | 45 | | GPT-4o-mini | OpenAI API | US-West | 620ms | 80 | | Llama-3-70B-Instruct | Centralized Cloud | US-East | 890ms | 25 |

Setting Up the Environment

To follow this tutorial, you need a basic understanding of TypeScript and an active Cloudflare developer account.

Prerequisites

  • Node.js version 20 or higher.
  • pnpm or npm installed.
  • A Cloudflare account with billing enabled (Workers AI includes a generous free tier of 10,000 free tokens/day).

Installation and Dependencies

Scaffold a new Astro project and add the Cloudflare adapter using the command-line helper:

main
src/ setup-astro-ai.sh
--:--

npx astro add cloudflare

This command automatically installs the @astrojs/cloudflare adapter package, configures Astro to run in SSR (Server-Side Rendering) mode, and updates your astro.config.mjs settings.

Next, open your project directory and configure your Cloudflare bindings. Ensure your wrangler.jsonc (or wrangler.toml) file at the root of the project contains the AI binding definition:

main
src/ index.json
--:-- 
{
  "name": "astro-ai-agent",
  "compatibility_date": "2026-06-01",
  "main": "dist/server/entry.mjs",
  "ai": {
    "binding": "AI"
  }
}

This binding injects the Cloudflare Workers AI SDK into the Astro runtime, exposing it under context.locals.runtime.env.AI.

Implementing the AI Action

Astro Actions allow us to define server-side logic that our frontend can call like a regular function. Here is how we define an AI inference action:

main
src/ index.typescript
--:-- 
// src/actions/index.ts
import { defineAction } from "astro:actions";
import { z } from "astro:schema";

export const server = {
  askAgent: defineAction({
    input: z.object({
      prompt: z.string().min(1, "Prompt cannot be empty"),
      systemPrompt: z.string().optional(),
    }),
    handler: async ({ prompt, systemPrompt }, context) => {
      // 1. Locate Cloudflare Workers AI binding
      const env = context.locals.runtime?.env;
      if (!env || !env.AI) {
        throw new Error("Cloudflare Workers AI binding was not found.");
      }

      const ai = env.AI;

      // 2. Set up safety boundaries and model defaults
      const systemMessage = systemPrompt || "You are a helpful assistant.";

      try {
        // 3. Invoke model execution on GPUs close to the user
        const response = await ai.run("@cf/meta/llama-3-8b-instruct", {
          messages: [
            { role: "system", content: systemMessage },
            { role: "user", content: prompt },
          ],
          stream: true,
        });

        // 4. Return the raw ReadableStream directly to the handler client
        return response;
      } catch (error) {
        console.error("Workers AI inference error:", error);
        throw new Error("Failed to process request through edge AI model.");
      }
    },
  }),
};

Streaming the Response to the UI

One of the biggest mistakes in AI UX is making the user wait for the full response. Streaming is non-negotiable for agents.

By using the stream: true option in Cloudflare Workers AI, the response is returned as a ReadableStream. In your Astro component, you can consume this stream and update your state in real-time. Here is a practical implementation showing how to bind the action to a React or Vanilla JS interface:

main
src/ index.html
--:-- 
<!-- src/components/AgentChat.astro -->
<div class="chat-container mx-auto max-w-xl rounded-lg border bg-black/20 p-4">
  <div id="chat-output" class="mb-4 h-64 overflow-y-auto border-b p-2"></div>
  <form id="chat-form" class="flex gap-2">
    <input
      type="text"
      id="chat-input"
      placeholder="Ask your agent..."
      class="flex-1 rounded-full border border-zinc-700 bg-zinc-800 px-4 py-2 text-white"
    />
    <button
      type="submit"
      class="rounded-full bg-purple-600 px-6 py-2 hover:bg-purple-700"
    >
      Send
    </button>
  </form>
</div>

<script>
  import { actions } from "astro:actions";

  const form = document.getElementById("chat-form") as HTMLFormElement;
  const input = document.getElementById("chat-input") as HTMLInputElement;
  const output = document.getElementById("chat-output") as HTMLDivElement;

  form?.addEventListener("submit", async (e) => {
    e.preventDefault();
    if (!input || !output || !input.value.trim()) return;

    const userPrompt = input.value;
    input.value = "";

    // Append user input to UI
    output.innerHTML += `<div class="user-message mb-2 text-right"><strong>You:</strong> ${userPrompt}</div>`;

    // Add placeholder for streaming AI response
    const aiMessageContainer = document.createElement("div");
    aiMessageContainer.className = "ai-message mb-2 text-left";
    aiMessageContainer.innerHTML = `<strong>Agent:</strong> <span class="content">Thinking...</span>`;
    output.appendChild(aiMessageContainer);
    const contentSpan = aiMessageContainer.querySelector(".content") as HTMLSpanElement;

    try {
      // Invoke Astro action
      const { data, error } = await actions.askAgent({ prompt: userPrompt });
      if (error || !data) {
        contentSpan.innerText = "Error: " + (error?.message || "Inference failed");
        return;
      }

      // Read response stream
      const reader = (data as ReadableStream).getReader();
      const decoder = new TextDecoder();
      contentSpan.innerText = ""; // Clear "Thinking..." placeholder

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const chunkText = decoder.decode(value, { stream: true });

        // Parse server-sent event formats if necessary, or output raw tokens
        contentSpan.innerText += chunkText;
        output.scrollTop = output.scrollHeight;
      }
    } catch (err) {
      console.error(err);
      contentSpan.innerText = "Failed to communicate with agent.";
    }
  });
</script>

Pro Tip: Use a “Summary” component or a “Key Takeaways” block at the start of your long-form AI responses to improve AEO visibility for answer engines.

State Persistence at the Edge with Cloudflare D1

In a true agentic workflow, maintaining state and session history is critical. A stateless agent cannot perform multi-turn reasoning or remember user preferences. However, traditional database connections (such as TCP-based PostgreSQL or MySQL pools) introduce significant connection overhead and latency when initialized from short-lived edge workers.

This is where Cloudflare D1—Cloudflare’s serverless, zero-overhead SQLite database built directly on the Workers runtime—becomes essential. By storing chat history close to the compute environment, we can fetch and append context in single-digit milliseconds.

Data Flow for Stateful Edge Agents

Here is the architecture of a stateful edge agent utilizing D1 for persistent history and Workers AI for localized inference:

main
src/ index.plaintext
--:-- 
+--------+            +----------------------------+            +-------------+
|        |            |        Astro Server        |            |             |
|        | --(POST)-->| (Astro Action: askAgent)   |            |             |
|        |            |  +----------------------+  |            |             |
|        |            |  | 1. Query D1 History  |<------------->| Cloudflare  |
| Client |            |  +----------------------+  |            | D1 Database |
| (UI)   |            |  +----------------------+  |            |             |
|        |            |  | 2. Fetch KV Cache    |<------------->| Cloudflare  |
|        |            |  +----------------------+  |            | KV Store    |
|        |            |  +----------------------+  |            |             |
|        |            |  | 3. Run Inference     |<------------->| Workers AI  |
|        |<-(Stream)--|  +----------------------+  |            | GPU Node    |
|        |            |  +----------------------+  |            |             |
|        |            |  | 4. Save to D1 / KV   |  |            |             |
+--------+            +----------------------------+            +-------------+

1. Database Schema Configuration

First, define the SQL schema for your chat history. Create a schema.sql file in your project:

main
src/ index.sql
--:-- 
-- db/schema.sql
CREATE TABLE IF NOT EXISTS chat_history (
  id TEXT PRIMARY KEY,
  session_id TEXT NOT NULL,
  role TEXT CHECK(role IN ('system', 'user', 'assistant')) NOT NULL,
  content TEXT NOT NULL,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
CREATE INDEX IF NOT EXISTS idx_sessions ON chat_history(session_id);

Initialize your D1 database locally and globally using the Wrangler CLI:

main
src/ setup-d1.sh
--:--

npx wrangler d1 create astro-agent-db

Add the binding to your wrangler.jsonc file:

main
src/ index.json
--:-- 
{
  "name": "astro-ai-agent",
  "compatibility_date": "2026-06-01",
  "d1_databases": [
    {
      "binding": "DB",
      "database_name": "astro-agent-db",
      "database_id": "YOUR_DATABASE_UUID_HERE"
    }
  ]
}

2. State-Aware Astro Action Implementation

We can now modify our Astro Action to fetch previous messages from D1, inject them as conversation context to the LLM, and store the user query and the LLM’s response.

Here is the complete implementation of a stateful, edge-native action:

main
src/ index.typescript
--:-- 
// src/actions/index.ts (Extended with D1 persistence)
import { defineAction } from "astro:actions";
import { z } from "astro:schema";

export const server = {
  askStatefulAgent: defineAction({
    input: z.object({
      sessionId: z.string().uuid("Invalid session identifier"),
      prompt: z.string().min(1, "Prompt cannot be empty"),
    }),
    handler: async ({ sessionId, prompt }, context) => {
      const env = context.locals.runtime?.env;
      if (!env || !env.AI || !env.DB) {
        throw new Error("Missing Cloudflare runtime bindings (AI or D1 DB).");
      }

      const ai = env.AI;
      const db = env.DB;

      try {
        // 1. Fetch the last 10 messages from D1 to avoid context window bloat
        const { results } = await db
          .prepare(
            "SELECT role, content FROM chat_history WHERE session_id = ? ORDER BY created_at ASC LIMIT 10",
          )
          .bind(sessionId)
          .all<{ role: string; content: string }>();

        // 2. Format history for the Llama 3 engine
        const conversationHistory = results.map((row) => ({
          role: row.role as "system" | "user" | "assistant",
          content: row.content,
        }));

        // 3. Append current user query
        const currentMessages = [
          {
            role: "system" as const,
            content: "You are a stateful AI agent running at the edge.",
          },
          ...conversationHistory,
          { role: "user" as const, content: prompt },
        ];

        // 4. Save the user prompt immediately to D1
        const userMessageId = crypto.randomUUID();
        await db
          .prepare(
            "INSERT INTO chat_history (id, session_id, role, content) VALUES (?, ?, 'user', ?)",
          )
          .bind(userMessageId, sessionId, prompt)
          .run();

        // 5. Query Workers AI with stream enabled
        const responseStream = (await ai.run("@cf/meta/llama-3-8b-instruct", {
          messages: currentMessages,
          stream: true,
        })) as ReadableStream;

        // 6. Set up a dual-stream: stream back to user while accumulating full response for D1
        const [clientStream, serverStream] = responseStream.tee();

        // We write to D1 asynchronously without blocking the client stream
        (async () => {
          const reader = serverStream.getReader();
          const decoder = new TextDecoder();
          let completeResponse = "";

          while (true) {
            const { done, value } = await reader.read();
            if (done) break;
            completeResponse += decoder.decode(value, { stream: true });
          }

          const assistantMessageId = crypto.randomUUID();
          await db
            .prepare(
              "INSERT INTO chat_history (id, session_id, role, content) VALUES (?, ?, 'assistant', ?)",
            )
            .bind(assistantMessageId, sessionId, completeResponse)
            .run();
        })().catch((err) =>
          console.error("Failed to persist assistant response:", err),
        );

        return clientStream;
      } catch (error) {
        console.error("Stateful edge execution failed:", error);
        throw new Error("Unable to complete stateful agent processing.");
      }
    },
  }),
};

Caching Agent Responses with Cloudflare KV

To reduce compute costs and optimize latency for repetitive inputs (e.g., standard questions like “What are your hours?”), you can introduce a caching layer using Cloudflare KV (Key-Value storage).

By hashing the prompt (or session context), you can check KV first. If a cached answer exists, you bypass Workers AI inference entirely:

main
src/ index.typescript
--:-- 
// Caching demonstration snippet
const promptHash = await crypto.subtle.digest(
  "SHA-256",
  new TextEncoder().encode(prompt),
);
const cacheKey = `agent-cache:${Array.from(new Uint8Array(promptHash))
  .map((b) => b.toString(16).padStart(2, "0"))
  .join("")}`;

// Check KV
const cachedResponse = await env.KV_STORE.get(cacheKey);
if (cachedResponse) {
  // Directly stream the cached value or return immediately
  return new ReadableStream({
    start(controller) {
      controller.enqueue(new TextEncoder().encode(cachedResponse));
      controller.close();
    },
  });
}

This hybrid pattern (D1 for memory, KV for high-speed caching) results in highly efficient, low-overhead architectures suitable for enterprise-scale deployments.

Cost Estimates

Running edge inference is significantly cheaper than calling API-based LLM providers. Cloudflare Workers AI offers competitive pricing based on the size of the model and the compute resources required.

| Model Size | Metric | Cost (Free Tier Included) | Estimated Cost per 1M Tokens | | :------------------------------------- | :---------------------------- | :------------------------ | :--------------------------- | | 8B Parameters (e.g., Llama 3) | Per 1,000 input/output tokens | $0.000077 | $0.077 | | 70B Parameters (e.g., Llama 3 70B) | Per 1,000 input/output tokens | $0.000343 | $0.343 | | GPT-4o-mini | Per 1M input / output | OpenAI standard pricing | $0.150 / $0.600 |

Conclusion & Next Steps

Combining Astro and Cloudflare isn’t just about speed; it’s about simplicity. You get type-safety from TypeScript, the power of Llama 3 from Cloudflare, and the developer experience of Astro.

For our next post, we will explore adding memory to agents using Cloudflare D1 and Vectorize database, enabling persistent session states at the edge.

Sources & Further Reading

Henrique Bonfim

Author

Henrique Bonfim

Senior Software Engineer

Senior Software Engineer with extensive experience building production web systems. Creator of ZettaBytes. Contributor to open-source Astro tooling. Specializes in edge-first architectures, AI integration, and web performance.

Related articles