Skip to content

Instantly share code, notes, and snippets.

@PutziSan

PutziSan/Stream ChatGPT API.md

Last active July 22, 2025 19:55
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save PutziSan/468a40ce06b485f0f9aa3c17dfb057ff to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save PutziSan/468a40ce06b485f0f9aa3c17dfb057ff to your computer and use it in GitHub Desktop.
Download ZIP
Streaming ChatGPT API Responses in Node.js using async generators
Raw
Stream ChatGPT API.md

Streaming ChatGPT API Responses in Node.js

This repository contains a reusable function leveraging async generators to stream real-time responses from OpenAI's ChatGPT API, word by word, in a Node.js environment. Instead of waiting for the entire response to be generated, this function allows for an interactive experience where the AI's thoughts are streamed as they are produced.

Prerequisites

  1. Node.js 18.14 or higher (since we're using nodejs native fetch)
  2. A valid API key for OpenAI's ChatGPT

Usage

First, make sure to set the CHAT_GPT_API_KEY constant in the script to your ChatGPT API key.

Here's a quick example of how to use the function:

import { streamChatgptApi } from "./streamChatgptApi.js";

for await (const responsePart of streamChatgptApi("Hello AI, I am a human.")) {
  if (responsePart.finish_reason) {
    console.log("finished execution with reason:", responsePart.finish_reason);
  } else {
    process.stdout.write(responsePart.delta.content);
  }
}

How It Works

The function streamChatgptApi makes a POST request to ChatGPT with the stream parameter set to true. It then reads the incoming stream and yields the message parts as they arrive.

This makes it possible to process each chunk of the response in real-time, as illustrated in the usage example above.

For a deep dive into the intricacies of this approach, including how server-sent events work, and the significance of streaming via POST, check out my detailed blog post:

Stream Real-time Feedback with ChatGPT: SSE via Fetch in Node.js

Raw
streamChatgptApi.js
import assert from "node:assert";
const CHAT_GPT_API_KEY = "<YOUR_API_KEY>"; // or from env, e.g. process.env.CHAT_GPT_API_KEY
/**
* Example usage:
*
* for await (const responsePart of streamChatgptApi("Hello World.")) {
* if (responsePart.finish_reason) {
* console.log("finished execution with reason:", responsePart.finish_reason);
* } else {
* process.stdout.write(responsePart.delta.content);
* }
* }
*
* @param {string} userMessage
* @return {AsyncGenerator<Choice, void, *>}
*/
export async function* streamChatgptApi(userMessage) {
/** @link https://platform.openai.com/docs/api-reference/chat/create */
const body = {
model: "gpt-3.5-turbo-0613",
stream: true,
messages: [{ role: "user", content: userMessage }],
};
const response = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${CHAT_GPT_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
const reader = response?.body?.getReader();
// make sure the reader exists
assert.ok(reader);
const textDecoder = new TextDecoder("utf-8");
while (true) {
const { done, value } = await reader.read();
if (done) break;
const str = textDecoder.decode(value);
// get all data objects from the string which start with "data: " and remove the first one because it must be empty (since it started with DATA)
const dataObjects = str.split("data: ").slice(1);
for (const dataString of dataObjects) {
if (dataString.trim() === "[DONE]") break;
/** @type {ChatCompletionChunk} */
const data = JSON.parse(dataString);
yield data.choices[0]; // this will be returned on each iteration
}
}
}
/**
* Represents a chunk completion of chat.
* @typedef {Object} ChatCompletionChunk
* @property {string} id - The ID of the chunk.
* @property {Choice[]} choices - The choices for the chunk.
* @property {number} created - The timestamp of creation.
* @property {string} model - The model used.
* @property {string} object - The type of object.
*/
/**
* Represents a choice.
* @typedef {Object} Choice
* @property {Delta} delta - The delta of the choice.
* @property {("stop" | "length" | "function_call" | null)} finish_reason - The reason for finishing.
* @property {number} index - The index of the choice.
*/
/**
* Represents a delta.
* @typedef {Object} Delta
* @property {string|null} [content] - The contents of the chunk message.
* @property {("system" | "user" | "assistant" | "function")} [role] - The role of the author of this message.
*/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment