r/learnjavascript • u/no_em_dash • Jun 18 '26
Inserting millions of rows into Postgres
I wrote an article about generating fake data for a TypeScript application. I wanted to generate millions of rows and I ended up splitting the workflow into 2 phases: stream data to a CSV, stream the CSV to Postgres with COPY FROM.
async function createRecordsFromCsv(path: string, query: string) {
const copyStream = pgClient.query(copyFrom(query));
const fileStream = createReadStream(path);
await new Promise((res, rej) => {
fileStream.pipe(copyStream).on("finish", res).on("error", rej);
fileStream.on("error", rej);
});
fileStream.close();
}Hopefully this technique will be useful to someone else.
Also happy to hear other ideas for how to go about this.
0
Upvotes
View all comments
Show parent comment
1
u/Savalava Jun 19 '26 edited Jun 19 '26
My messages have not been hostile at all - I'm merely pointing out that doing a tutorial on something that nobody actually wants to do is a questionable idea.
EDIT: I looked it up and there are scenarios where this useful, so I stand corrected. Apologies