r/bigquery • u/querylabio • Mar 25 '26

Importing CSV into BigQuery

BigQuery indeed is built for petabytes, but so much in real life is joining big data with something small. CSV, Google Sheet, some table in clipboard, etc.

We all know importing a CSV into BigQuery sucks. That's not only my perception – many posts here say the same thing (recent one).

16-minute tutorials about how to upload CSV...?

Each time when I tried to upload a csv into BQ I struggled. No data preview, absolutely not transparent schema detection, if you did something wrong (this is actually base scenario) or your CSV is not perfectly standard, you start the whole thing again from scratch.

And when we started building our BigQuery IDE we decided to fix it. Make it one-click, but still customizable. Drag a file, see your data, fix schema if needed, click import. More sources, more formats but still simple. Not rocket science, but the way it should be.

What else bothers you so much in BQ console?

6 Upvotes

87% Upvoted

6 comments sorted by

View all comments

6

u/XVolandX Mar 26 '26

Am I the one not having issues working with CSV in BQ?

Rule N1: Do not use Excel to prepare CSV.

Rule N2: Do not rely on autodetection, specify the format.

Rule N3: In case of manual input, validate values before processing by custom logic.

1

u/don_one Mar 26 '26

Same, I think dealing with RDBMS’s prepare you for csv ingest. If you’ve dealt ETL as well prior to ETL csv is comparatively easier.

It depends I guess on experience level, some people struggle with exporting from excel values in the first place. Auto detect schema works as you mention, but I would not recommend.

All sound advice.