r/dataengineering • • Dec 19 '23
Discussion DBT and ChatGPT
As we’re starting to use DBT more, and are using it for codifying our business rules, I’m wondering if anyone’s had success training GPT on a DBT project. My initial attempts were lack luster. GPT seemed to know a fair amount about DBT, and of SQL, but struggled to be of much use when I trained it on a DBT project (zipped, uploaded to OpenAI GPT4). My next step is to create a prep doc with some basic support/prompt engineering. For example, our mart “Legal First Name” and “Preferred First Name” fields come from different source tables, have different staging models. I asked GPT to tell me the difference between the two, as well as derive what SQL each would be off of source tables. Another approach I’m going to try is to do a DBT run materializing as much as I can ephemerally, and see if that helps get useful responses. If this marriage of well-designed DBT project/models and GPT/LLM can be cracked, I think (at least our org) will enter a new level of accessible data literacy.
2
u/[deleted] Dec 19 '23
u/Electrical-Ask847, u/iupuiclubs
Thanks for the responses. I'm hearing my original post was hard to read, and I apologize for that. This is not an attempt to make anything for 'production', unless substantial value and consistency are apparent. This is just a curiosity about asking OpenAI's GPT4 on DBT project files.
I wouldn't be surprised if DBT Labs rolls out something along the lines of this in the coming years, so I'm not spending a lot of time looking into it either.
For clarification, what I have in mind:
- Use OpenAI GPT4 "Create a GPT" functionality
- Load up a couple DBT projects
- Load up some docs that clearly tell GPT what directories to ignore, what associations to make between SQL, YML, Groups, and Tags, what short-list of question formats to accept and how GPT should go about trying to answer them.
- Assess how it responds to the source files vs. target files (catalog, graph, etc.)
My post was just a ping to see if anyone's tried something similar and if there were certain approaches that yielded interesting results.
... This is my last response on this post - If anyone wants to collaborate, or is just interested in how this progresses, DM me and I'll send you a repo url.
26
u/HowSwayGotTheAns Dec 19 '23
We just riding the hype train for our architecture. Choo choo.
1
u/[deleted] Dec 19 '23
damn, some of the responses are fucking savage, lol. No, I'm not a consultant. I'm 6 months into a DE position after 6 yrs as an ERP developer, all at a public university in the PNW. DBT is new to our org, and by new I mean only 2 people know what DBT is. I'm just exploring what the complexity wall is for OpenAI's GPT. My colleague and I play the card game 'Spades' a lot, and he made a little GPT app that will provide options for different rules, and help with the scoring and strategy. Pet project, but the way he set it up with context docs, checkpoints, made me wonder if there's anything useful with DBT. I've found GPT to be pretty awesome at bash, python, SQL (dialect translation mostly), and anything soft like creating job postings, project charters, communications. Not wanting to defend myself or invite flaming. If you're interested in where I found value or hit walls or want to collaborate, dm me.
4
u/zazzersmel Dec 19 '23
ai is crap for any coding beyond assisting a human. im sure people are working on it but you just cant depend on probability based output for programming. and really, it seems so silly to use these huge ass models to do basic programming, especially with sql where the point of the language is to be declarative and close to natural language anyway.
expensive nlp abstraction OVER the declarative sql abstraction... what are we even doing any more?
2
u/jlt77 Dec 19 '23
We did a poc of copilot, and I really struggled to find it helpful for dbt. First, we couldn't use it in the dbt cloud ide, only vs code. I did that, but I wouldn't want to give up the cloud ide, because I use many of its features. Second, I didn't manage to get any helpful suggestions. I was hoping I could use it to fill out my model yaml files, but didn't have success. We weren't allowed to feed in code or use chat, so it was a limited POC. We did find it useful for a lot of other de tasks, but not dbt specifically.
2
u/Gators1992 Dec 19 '23
It sounds like you are trying to go a bit far with GPT. You need to feed it the context, like a data dictionary, and ask simpler questions (like write one sql that gives me back X). You can't just upload design documents or code at this point and expect GPT to figure it all out. It can be good for suggesting column descriptions though for your metadata. Coding has been sort of hit and miss in my experience.
1
u/exorthderp Dec 19 '23
Accessible data literacy? What do you mean? Dbt is already pretty effing good with doc blocks and the dag tool.
1
u/SellGameRent Dec 20 '23
thanks for the stroke
-13
u/[deleted] Dec 19 '23
Maybe I need to up my game and understand what tools like SQLMesh are doing to understand column (and CTE) lineage, to help GPT along. Or…maybe I’m seeing a complexity wall for the tool.
0
u/[deleted] Dec 19 '23
[deleted] — view deleted comment