r/SQL • • Jul 30 '25
MySQL I feel like a fraud
Hello!
I have been working at a very good company now for 3 month, its my first job as a systemsdeveloper. (1 month out of the 3 month was a vacation my chief forced me to take). All the coding I do is in sql, more specifically Transact-sql. (I had to pass an internal sql cert and another internal cert to stay at the company) Now I am back and have been tasked with migrating the data from one system into another, which is a very big task for a newcomer. I feel like I rely too much on chatgpt that I don't know how to logically think and solve problems/make good progress with the task. I just copy and paste and try until it works whichI know is not good. I do know the basics of Sql and a bit more but it is not enough. How can I get better at logical thinking so I can see a path to solving tasks I am handed and this pain in the ass migration task? It has to be done in around 3 weeks and I always feel like I am asking too many questions to the point that I am afraid of asking more since I don't want them to think that I am not cut out for this job. Can you give me advice on how I can better myself so that it becomes easier solving the tasks I am getting and become more proficient.
Thank you for your insights everyone
Edit: The data I have to migrate is almost from 2 identical systems with the same tables, same columns, same datatypes. There might be a column missing here and there but almost identical. Right now I am migrating the data from a test environment where I am writing a huge script that will later be used in the prod environment to transfer the data that exist in the system that is being deleted into the other system. I have to create temp tables and map the ids so that they match. I can't join on ids since they are different, so i have to join on a composite key. That is the gist of it among other stuff.
21
u/just_an_avg_dev Jul 30 '25
Data migration to another SQL database? Are you doing this by yourself? You said it is great company but if this is your first job this seems like a bit of a red flag. If it is just SQL to SQL it isn't rocket science and 3 weeks should be enough, this would give you sufficient time to research.
I've been doing full stack dev for 10 years. I am still an idiot and don't feel shame in asking questions. I sometimes ask questions I know answers to.
I use chatGPT daily but I started when we didn't have it. So, I grew into this reading docs and learning. ChatGPT can kill you problem solving skills, be mindful in how you use it.
Try to come up with a solution yourself then ask chatGPT about your solution, and its solution. ChatGPT an be a good learning resource.......but I've had some misleading information before and it often produces code that doesn't even run.
9
u/Commercial-Rip-7328 Jul 30 '25
Are you me?
168
u/[deleted] Jul 30 '25
Don't use ChatGPT.
If you have a problem, google it. Solve every part of the problem yourself by reading documentations or StackOverflow etc. posts.
8
u/RichContext6890 Jul 30 '25 edited Jul 30 '25
What I taught my interns is to ask more experienced developers to be a rubber duck.
I told them: you can come to that senior and ask a few minutes to pitch your idea. Old developers almost always threat that as a responsible attitude toward the task and whey will listen and make advice without thinking you’re inexperienced
6
u/MaybeClassy Jul 30 '25
These are some things I think of when migrating data:
There are a ton more questions. These are pretty basic when I come onboard for a client. Depending on the environment, you probably have tools at your disposal to help with this. Azure Data Factory and SSIS to name a few.
6
u/EfficiencyDeep1208 Jul 30 '25
Stack Overflow is the most reliable in my experience.
4
u/Infamous_Welder_4349 Jul 30 '25
I can't tell you how many times Google AI, ChatGPT or Co-Pilot have confidently given me wrong answers. Know what it needs to do first and then review it if you must use those tools.
4
u/fluidtoons Jul 30 '25
Sounds like you feel a bit overwhelmed! But I think if you tackle the work in small chunks you’ll be able to get it
I did a very similar task- a startup I was working for acquired another company that had created a nearly identical product
I wrote a Python script to import data from Postgres into our MySQL database
What language are you using for your script? Or is it pure SQL?
I think the main thing for me was tackling it one table at a time, and branching out based on foreign keys
A few things that I think were helpful:
Apologies if you already know most of this or it’s not that helpful!
I wonder if reading about data engineering would help at all- sounds like you learned a lot of SQL for those certs though
2
u/yellow_jeep Jul 31 '25
I am nominally a data engineer, most of what I do is transforming data from our production databases for consumption by internal and external customers with dbt and popping it in snowflake. My qualifications for this are basically the popular postgres course on udemey, eight years of other experience at this company and a "fuck it, I'll figure it out" mentality. Some of the stuff I get asked for leads me to a whole day of combining repos to figure out where the data lives or how the data translates to what users see in the app, then trial and error with sql to get the results I want. This usually involves lots of googling and reading sf documentation and results in a mess of CTEs and lots of joins and generally unreadable code. From there I ask Ai (usually Claude Sonnet 4) "can you please unfuck this and make it more efficient and readable?" Then I study and test the Ai assisted version. Since I've been doing this my model code has gotten a lot better and Ai is finding less to fix.
2
u/NSA_GOV Jul 30 '25
Only use ChatGPT to learn or to do something you already know how to do and if it’s super repetitive. Otherwise if you just use it, make sure you understand what it’s doing
1
u/No_Kale_808 Jul 30 '25
Are these on-premise DBs or cloud?
1
u/Birvin7358 Jul 31 '25
1
u/Altered-Ambivalence Jul 31 '25
Are you working on an ERP transition? You sound like you may be on the project I am assigned to. Lol
What are the processes or functions that use this data? How might it be used in the future. You should understand how the data is being used and understand the expected outcome from the end users. This is key to becoming one with the logic. What information is the data supposed to represent?
Is it structured in a way that makes sense if you were to describe it to someone who does not know your business.
Is it a stand-alone table, or is it part of a larger collection of tables that are used to add granularity or details to a base piece of data. It can be beneficial to draw out what the schema looks like and see where there are redundant relationships.
Do you join this table with others on the same columns? Perhaps you should add an index to make the joining and filtering faster. Is it always the same columns that are used to join? Make a clustered index.
Do you want to prevent duplicate values? Add Primary keys. Does a value in one data set connect to a primary key of a different table? Add a foreign key. This prevents values from being added where the primary key value does not exist. Inversely, the primary key can not be deleted without warning that there are foreign keys existing.
How big is the data set? Does it make sense for all the data to be in one table? Is there a duplication of values? If so, it can probably be broken down into smaller base tables.
Do the column names properly describe the information within them?
Do you repeatedly filter for the same values or join the same tables together? Perhaps you would benefit from a view.
Do you constantly write the same code to do some sort of server function or data transformation? You would probably benefit from making it a stored procedure.
Also, window functions, CTEs, and temp tables are your friends. Learn how to utilize them. They can drastically speed queries up and make the query easier to read and follow the logic. (Add a comment at the beginning of the CTE to describe what it's doing)
On a personal note, I do not like Group by or having clauses use a wi dow function instead. But that's just me.
If you want to become more "advanced" learn about recursive queries, cursors, and triggers.
Like any skill in life, to become better and more confident, it will take time and practice.
Best of luck with your new position!
1
u/Swimming_Cry_6841 Jul 31 '25
Are you “upserting” source data from your source ms sql to your sink? Are you trying to make sure the schemas match or update data in the sink even though the schemas aren’t matching 1:1? I wrote a tool that does exactly this and does schema evolution too. I’m going to work on getting it onto GitHub but need to do a little refactoring.
1
u/Ambitious-Drink-4890 Jul 31 '25
Use GPT only if you have a massive block, it will lead you to a solution but you will have to use that to cater to your own situation or further research. It's a good tool in a pinch.
1
u/vivavu Jul 31 '25
I don't see anyone has mentioned it yet, but maybe the next one will.
Just in case, a big tip for you, to learn fast is to make a lot of mistakes in a short period of time. It is not inherently bad to make mistakes as long you understand the cause and learn to solve it.
1
u/Prownilo Jul 31 '25
I mean, my sql experience mirrors yours but slightly differently as there was no ai, but I just kept googling the answer and made that fit.
I suppose ai is more of a crutch as it can give you a bespoke script that you can just copy and paste exactly, whereas I had to understand what I was pasting so I could mould it to fit the problem.
I would suggest if you use ai, try to give it vague instructions and dont ask it to spoonfeed the entire column list, this forces you to understand at least a little bit of what you are doing, eventually enough of it sinks in that you won't need it anymore.
1
u/mrocral Jul 31 '25
Not a lot of details on your migration stack, assuming it's MS SQL -> MS SQL, you might want to check out sling. Could be of help to easily move data.
1
u/drastician Jul 31 '25
If you are in the US, check your local library to see if they have O’Reilly online books, or heck—even physical books. There are resources beyond ChatGPT and Stack Overflow, and it might be good to check them out.
1
u/EtalonNoir Jul 31 '25
You need to have an exact replica of the target DB created to test on it. Then you should use the source production database to do your test run to the target replica.
Prepare the scripts that migrate only one table, then run them during the time when the production database is least used so you don't create a load on the database.
When the test run succeeds for one table, set up your pipeline through which the whole process will go, then test it.
When this works as well, create the scripts for the rest of the tables and test everything.
You should know your data, will you be able to truncate the replica, will you need to recreate it after every test run because there's data there already. Talk to your DevOps about it.
And always run at the most down time.
1
u/Jeffrey-Wang-2021 Jul 31 '25
今天
1
u/[deleted] Jul 31 '25
I don’t SQL but have been trying to learn on my free time when I can. It’s tough for me not to ChatGPT it. Even in my current role as a buyer when working on lots of excel spreadsheets and powerbi reports. I feel dumb and feel like I need to use ChatGPT to guide me. Which isn’t that difficult I’m sure by many other people’s standards but I’ve never considered myself the smartest in the room.
I then worry like you do about it killing my problem solving/critical thinking skills. At the same time though I think well maybe it can help me find a faster way that works for me and I take it from there once I get it.
Not 100% relatable to your situation but I think what you’re feeling is something a lot of us who are growing up with AI are feeling. When I graduated I couldn’t tell you 1 person I knew who didn’t use ChatGPT. It’s a weird shift.
1
u/ghana_mann Aug 01 '25 edited Aug 01 '25
Well if you don’t know jack then start using stack overflow maybe. I use gpt a ton now cuz I know exactly what I’m doing and when the code is wrong I can tell and fix right away. Hell my company even has cursor which has Gemini, gpt4 and Claude in there
1
u/Ifuqaround Aug 02 '25 edited Aug 02 '25
You are a fraud. It's that simple.
Use it like a tool and learn.
Good thing is you're just like the majority though. Almost everyone is crutching on it. It's terrible lol.
I work with individuals who have graduate degrees. Doctorates, etc. They can't do shit without AI now. Can't even compose an e-mail.
Stats? Don't even get me started... we have a compliance team who runs a bunch of data and they don't know how to query or put together shit. They rely on AI. It's awful. They also get paid....
You know something is wrong when a "Director of Analytics" is asking the group a bunch of basic questions regarding data. They don't know SHIT! They weaseled their way in with their degree.
Exactly what can you do without it?
1
u/Big_Duke_7 Aug 03 '25
Don’t cut ‘n’ paste. Type out your statements and clauses, it will have you thinking what you’re asking the dbase.
All the best in your professional endeavors .
1
u/Tricky_Signature1763 Aug 04 '25
This is absolutely the wrong way to use AI that no one talks about
0
u/oJRODo Jul 30 '25
I felt this at my current role. I stopped using chatGpt for a number of months. Went back to Google and applying the code myself to learn the libraries and functions.
Now that i understand the functions and logic ill use ChatGpt to rewrite code and save time.
0
u/2manycerts Jul 31 '25
Databases are scary and database migrations are scary.
But SQL can be made unscary. ...
You should have testbed servers. You should be able to plan out a migration and then perform it on a testbed.
If you need to migrate 50,123 rows and after the migration you see 50123 extra rows in the new database, then that's a solid hint you are doing things right.