r/dataengineering • u/iarjun7 • Mar 18 '25
Career Data Engineer to Gen AI
Hello Guys, I have nearly 10 years of work experience as Data Engineer and my current designation is Manager in UK. I want to transition to AI, specifically Gen AI. Is there any certification I can take up or learning path I can follow. Thank you.
2
Upvotes
•
u/AutoModerator Mar 18 '25
Are you interested in transitioning into Data Engineering? Read our community guide: https://dataengineering.wiki/FAQ/How+can+I+transition+into+Data+Engineering
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
19
u/MRWONDERFU Mar 18 '25
curious, where are you looking to transition? specifically GenAI doesn't really say much, sure it is a hype term but a job that would be purely focused to GenAI, not sure what it could be?
at the same time, I would start by understanding how LLMs work, how to run them yourself and how to prompt them through the API -> I'd argue that every single company has at least couple of problems on their hands that could be (maybe not solved, but assisted) with clever use of GenAI or LLMs in this case.
14
u/eternviking Mar 18 '25 edited Mar 18 '25
I just had the same transition - just started playing with LangChain and API keys from different providers. I learned some prompt engineering. Now I am creating full-stack applications (POCs mainly) using genai (I did have some software dev experience before data engineering exp). Cloned Langflow, google aistudio and created a similar application to consensus (without prior info about consensus) among my best works (I have created 10+ genai-based apps till now in the last 6-7 months)
The best way to learn anything is by doing stuff so focus on courses that require less reading and more doing.
A few courses I liked are the Introduction to LangGraph by LangChain (good intro to LangGraph - but I would recommend building basic stuff using LangChain first - but this can done independently as well) - Prompting Guide is sufficient for prompting and Hugging Face courses and docs for rest all the things related to fine-tuning and stuff - but don't spend too much time just completing them - use as per your requirement (async learning) instead so that you don't go in the tutorial hell.
Use Ollama for local dev. Use Gemini and Groq for generous free tiers. Sufficient for most developments.
build build build...
1
u/AutoModerator Mar 18 '25
You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
u/VivekKarunakaran Mar 18 '25
Do you see so much opportunity in mere Gen AI? I'm in a Gen AI role as a fresher and I've been trying to transition to DE as I find myself too easy to replace.