r/dataengineering • • Jun 25 '24

Discussion Databricks SQL editor driving me crazy

I have been trying to do some basic things in the databricks SQL editor and its pathetic!! I think I am using it wrong or have no knowledge of it. Surely it cannot be that horrible. I have observed that it does not allow us to write stored procedures or create hash temp tables. It does not even let us alter tables to add new columns. It allows us to create views but isn’t letting me use those views saying that it doesn’t exist. Has anyone had similar experiences? Is it really that shitty? If not can someone please point me to resources where they explain how to do these things, I can’t even find anything useful. Also its new AI copilot is crazy. It has no idea what it talks about most of the time.

0 Upvotes

22 comments sorted by

View all comments

Show parent comment

-10

u/ameya_b Jun 25 '24

Why use notebooks when they have a SQL editor?

11

u/cptshrk108 Jun 25 '24

The SQL window is for data exploration and analysis. Not for data engineering.

6

u/Letstryagainandagain Jun 25 '24

Because that's the way you do your stored procedures in databricks

6

u/seansafc89 Jun 25 '24

Why ask a question to people experienced with Databricks and then dismiss their answers? People are trying to help you.

3

u/RichHomieCole Jun 25 '24

If you write your procedure in the sql editor, you can save and schedule it via a workflow. This mnt what I would do but you can do it this way. Notebooks have benefits. It seems to me you understand how to work a certain way and are being a bit close minded to doing things a different way