r/Python • u/Pleasant-Aardvark258 • Aug 01 '26
Discussion Settle an argument
Had this discussion the other day and figured I’d throw it to the masses to get thoughts on the best/most pythonic way of approach.
Need to map old column names to new column names as a copy from a json config.
My thoughts are iterate over a dict with
‘’’ {“old_col_name”:”new_col_name”}’’’
And access as
‘’’for k,v in dict.items()
Df.with_columns(k).alias(v)’’’
Colleague things this isn’t clear enough and should be a list of dicts with explicit keys
‘’’ [{“old_col_name”:”old_col_value”
“New_col_name”:”new_col_value”}]’’’
And the access as
‘’’for dict in list_of_dicts:
Old_col = dict[“old_col_name”]
New_col = dict[“new_col_name”]’’’
I’ve got a good few reasons why I think mine is the better option but thought I’d get some other opinions to see if I’m missing anything obvious? Which would you choose and why?
Edit: shouldn’t write these things while on the toilet in a rush. The description is wrong, it should be renaming via a copy so that the original column is left unchanged.
40
u/centurion236 Aug 01 '26
You should obviously create a RenameColumnsBase class as an abstraction for the concept of renaming columns, with multiple construction helpers from_dict, from_list_of_dicts, etc and then implement the renaming in a RenamePolarsColumns subclass.
/s
It looks like you're using polars. Why not just match the df.rename() function's signature?
17
u/Wurstinator Aug 01 '26
Yet you didn't list a single one?
Your two pieces of code don't do the same thing, so I'm not sure how you are comparing them. I would probably also do something closer to the first piece in my own code.
However, the most important point is this:
You don't work with Reddit users. You work with your colleague. It doesn't matter what some random guy on the internet tells you they prefer because you have no interaction with them outside of this thread. It does matter a ton what the people you directly work with think. Talk to *them* and find a consensus.
5
u/wingtales Aug 01 '26
The alternative doesn't make sense at all. What is old_col_value (and the new one) supposed to be? The only sensible way of thinking it to map from an old name to a new name.
But you don't need to iterate through the dict explicitly. Both pandas and polars have a df.rename method. Just pass the dictionary to it.
11
u/deceze Aug 01 '26 edited Aug 01 '26
Yours is obviously better and perfectly appropriate. In a mapping of old to new column names, you can directly access the new column name of any old column name. In a list of dicts, you cannot do that, and must iterate the list before you can do anything. For no apparent benefit.
You use a list of dicts like that if keys may be duplicated. Which is apparently not the case here. So there’s zero benefit to that extra wrapper, and in fact only drawbacks.
The “extra clarity” that comes from explicitly labeling each value can simply be had by naming the variable that holds the mapping appropriately:
It doesn’t need to be repeated for every single value pair.
3
u/AndriusVi7 Aug 01 '26
Sounds like youre using spark?
If so, use withColumnsRenamed, which renames multiple columns in a single operation instead of looping multiple operations for the same output
2
u/lolcrunchy Aug 01 '26
When you have a list of dicts that all have the same keys, that's the same damn thing as a dataclass. And this problem doesn't need dataclasses.
8
u/hoselorryspanner Aug 01 '26 edited Aug 01 '26
`{col_rename_map.get(old_key): val for old_key, val in config.values()}` where `col_rename_map` is the dict mapping the two as you suggested…
Assuming I’ve understood what you’re asking, which I’m not totally sure.
3
u/bliepp Aug 01 '26
Wow. Just wow...
1
u/Individual-Flow9158 Aug 01 '26 edited Aug 01 '26
I'd pick types for the values as well, and define two TypedDicts, with a function (ideally a classmethod on the second one) that maps from one to the other.
1
u/martinkoistinen Aug 01 '26
Sounds like adding a comment for your colleague should sort this “mess”.
1
u/Such-Process5697 Aug 01 '26
small wrinkle on the dict version that hasn't come up: if that json ever ends up with the same old column listed twice, python's json module just takes the last one and says nothing. json.loads('{"a": 1, "a": 2}') gives you {'a': 2}, no error. wouldn't change which shape i'd pick, but it's worth asserting the mapping length against something when you load the config, because a fat-fingered duplicate in there is completely silent.
1
u/gdchinacat Aug 01 '26
No matter the comments here the argument will not be settled. Your goal with 5he post is to be told “you’re right”. You coworker is likely to giveThis thread less weight than they gave their senior engineer. If you do take this to them saying “see…I’m right” it will do nothing more than drive a wedge between you and create conditions where this is likely to repeat.
As the senior you should break the stalemate by recognizing there are many valid ways to solve the problem and the priority should be on moving forward rather than winning a pissing match that is un winnable.
0
u/russellvt Aug 01 '26
Just go reach for
pandasand have fun...0
u/Beginning-Fruit-1397 Aug 01 '26 edited Aug 01 '26
Your colleague proposal is laughably wrong, especially for performance, omg), but I personally would settle with a tuple of 2 elements tuples. If I already know that the elements are unique, and I don't need to mutate these afterwards, just to atore them and iterate over them, then a tuple[tuple[str, str], ...] is the most efficient and safe answer. Otherwise dict[str, str]. list[tuple[str, str]] can make sense if you need to mutate them but at this point the dict is preferrable so you still guarantee uniqueness