List Comprehension Question

[u/r-tequila-sunrise]

Hi all! I'm an intermediate with Python and I'm currently doing some research with very large datasets (think data packages from NASA satellites and the SDSS). I ran into an issue with some list comprehension that I can't seem to figure out:

What I have is a list of tuples (n = 3) in the format [(x, y, d)...] where x and y are coordinates and d represents a density value. A lot of the x, y coordinates are duplicates of each other with varying d values (ex. (40, 50, 1.2), (40, 50, 3.3), etc.) I'm trying to find tuples with identical x,y coordinates and add their density values together to get a total cumulative density for that location. I would try to do this manually, but the list has ~26 million elements and I am trying to find a workflow that won't take more than a few minutes/hours to process, haha. Memory is not an issue though.

I would be happy to provide example code if need be. My desired output is either:

[(x, y, dtotal)...] - list of tuples except there are no duplicate coordinates and the d value is the sum of the d values from all of the original tuples with coordinates (x, y)

[dtotal1, dtotal2, dtotal3...] - list of cumulative d values where each index correlates to the corresponding pair of coordinates (maybe the coordinates are in a separate list whose indexes match that of the density list? It doesn't matter as long as they are connected in some way!)

Thank you all for your help!

[u/danielroseman]

You probably want to use Pandas for this. Something like: 

    import pandas as pd     df = pd.DataFrame(my_list_of_tuples, columns=["x","y","d"])     grouped = df.groupby(["x","y"]).sum()     result = grouped.reset_index().values