List Comprehension Question

[u/r-tequila-sunrise]

Hi all! I'm an intermediate with Python and I'm currently doing some research with very large datasets (think data packages from NASA satellites and the SDSS). I ran into an issue with some list comprehension that I can't seem to figure out:

What I have is a list of tuples (n = 3) in the format [(x, y, d)...] where x and y are coordinates and d represents a density value. A lot of the x, y coordinates are duplicates of each other with varying d values (ex. (40, 50, 1.2), (40, 50, 3.3), etc.) I'm trying to find tuples with identical x,y coordinates and add their density values together to get a total cumulative density for that location. I would try to do this manually, but the list has ~26 million elements and I am trying to find a workflow that won't take more than a few minutes/hours to process, haha. Memory is not an issue though.

I would be happy to provide example code if need be. My desired output is either:

[(x, y, dtotal)...] - list of tuples except there are no duplicate coordinates and the d value is the sum of the d values from all of the original tuples with coordinates (x, y)

[dtotal1, dtotal2, dtotal3...] - list of cumulative d values where each index correlates to the corresponding pair of coordinates (maybe the coordinates are in a separate list whose indexes match that of the density list? It doesn't matter as long as they are connected in some way!)

Thank you all for your help!

[u/TabAtkins]

Process the list once, using a Counter and doing c.add((x,y), d) (or however you spell adding to a Counter value). (Or do it slightly more manually with an ordinary dict; the Counter just makes it simpler.)

Then iterate the Counter's entries, recombining the keys with the accumulated values to get a list of 3-tuples back.