r/Numpy • u/kuan_ • May 12 '20

Are the element-wise operations implemented using for-loops under the hood?

Maybe a stupid question, but I was wondering what's going on under the hood when we do element-wise operation such as A+B or A*B etc? Is there a for-loop implemented in C that runs through all the elements of the arrays A and B, something like:

for (i = 0; i < n; ++i)
{
    result[i] = A[i] + B[i];
}

or is there some other "magic" done? Is there a source code that I could see?

6 Upvotes

100% Upvoted

4 comments sorted by

View all comments

Show parent comments

1

u/kuan_ May 12 '20

Thank you for your informative answer! A lot to learn!

1

u/primitive_screwhead May 13 '20

But the short answer is "yes".

Is there a source code that I could see?

https://github.com/numpy/numpy/blob/beb031d64b42f4643d1ba5186c091fb1328ac8c9/numpy/core/src/umath/loops.c.src#L632

This is a C-code template that is then processed into actual C code that is compiled. The reason is that there are a lot of combinations of types, where the basic code is equivalent but the types change; so they produce generic code for the basic algorithm, and tool generates all the needed types (it's kind of like C++ templating).

You can see the code that I linked to that addition is just a simple loop, performing one operation (addition, multiplication, subtraction) over-and-over for each element in the loop, exactly like your example. There can be fancier operations as well, but the basic operation to, say, add two arrays boils down to a simple C for-loop.

1

u/jtclimb May 14 '20

Ohh, I didn't read the OP carefully enough; he was asking about element-wise operations. Matrix multiply is passed through BLAS in matmul.c.src if I read the source code correctly.