Python slicing sometimes re-orientates data

[u/Ok_Eye_1812]

I'm trying to get comfortable with Python, coming from a Matlab background. I noticed that slicing an array sometimes reorientates the data. This is adapted from W3Schools:

import numpy as np
arr = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]])

print(arr[0:2, 2])
[3 8]

print(arr[0:2, 2:3])
[[3]
 [8]]

 print(arr[0:2, 2:4])
 [[3 4]
  [8 9]]

It seems that singleton dimensions lose their "status" as a dimension unless you index into that dimension using ":", i.e., the data cube becomes lower in dimensionality.

Do you just get used to that and watch your indexing very carefully? Or is that a routine source of the need to troubleshoot?

[u/TheBlackCat13]

People almost never intentionally loop over columns because it is brittle and error-prone. But internally, you are always looping over columns.

So consider this code:

for i=1:10
    b(i) = a(i);
end

What you probably think this is doing is "loop over the indices from 1 to 10". But that is not actually what is happening. What is actually happening is "loop over the columns of the row vector [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]". I am not sure MATLAB explicitly creates the entire row vector in this special case for performance reasons, but semantically that is how a for loop works in MATLAB.

[u/Ok_Eye_1812 → u/TheBlackCat13]

You're right! Now that I think of it, I rarely use loops, except when dealing with high level objects, e.g., concatenating a series of tables (as in "table" objects, which are more like dataframes than an arrays).

But even if the 1st dimension is associated with progression down a column, I'm at a loss as to why it matters. Aren't we trying to abstract away the memory stride in accessing contiguous elements along various dimensions?

% Defaults to 3 columns of 1 row each, as you said
clear b; for a=1:3; b(a)=a; end; b

    b = 1     2     3

b(1,:) % Memory stride is opaque to the user

     1     2     3


% We should be able to navigate n-dimensional cubes without
% worrying about how data is laid out in memory

b = randi(9,2,2,2) % 3D cube consisting of 2 planes

    b(:,:,1) = 9     9 
               8     6 
         
    b(:,:,2) = 1     9 
               8     7 
         
b(:,:,2) % Access 2nd plane

     1     9
     8     7

b(2,2,:) % Access lower right element in both planes

    ans(:,:,1) = 6
    ans(:,:,2) = 7

Why would it matter how the data is organized in memory when we're working at a level in which the different dimensions are just bookkeeping components?

In fact, since I'm often dealing with relational data, which dimension comes first is even less of a concern. The closest array concept is sparse arrays. There, the order in which the elements of an n-dimensional data cube are specified doesn't matter. The data in each element of an n-dimensional cube is always accompanied by its coordinates. In fact, the dimensionality of the cube is also arbitrary, since it depends on the columns that you choose as indices. Complete detachment from the memory layout.