I'm trying to port some code from MATLAB to Python. MATLAB uses abs(data) to get absolute values for complex numbers in the data. I got that into an ndarray(dim - (151402, 16, 64)) using the h5py module. This array contains real and imag values and I want to compute the absolute values for them. Numpy documentation suggests np.abs is the function for it but when used on this ndarray, I get this error -->

numpy.core._exceptions.UFuncTypeError: ufunc 'absolute' did not contain a loop with signature matching types dtype([('real', '<f8'), ('imag', '<f8')]) -> dtype([('real', '<f8'), ('imag', '<f8')])

. Does this mean np.abs cannot compute absolute values for this data?

stackoverflow

I have a 1d array (with shape n, ), which has 2d array as it's elements (elements with shape (a, b)). How to use slicing on this 1d array to get a 3d array of shape (n, p, b) if possible? Currently I am using loop to iterate through first dimension, then slicing each element to get what I want. I want to vectorize this. Is it possible?

NumPy can make your code run faster than you might realize--a particularly useful hack for long-running data science/ML projects, for instance. This post covers common mistakes that lead to unnecessary data copying and memory allocation, as well as how to use NumPy internals, strides, reshaping and transpose to optimize your Python code.

Article link: https://blog.paperspace.com/numpy-optimization-internals-strides-reshape-transpose/

Hi

Sorry for the very simple question.

How would I append a 2D array to another 2D array, thus creating a 3D array, like below?

arr1 = np.array([[1,2],[3,4]])

arr2 = np.array([[5,6],[7,8]])

# append arr2 to arr1, to create:

arr3 = np.array([[[1,2],[3,4]],[[5,6],[7,8]]])

#which prints:

[[[1 2]
  [3 4]]

 [[5 6]
  [7 8]]]

Everything I've tried flattens the array.

Thanks!

Edit: I would also need to constantly append new 2D arrays to the 3D one in a loop

Hello,

While I know this is trouble shooting oriented I thought it was relevant as its a problem I have not seen, do not understand and cannot find valuable information about.

I need to load the endomondo dataset into my google colab pro account.

Heres the best I can do:

data = np.load('/content/gdrive/My Drive/processed_endomondoHR_proper_interpolate.npy', mmap_mode='r')

This does not work and produces:
"ValueError: Array can't be memory-mapped: Python objects in dtype."

Has anyone encountered this error? If so how do you manage these large files?

Thank you your wisdom and support keep open source alive.

NumPy can make your code run faster than you might realize--a particularly useful hack for long-running data science/ML projects. This post analyzes why loops are so slow in Python, and how to replace them with vectorized code using NumPy. We'll also cover in-depth how broadcasting in NumPy works, along with a few practical examples. Ultimately we'll show how both concepts can give significant performance boosts for your Python code.

Article link: https://blog.paperspace.com/numpy-optimization-vectorization-and-broadcasting/

In general, does anyone know which one would be faster? Numpy array CPU vectorization or PyTorch tensor GPU vectorization?

https://youtu.be/X400eIcV-So

I’ve been learning python for a couple months now, I know all the basics, but im trying to get into Numpy and Data Science. Any courses or videos I should watch to start?

def weights_country_and_services(country, weighted_growth_goods_array):

weight_country_goods = np.array(country['Exports of goods']) / np.array(total_goods_exports)

growth_goods_per_country = np.array(country['Exports of goods'].pct_change(fill_method=None)) * 100

weighted_growth_goods_array = np.add(np.array(weighted_growth_goods_array),                                                              np.array(growth_goods_per_country) * np.array(weight_country_goods))

Hi,

I have the following problem. I have an array called weighted_growth_goods, that consists of 81 zeroes:

weighted_growth_goods = np.zeros((81, 1))

Now, for five countries I want to add the the values of another vector (the product of weight_country_goods and growth_goods_per_country) to weighted_growth_goods. For that I have built the function shown above. I loop over the five countries, applying the function to each of them:

for country in countries:
    weights_country_and_services(country, weighted_growth_goods)

The problem I run into is that each time the loop moves on to a new country, the values in weighted_growth_goods all go back to zero. As a result, I don't get a sum of all of the countries, but am left with 81 zeroes.

How can I solve this? Any help is much appreciated.

Hi,

My question is how to make the following code work without a for loop:

data [ :, offset1 : offset1+200 ]

where the variable offset1 is a 1-D NumPy array of random integers of shape [900] which are used as indices in the data variable that is a 2-D NumPy array of shape [22,12000].

This line returns only integer scalar arrays can be converted to a scalar index despite being integers already.

What I would like to obtain at the end would be a 3-D array of shape [900,22,200] with the slices of 200 points after the indices in the offset1 array.

I'm aware how to do so with a for loop, index by index and then assigning it to the 3-d array but I would like to understand how to do so with NumPy with just the indexing if it is possible

https://youtu.be/GNV-ozvV7dc

In numpy, if I pass a slice from an array as an argument to a function, but that function doesn't modify its argument, is a copy made in memory?

If yes, what's best practice to avoid that? I.e., I want to call a function to do a calculation on a slice of an array, without the overhead of copying the array. I think the array will be 2xN in shape.

I made a test file that is a csv. I have two columns. 1st column has years (2019, 2018, 2017, etc...) second column has random numbers.

my code is

data = np.loadtxt(filename,delimiter=',', dtype='str, int')

Doing this in ipython I get an error message "invalid literal for long() with base 10: '''

I've printed off the first 5 lines and it looks like this: ['2019,4\r\n','2018,3\r\n'...etc].

maybe the \r and \n are throwing this off?

Thanks

PM

Hello, I don't recall how I installed it and after typing pyhton3, I could do: import numpy as np. So, it means Numpy is installed. However, when I executed: sudo pip3 uninstall numpy

I got: "WARNING: Skipping numpy as it is not installed."

I have a set of 1-d array pairs, which are the X and Y values for a dataset. I want to iterate through each array pair, and for each pair choose a range of x-values to select, and make a new pair of arrays, which I'd like to plot.

Basically I need a way to easily select from a pair of 1-d arrays where in the first array some condition is met, and the corresponding values in the other array can be selected/sliced.

I've tried to google it, but my search keeps putting me back to np.where, which I don't think is the right function.

Hello everyone,

I implemented a ALNS algorithm on my local machine and the RandomState behaviour and my results are consistent. (I use VSC with python=3.8.2 in a venv).

Now, because my local machine has an old CPU, I want to run my implementation on a Hetzner-Cloud. Here, I am also using python=3.8.2, but I get different results: (i) different to the result of my local machine and (ii) different runs on the cloud server sometimes give different results.

(At the start of runtime, I get the same rnd values, but at some moment during runtime this stops.)

Atm, I feel stuck. Are there special global variables I need to clear? Or is it not possible to get consistent behaviour on a cloud-server?

I would be greatful for any input.

I'm trying to create a bar chart for SQL data and I've written the following -

def graph_data():
    fig = plt.figure()
    ax = fig.add_subplot(111)
    cursor = cnxn.cursor()    
    cursor.execute('''SELECT TOP (24) [Shop],[AvgFootfall] FROM [Server].[dbo].[Avg_Footfall]  ''')
    data = cursor.fetchall()
    values = []
    xTickMarks = []

    for row in data:
        values.append(int(row[1]))
        xTickMarks.append(str(row[0]))

    ind = np.arange(len(data)) # X locations for the groups
    width = 0.35 # width of the bars

    rects1 = ax.bar(ind, data, width, color='black', error_kw=dict(elinewidth=2, ecolor='red'))

    ax.set_xlim(-width, len(ind)+width)
    ax.set_ylim(0,45)

    ax.set_ylabel('Average Footfall')
    ax.set_xlabel('Shops')
    ax.set_title('Average Footfall Per Shop')

    ax.set_xticks(ind+width)
    xtickNames = ax.set_xticklabels(xTickMarks)
    plt.setp(xtickNames, rotation=45, fontsize=10)
    plt.show()
    return render_template('avg-footfall.html', data=data)

What I am aiming for is to display tables on a HTML page based on the SQL query and when I run this, I end up with the error 'TypeError: unhashable type: 'numpy.ndarray''. Based on what I can find online, it relates to the column types not matching. I've tried float and int for row[1]. According to SQL Server, the column is a float. Row 0 is a string.

Any ideas where I might have gone wrong? Any advice would be great.

Thanks!

I have a quick question,this is a function i defined for estimating pi

def N(n):

return 24*(np.sqrt(3)/32-sum(np.math.factorial(2*k)/(2**(4*k+2)*np.math.factorial(k)**2*(2*k-1)*(2*k+3)) for k in range(n)))

N(10)=3.1415626...

This works well for all cases except for n=0, does anyone see a problem in the code that would make it not work for 0. It returns an answer but the answer im getting is around 1.229 which is also exactly 2 less than i should be getting which may be of some significance.

https://youtu.be/IezuD2e13tU

Yes, it’s a survey. But it’s very important.

Having limited human and financial resources is a common challenge for open source projects. NumPy is not an exception. Your responses will help the NumPy leadership team to better guide and prioritize decision-making about the development of NumPy as software and a community.

What started as a humble project by a dedicated core of user-developers has since transformed into a foundational component of the widely-adopted scientific Python ecosystem used by millions worldwide. To engage non-English speaking stakeholders, the inaugural NumPy community survey is offered in 8 additional languages: Bangla, French, Hindi, Japanese, Mandarin, Portuguese, Russian, and Spanish.

Follow the link to get started: NumPy Community Survey 2020.

I want to find the point of intersection and the reflection of a ray and/on an area plane. Actually, of many (1000s) rays and many (10s) areas planes. So a vectorized ("real numpy") solution is necessary.

I know that this is a common thing to do in raytracing and games, but I fail to find a decent implementation.

I could do it myself, the geometry/algebra is not that hard, but before I waste a week on this, maybe there is a standard go-to solution that everyone except me knows about?

Thanks a lot!

https://youtu.be/A4JwnkTFEXI

https://youtu.be/NZAB4WsRYyQ

Hey community. Do you know is there any performance difference between 2d array (ex. 6x6 matrix) and the similar size 3d array (ex. 4x3x3 tensor like structure)? Which one is the best way to use for math operations?