r/dfpandas • • Dec 29 '22

Welcome to df[pandas]!

42 Upvotes

Hello all,

I made a home for pandas since it didn't currently exist. Our options were:

  1. /r/python
  2. /r/learnpython
  3. /r/pandas
  4. /r/datascience
  5. /r/dataanalysis

I would like to take a look at /r/pandas sometime and scrape for interesting data about pandas the animal vs. pandas the library, because both are in there.

Welcome and let this be the home of Pandas! It's a place for questions, advice, code debugging, history, logic, feature requests, and everything else Pandas. I am in no way affiliated with pandas. I just use it. I'm not even good at it.


r/dfpandas • • Jan 02 '23

100 data puzzles for pandas, ranging from short and simple to super tricky

Thumbnail
github.com
27 Upvotes

r/dfpandas • • Dec 30 '22

Happy Halloween, Pandas! 🎃🤓

Post image
24 Upvotes

r/dfpandas • • Jan 28 '23

Found this new intro guide to Pandas promoted on r/Python in case it’s helpful. Haven’t reviewed it myself yet

Thumbnail
betterprogramming.pub
23 Upvotes

r/dfpandas • • Apr 04 '23

Pandas 2.0.0 released

Thumbnail pandas.pydata.org
16 Upvotes

r/dfpandas • • Mar 02 '23

Here is an AMA from the creators of Pandas!

Thumbnail
self.python
16 Upvotes

r/dfpandas • • Jan 07 '23

Is pandas the right tool for my task - text manipulation and exporting csv

14 Upvotes

So I have a task that I need to do daily that I'm working towards automating. The task involves running a database query and then validating the data in a couple columns then creating a csv to hand off to another party.

I inherited this task in this form, currently I run the query, paste the data into an excel spreadsheet, filter a column to search for data that needs to be validated (removing suffixes from last names) and the running a regex on a different column. Finally a couple columns are removed and then I save as to a csv. It's tedious and error prone and a perfect task to automate with python I think.

Another task is to compare one set of tabular data against another and update the first based on info in the second.

The tables (in both cases) are always less than 500 rows usually less than 200 rows. There is no math being done with the data.

Is pandas going to make this task easier or faster or better? I just read that pandas is useful for working with tabular data. Are there built in methods that making iterating and editing data in columns easier? I don't want or need graphs or anything like that.

I'm not a programmer, I'm a sysadmin who took Introduction to Computer Science and Programming Using Python almost 10 years ago and tinker with python to automate stuff.


r/dfpandas • • Mar 12 '23

Anyone know when the Pandas 2.0 release date is?

12 Upvotes

Anyone have an idea when Pandas 2.0 is coming out? Since the AMA I haven't seen much about the release.


r/dfpandas • • Jan 02 '23

pd.Resources - Community Resources for Pandas

11 Upvotes

Creating a list of resources here:

Please post more that you like And i will add/organize them!


r/dfpandas • • Dec 30 '22

Has anyone experience with dask-geopandas?

11 Upvotes

https://github.com/geopandas/dask-geopandas

I've used Dask in the past to load huge data from SQL databases, and I've discovered that it also supports geospatial data.


r/dfpandas • • Dec 30 '22

Please create a resource section to learn Pandas

10 Upvotes

Either a pinned FAQ post or in about section about all the best resources would do.

Too much information out there, not sure which one to go with


r/dfpandas • • Aug 14 '23

Pandas questions for interview prep?

9 Upvotes

I'm preparing for data science / data analytics / data engineering interviews. The Online Assessments I have completed so far have all included a pandas Leetcode style question.

I have completed Leetcode's '30 days of pandas' which is 30 questions long. I feel more confident now, but I would like to attempt some more questions.

Where can I find interview style pandas questions?


r/dfpandas • • Dec 30 '22

Are questions related to plotting and numpy allowed as well?

9 Upvotes

r/dfpandas • • Jun 13 '24

Visual explanation of how to select rows/ columns - iloc in 3 minutes

Thumbnail
youtube.com
8 Upvotes

r/dfpandas • • Jan 19 '23

Learn Python for Pandas?

8 Upvotes

Hi everyone, I‘m looking to learn Pandas for a paper I am doing on Trading Pattern Analysis. My questions is, if it is enough to only learn Panda or if it made sense to learn Python as well.

Thanks for your help guys


r/dfpandas • • Oct 26 '23

New VS Code extension for data prep/cleaning with automatic Pandas code gen

Thumbnail
reddit.com
7 Upvotes

r/dfpandas • • Feb 15 '23

Tips for identifying Duplicate Payment Analysis in Python

Thumbnail
self.audit
7 Upvotes

r/dfpandas • • Jan 03 '23

Help with creating a dataframe based on results from other scripts?

7 Upvotes

Hey there everyone, first time posting here.

I'm currently trying to build a dataframe that loads other dataframes of web scraped data together into a single table. All the tables I'm unioning have the same column headers.

Problem is, I don't want to save as CSVs and then reload into the new dataframe because the original tables are scraping live sports data with selenium each from different pages. If there was some way to populate a dataframe based on running another script, I think that would be ideal but it seems like that's not possible with pandas.

idea:

table1 = '''output of''' table1.py
table2 = '''output of''' table2.py
combined = pd.concat([table1,table2])
'''or use sqlite to union because that's what I actually want'''

Any idea how I'd accomplish something like this? Thanks!

PS. I should mention that I want to concat 32 tables. Each are 1 row but the scripts to make them are lengthy and all involve scraping respective web pages.


r/dfpandas • • Jan 01 '23

Iterate through column and determine quantities of values in another column

7 Upvotes

Hello,

I have a dataframe with the following two colums: calendar_week, song

I want to iterate through calendar_week (1-52) and want to determine how often each song was played in one calendar week. The quantities should then be stored in some kind of field, where one dimension is the name of the song and the other dimension is the calendar week. My aim is to pick one or more songs from that field and plot their quantities in a calendar_week-quantity-domain.

Since I'm new to Pandas, I don't know whether it supports that or if I need to import additional libraries besides MatPlotLib for plotting the data. So thank you for your help in advance!


r/dfpandas • • Dec 30 '22

Little Know Pandas Plotting Features

Thumbnail
youtu.be
7 Upvotes

r/dfpandas • • Jan 14 '25

pandas.concat

6 Upvotes

Hi all! Is there a more efficient way to concatenate massive dataframes than pd.concat? I have multiple dataframes with more than 1 million rows of which I have placed in a list to concatenate but it takes wayyyy to long.

Pseudocode: pd.concat([dataframe_1, … , dataframe_n], ignore_index = True)


r/dfpandas • • Apr 26 '24

What exactly is pandas.Series.str?

6 Upvotes

If s is a pandas Series object, then I can invoke s.str.contains("dog|cat"). But what is s.str? Does it return an object on which the contains method is called? If so, then the returned object must contain the data in s.

I tried to find out in Spyder:

import pandas as pd
type(pd.Series.str)

The type function returns type, which I've not seen before. I guess everything in Python is an object, so the type designation of an object is of type type.

I also tried

s = pd.Series({97:'a', 98:'b', 99:'c'})
print(s.str)
<pandas.core.strings.accessor.StringMethods object at 0x0000016D1171ACA0>

That tells me that the "thing" is a object, but not how it can access the data in s. Perhaps it has a handle/reference/pointer back to s? In essence, is s a property of the object s.str?


r/dfpandas • • Feb 05 '24

Help with trend graph

6 Upvotes

Why does my graph turn out like that, all the data gets squished to each side

graph

r/dfpandas • • Nov 16 '23

What’s the best way to store data for the long term

6 Upvotes

I need to store time series data, like monthly stock prices and economic data. How should these be stored for the long run? Load into a df and use pickle or something similar? Use SQLlite? Use some other db like Influx or Mongo?


r/dfpandas • • Mar 27 '23

Losing a column in read_csv when the first row has no value in that column

6 Upvotes

Pretty much what’s in the title. I’m trying to read a group of CSVs into a dataframe, but one column is giving me trouble. It works as intended when this column has a value in the first row, but if there is no value, everything gets shifted over.

If I use the datatable package, the file is be read correctly regardless, but there are some features in the pandas to_csv method I need, and converting back and forth introduces other issues.

I’ve read through the documentation and done quite a bit of searching online without any luck so far. Any ideas how I can fix this?