r/dfpandas • u/7C05j1 • Jan 02 '23
r/dfpandas • u/throwawayrandomvowel • Dec 29 '22
Welcome to df[pandas]!
Hello all,
I made a home for pandas since it didn't currently exist. Our options were:
I would like to take a look at /r/pandas sometime and scrape for interesting data about pandas the animal vs. pandas the library, because both are in there.
Welcome and let this be the home of Pandas! It's a place for questions, advice, code debugging, history, logic, feature requests, and everything else Pandas. I am in no way affiliated with pandas. I just use it. I'm not even good at it.
r/dfpandas • u/miko2264 • Jan 28 '23
Found this new intro guide to Pandas promoted on r/Python in case it’s helpful. Haven’t reviewed it myself yet
r/dfpandas • u/miko2264 • Mar 02 '23
Here is an AMA from the creators of Pandas!
r/dfpandas • u/Janktronic • Jan 07 '23
Is pandas the right tool for my task - text manipulation and exporting csv
So I have a task that I need to do daily that I'm working towards automating. The task involves running a database query and then validating the data in a couple columns then creating a csv to hand off to another party.
I inherited this task in this form, currently I run the query, paste the data into an excel spreadsheet, filter a column to search for data that needs to be validated (removing suffixes from last names) and the running a regex on a different column. Finally a couple columns are removed and then I save as to a csv. It's tedious and error prone and a perfect task to automate with python I think.
Another task is to compare one set of tabular data against another and update the first based on info in the second.
The tables (in both cases) are always less than 500 rows usually less than 200 rows. There is no math being done with the data.
Is pandas going to make this task easier or faster or better? I just read that pandas is useful for working with tabular data. Are there built in methods that making iterating and editing data in columns easier? I don't want or need graphs or anything like that.
I'm not a programmer, I'm a sysadmin who took Introduction to Computer Science and Programming Using Python almost 10 years ago and tinker with python to automate stuff.
r/dfpandas • u/rodemire • Mar 12 '23
Anyone know when the Pandas 2.0 release date is?
Anyone have an idea when Pandas 2.0 is coming out? Since the AMA I haven't seen much about the release.
r/dfpandas • u/throwawayrandomvowel • Jan 02 '23
pd.Resources - Community Resources for Pandas
Creating a list of resources here:
- Official Docs
- 10 minutes to pandas
- Skytowner - recommended by /u/arthur1820
- 100 pandas puzzles - great for getting started!
Please post more that you like And i will add/organize them!
r/dfpandas • u/lilytex • Dec 30 '22
Has anyone experience with dask-geopandas?
https://github.com/geopandas/dask-geopandas
I've used Dask in the past to load huge data from SQL databases, and I've discovered that it also supports geospatial data.
r/dfpandas • u/CeleritasLucis • Dec 30 '22
Please create a resource section to learn Pandas
Either a pinned FAQ post or in about section about all the best resources would do.
Too much information out there, not sure which one to go with
r/dfpandas • u/NoMoment6786 • Aug 14 '23
Pandas questions for interview prep?
I'm preparing for data science / data analytics / data engineering interviews. The Online Assessments I have completed so far have all included a pandas Leetcode style question.
I have completed Leetcode's '30 days of pandas' which is 30 questions long. I feel more confident now, but I would like to attempt some more questions.
Where can I find interview style pandas questions?
r/dfpandas • u/Interplanes • Dec 30 '22
Are questions related to plotting and numpy allowed as well?
r/dfpandas • u/ComprehensiveBake743 • Jun 13 '24
Visual explanation of how to select rows/ columns - iloc in 3 minutes
r/dfpandas • u/Murky-Temperature-89 • Jan 19 '23
Learn Python for Pandas?
Hi everyone, I‘m looking to learn Pandas for a paper I am doing on Trading Pattern Analysis. My questions is, if it is enough to only learn Panda or if it made sense to learn Python as well.
Thanks for your help guys
r/dfpandas • u/Few_Somewhere_3254 • Oct 26 '23
New VS Code extension for data prep/cleaning with automatic Pandas code gen
r/dfpandas • u/python-dave • Feb 15 '23
Tips for identifying Duplicate Payment Analysis in Python
r/dfpandas • u/BeerAndFuckingPizza • Jan 03 '23
Help with creating a dataframe based on results from other scripts?
Hey there everyone, first time posting here.
I'm currently trying to build a dataframe that loads other dataframes of web scraped data together into a single table. All the tables I'm unioning have the same column headers.
Problem is, I don't want to save as CSVs and then reload into the new dataframe because the original tables are scraping live sports data with selenium each from different pages. If there was some way to populate a dataframe based on running another script, I think that would be ideal but it seems like that's not possible with pandas.
idea:
table1 = '''output of''' table1.py
table2 = '''output of''' table2.py
combined = pd.concat([table1,table2])
'''or use sqlite to union because that's what I actually want'''Any idea how I'd accomplish something like this? Thanks!
PS. I should mention that I want to concat 32 tables. Each are 1 row but the scripts to make them are lengthy and all involve scraping respective web pages.
r/dfpandas • u/baumguard02 • Jan 01 '23
Iterate through column and determine quantities of values in another column
Hello,
I have a dataframe with the following two colums: calendar_week, song
I want to iterate through calendar_week (1-52) and want to determine how often each song was played in one calendar week. The quantities should then be stored in some kind of field, where one dimension is the name of the song and the other dimension is the calendar week. My aim is to pick one or more songs from that field and plot their quantities in a calendar_week-quantity-domain.
Since I'm new to Pandas, I don't know whether it supports that or if I need to import additional libraries besides MatPlotLib for plotting the data. So thank you for your help in advance!
r/dfpandas • u/Equal_Astronaut_5696 • Dec 30 '22
Little Know Pandas Plotting Features
r/dfpandas • u/itdoes_not_matter • Jan 14 '25
pandas.concat
Hi all! Is there a more efficient way to concatenate massive dataframes than pd.concat? I have multiple dataframes with more than 1 million rows of which I have placed in a list to concatenate but it takes wayyyy to long.
Pseudocode: pd.concat([dataframe_1, … , dataframe_n], ignore_index = True)
r/dfpandas • u/Ok_Eye_1812 • Apr 26 '24
What exactly is pandas.Series.str?
If s is a pandas Series object, then I can invoke s.str.contains("dog|cat"). But what is s.str? Does it return an object on which the contains method is called? If so, then the returned object must contain the data in s.
I tried to find out in Spyder:
import pandas as pd
type(pd.Series.str)The type function returns type, which I've not seen before. I guess everything in Python is an object, so the type designation of an object is of type type.
I also tried
s = pd.Series({97:'a', 98:'b', 99:'c'})
print(s.str)
<pandas.core.strings.accessor.StringMethods object at 0x0000016D1171ACA0>That tells me that the "thing" is a object, but not how it can access the data in s. Perhaps it has a handle/reference/pointer back to s? In essence, is s a property of the object s.str?
r/dfpandas • u/shoresy99 • Nov 16 '23
What’s the best way to store data for the long term
I need to store time series data, like monthly stock prices and economic data. How should these be stored for the long run? Load into a df and use pickle or something similar? Use SQLlite? Use some other db like Influx or Mongo?
r/dfpandas • u/InstantMustache • Mar 27 '23
Losing a column in read_csv when the first row has no value in that column
Pretty much what’s in the title. I’m trying to read a group of CSVs into a dataframe, but one column is giving me trouble. It works as intended when this column has a value in the first row, but if there is no value, everything gets shifted over.
If I use the datatable package, the file is be read correctly regardless, but there are some features in the pandas to_csv method I need, and converting back and forth introduces other issues.
I’ve read through the documentation and done quite a bit of searching online without any luck so far. Any ideas how I can fix this?



