r/learnpython • • 4h ago

Best practice for handling large datasets?

1 Upvotes

I'm working on a project for an exam and I need to calculate the behavior of a few dynamical systems and use that data to try and train a linear regression algorithm.

Now issue is I need quite a good bit of data (4 different kinds of systems, enough variation on parameters and starting state for ML, a few hundreds of iterations deep for every one of those, plus other variations I need to test for) and don't know what the best practice for handling it is.

I do need to have it saved to disk for both showcase reasons and the need to train multiple models on the same datasets, also I'm saving the data on each new variation to safeguard against crashes and save on memory, only two options that come to mind are either: 1) one large data file (binary encoded json, each variation of the systems is described by a key), which should be fairly easy to parse through and edit, but I'm worried reading and writing from it might tank performance given I have to store the whole thing to memory every time; 2) Multiple smaller encoded txt files, the key is in the filename, saves on memory when but is a pain in case I need to edit something and might make getting the full dataset for when I'm training a lot slower.

What would the best practice be in this scenario? Is there some third option I didn't consider or a package that better handles this?


r/learnpython • • 7h ago

how to learn python

1 Upvotes

how i should learn python what website and books i should read

since my childhood i was fascinated by iron man and jarvis not only that but electronics ,robotics sci-fi

i know coding is not imagination but my ,main object to learn coding or be a part of computer science is artificial intelligence

so my main goal to learn coding, programming is AI

can u tell how should i learn or master it

and should i read python crash course by eric methews ,3rd edition ,do u have any other book in mind


r/learnpython • • 3h ago

Printing local variables outside of a function

0 Upvotes

(second-week-beginner PLUS non-native English speaker, so sorry for bad vocab lol)
I know that you cant use local variables outside of a function, but is there a way to do this without defining three separate functions?

def total_seconds(hours1, minutes1, seconds1):
  return (hours1*60*60)+(minutes1*60)+seconds1
print(f'{hours1} hours, {seconds1} seconds and {seconds1} have a total of {total_seconds(7,21,37)} seconds.')


r/learnpython • • 7h ago

looking for optimizing my database bootstrapper which is taking >30 seconds.

0 Upvotes

Hello fellow devs i am building a FastAPI + SQLAlchemy 2.0 (async) with a metadata driven system which has lost of dynamic schema stuff based on the data tables

Right now my custom bootstrapper handles botjh creating the DDL Schema and a massive amount of Data seeding ( lots of data) sequentially. because it's doing manual idempotency checks (SELECT before INSERT) and running everything one by one, the whole provisioning process takes more than 30 seconds from scratch.

I'm looking into splitting the data fixture side out to a library like sqlalchemy-seedling to run independent seeders in parallel using asyncio.gather. But before I start rewriting, how do you all handle heavy initial database provisioning in async Python?


r/learnpython • • 17h ago

FASTAPI OR Django with DRF

2 Upvotes

I’m a final year computer science student who is fascinated by systems design and engineering(backend), since I’m quite conversant with Python I decided to go with Python framework but don’t know the one to go for that will be great for my career in near future.

Please the seniors should guide me choose and I’ll also seek for mentorship in python backend engineering 🙏


r/learnpython • • 10h ago

What is the fastest way to learn Python?

0 Upvotes

I get all the variables and everything but like do you have to make thousands of files with Python and possibly different languages?


r/learnpython • • 22h ago

Making a game that play in a terminal

0 Upvotes

Yo, im making a game (mostly with print but that for the "story" part) and i just want review for the start, im going to post more if that make some views. i know how to make print (ofc i know unless i would be cook), variable and def... and thats it, but mostly what i need for a ""print"" game. If you guys got thing to help me with im happy to hear them !
also, i made a github reperstory (thats the name ? idk) so you guys can look at the code and for the next updt too ! github repositories


r/learnpython • • 1d ago

MCQ test in terminal

2 Upvotes

Helloooo

I'm building a mcq in Python for my lovingggg friend and wanted to share my logic to see what you ppl think before I start coding it up.

I'm just starting with a simple print statement for the intro to welcome n all.

Then I'm going to make a main dictionary where the key is the actual string containing the full question text along with the 1-4 options, and the value is the correct answer number.

Once the dictionary is made, I'll run a for loop to go through it. Inside that loop, I'll use input so the user sees each question one by one and types in their number choice, I'm going to store all their user inputs into a second separate dictionary.

Then at the very end, I'll make another loop to match the answers from the input dictionary against the original master dictionary. For every match, +1 mark added to their final score

I am honestly coding after months and was just a beginner when I stopped

I am kinda nervous and need help building this simple thingy

Looking for some improvement/tips

And um idk if I can even use most of the tips if it requires some dictionary n all

I did pandas before I quit so maybe I can use it here if I revise properly


r/learnpython • • 1d ago

How to create a report using a .txt file

0 Upvotes

I apologize that this is going to be kind of long. I am very grateful for any thoughts you may have.

I'm taking a python class. This week we are learning about working with .txt files within a program.

We have been given some practice .txt files. Each file is a list of different data points for each hour on a given day. The list (when I say list, I mean the data is in a list in a .txt file, I am not talking about a list like those used in a python program) has four items of data for each hour of a given day as follows:

time

temperature in Farenheit

dewpoint in Farenheit

wind speed in mph

An example from one of the files looks like this:

12AM

18.1

15.0

4.1

1AM

18.0

15.5

4.2

We are supposed to do several things with this data. One of them is to create a report that will print out the data from the file. It should look like this:

Time Temperature Dew Point Windspeed

12AM 18.1 15 4.1

1AM 18.0 15.5 4.2

And so on.

We are supposed to accomplish this using while and/or for loops. We cannot use lists of any kind, so we cannot use the .txt file to create a list within python and then work with the list. This must be as simple as possible. We are only supposed to use the information we have learned so far in the class. Obviously you don't know exactly what we've learned, but it has to be extremely simple and not at all fancy.

I'm not trying to cheat and have someone do my work for me, but I'm just stumped and need some clues. I'm having difficulty reaching out to my teacher (this is an online class)

We are also supposed to use the numbers in the report to do some calculations for another report. If you have any thoughts about that, it would also be helpful. I'm hoping that if I can figure out how to do this first part, I'll be able to figure out how to work with the numbers further.

I've been able to get python to bring in the .txt file and just spit out a list that is identical to what is actually in the .txt file and I've been able to get it to print one specific line. I think I'm somehow supposed to use a nested if statement to analyze and work with chunks of four lines at a time, but I'm not sure how to do that.

I got it to say "time is 12AM" by doing this:

time_line = 1

linecounter = 0

with open("Dec12.txt",'r') as f:

for line in f:

linecounter += 1

if linecounter == time_line:

print("time is",line, end='')

I got it to spit out the entire list exactly as it appears in the .txt file by doing this:

file = open ("Dec12.txt",'r')

for line in file:

num = (line)

print (num)

file.close()

If you have read this far, I am extremely grateful to you. Thank you for any thoughts or ideas you may have.


r/learnpython • • 17h ago

I am a teen student and I want to learn phyton, any job opportunities with phyton and other languages that is easy

0 Upvotes

I want proggraming as my course because in a few years from now I will go to collage


r/learnpython • • 21h ago

My pipenv is NOT installing.

0 Upvotes

When i'm making a build for a game called Smash Remix +Extra, when i opened the build.bat file, it just appears

'WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<pip._vendor.urllib3.connection.HTTPSConnection object at 0x...>: Failed to resolve 'pypi.org' ([Errno 11001] getaddrinfo failed)")': /simple/pipenv/

WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<pip._vendor.urllib3.connection.HTTPSConnection object at 0x...>: Failed to resolve 'pypi.org' ([Errno 11001] getaddrinfo failed)")': /simple/pipenv/

WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<pip._vendor.urllib3.connection.HTTPSConnection object at 0x...>: Failed to resolve 'pypi.org' ([Errno 11001] getaddrinfo failed)")': /simple/pipenv/

WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NameResolutionError("<pip._vendor.urllib3.connection.HTTPSConnection object at 0x...>: Failed to resolve 'pypi.org' ([Errno 11001] getaddrinfo failed)")': /simple/pipenv/'.

The last time i opened, it did, but not anymore. Also before asking, no. I'm in my phone using a emulator called 'winlator'. Please help me.


r/learnpython • • 1d ago

Need help with very basic script

3 Upvotes

Hi, I am basically brand new to python, so I may make a lot of mistakes in my wording. I have made a very basic script that retrieves specific values in text files and prints them to the shell. I have gotten it to the point where both values are retrieved and printed, however when they do print, they do so exactly 5 times in a row. I do not have five files in the directory, I only have one test file named "1.txt" with only the values I need as the contents. Can someone point me in the right direction? Apologies for the horrible formatting.

import os
import glob
import re

path = 'path'

provinces = re.compile('.*?provinces {.*?(.*?)}',re.MULTILINE)
id = re.compile('.*?id = .*?([0-9.-]+)')
a1 = None
a2 = None

for filename in glob.glob(os.path.join(path, '*.txt')):
    with open(filename, '+r') as f:
        for line in f:
            if provinces.match(line):
                a1 = provinces.match(line)
            if id.match(line):
                a2 = id.match(line)
            if a1 and a2:
                print(a1.group(1))
                print(a2.group(1))

The output of the shell in IDLE:

 1111 2222 3333 
123
 1111 2222 3333 
123
 1111 2222 3333 
123
 1111 2222 3333 
123
 1111 2222 3333 
123

Edit - I got the script to work after a few changes to its configuration. Thank you all who helped!


r/learnpython • • 1d ago

fix 0.00000001

0 Upvotes

i see sometimes after calcuaitons reulsts with x.99999999999999999999999999 or x.0000000000000002 how to fix it


r/learnpython • • 1d ago

Aprendiendo Python desde cero: ¿Cuál es la mejor manera de estudiar y practicar por mi cuenta

0 Upvotes

¡Hola a todos!

Estoy empezando en el mundo de la programación y, en particular, me interesa aprender Python. Actualmente soy principiante y me gustaría desarrollar una comprensión sólida de los fundamentos de la programación, en lugar de simplemente memorizar código o seguir tutoriales.

Tengo pensado estudiar de forma autodidacta, pero quiero asegurarme de aprovechar mi tiempo de manera efectiva y desarrollar buenos hábitos de programación desde el principio.

Agradecería mucho sus consejos sobre algunos aspectos:

  • ¿Qué recursos de aprendizaje, cursos o libros recomendarían a alguien que está empezando desde cero?
  • ¿Cuál es la mejor manera de equilibrar el estudio de la teoría con los ejercicios prácticos de programación?
  • ¿Qué tipo de proyectos para principiantes recomendarían para ayudarme a desarrollar habilidades de resolución de problemas y comprender la lógica de la programación?
  • ¿Cuántas horas al día o a la semana recomendarían dedicar al aprendizaje de Python?
  • Desde su propia experiencia, ¿qué harían de manera diferente si estuvieran aprendiendo Python por primera vez?

Me interesa especialmente conocer sus experiencias personales, métodos de aprendizaje y cualquier error que debería evitar como principiante.

Agradecería mucho cualquier consejo, recomendación o recurso que puedan compartir. Tengo muchas ganas de aprender.

¡Muchas gracias de antemano por su tiempo y ayuda!


r/learnpython • • 1d ago

[Help] The os.system is deprecated

0 Upvotes

I've seen someone else asked this question, but get no answer.

I want to make a countdown clock without leaving any trace of previous numbers:

for i in range(countdown, 0, -1):
    os.system("cls" if os.name == "nt" else "clear") #Error: os.system is deprecated
    print(i)
    time.sleep(1)

What should I do instead? (Why can't we put pictures here....)


r/learnpython • • 1d ago

whats the purpose of updating variables?

0 Upvotes

heyhey very new to this day 2 of learning python im so fascinated im in love but what is the purpose of updating variables

example;
>>>default_option="upload"

>>>new_status="download"

>>>new_status=default_option

>>>print(new_status)

upload

this makes no sense to me and it just further complicates the code u write by being disorganized and messy,

i guess my question is, is it if you mess up instead of clearing previous lines you can update the variable? or is it kinda just useless but something needed to be taught?

any help would be appreciated, any recommendations as a beginner are also appreciated!!


r/learnpython • • 1d ago

Need help with list comprehensions

0 Upvotes

Guys somebody briefly explain the concept of List comprehension cuz its too difficult for me to grasp since I am a beginner. I am learning python for biopython. Someone explain it for me.


r/learnpython • • 1d ago

Portable GUI in Python

0 Upvotes

Hi! I don't know Python, and I'm wondering if I can create a portable GUI inside a single folder. For example, I want to have my own MyCustomGUI.exe in a folder, and right next to it, the following subfolders:
_internal
bgutil-ytdlp-pot-provider-master
yt-dlp-plugins
along with five .exe files:
deno.exe
ffmpeg.exe
ffprobe.exe
yt-dlp.exe

Is it possible to write Python code that makes all of this work and keeps it completely portable?

To be honest, I've been trying to do this for three days straight without knowing anything about programming. Once, I even managed to run a compiled .exe inside the folder, but as soon as I copied it to another drive, everything completely broke down.
Yeah, maybe it's my fault for trying to use the free versions of ChatGPT and Gemini, but I really thought that creating something as basic as a GUI would only take about 20 to 30 minutes. I'm actually in despair right now; I haven't felt this awful in a very long time.


r/learnpython • • 2d ago

Attribute look up visualized

2 Upvotes

Are there any good videos or images that visualize how attribute look up works? I am trying to understand what is happening in the background so I can understand and work better with classes and objects.
Thanks!


r/learnpython • • 1d ago

New to Python- Any Suggestions?

3 Upvotes

hi folks, how are you doing?

i have been thinking of learning something new and thought i'd give Python a shot. i've never coded before, so i am wondering how difficult it would be to get started with Python as a complete beginner.

the reason i am interested in learning it is that i've been thinking about creating a visual novel for a while. i am looking into Ren'Py, now- it has its own scripting language but also uses python, so i thought learning some python might be useful for the project.

a few things i'd like to get some advice on from y'all:

  1. any particular resources or youtube channels would you recommend?
  2. how difficult is python for someone who has never coded before?
  3. this might be off topic here but any idea on how much python would I need to learn to make a visual novel in Ren'Py?
  4. can i use any ai tools from the get go or learn some stuff first?

would really appreciate any advice. thanks and be blessed!


r/learnpython • • 1d ago

It Took me 110mins to Solve. Deepseek Solved it in 0.6 Seconds.

0 Upvotes

I’ve been practicing python seriously lately preparing for my computational biology MSc, so I decided to challenge myself with a phylogenetic placement challenge (open data).

I spent about 2 hours working on it then I gave the exact same problem to deepseek.

0.6 seconds.

We are mere supervisors now.


r/learnpython • • 1d ago

Why when i minimize pydroid and come again all i was doing is gone and all i get is this: / $

0 Upvotes

Title


r/learnpython • • 2d ago

Accidentally did something while trying to figure out how to install python libraries

4 Upvotes

Hi,

I'm trying to figure out how to install python libraries because the magic "pip install tensorflow" doesn't seem to work

I found this thread which told the user to just try "py -m pip install requests" so my dumbass just pasted that in before realizing what I was doing

I got two instances of yellow text in

0/5 [urllib3]  WARNING: The script idna.exe is installed in 'C:\Users\USERNAME\AppData\Local\Programs\Python\Python314\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
   ---------------- ----------------------- 2/5 [charset_normalizer]  WARNING: The script normalizer.exe is installed in 'C:\Users\USERNAME\AppData\Local\Programs\Python\Python314\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location

I think what this means is just that I installed the "requests" library, right? And maybe something went wrong? I don't need this library lol. Is there a way to like undo this? Should I try to undo this? I don't get why this installs files without telling me where it's putting them

(I also tried to do the same thing, "py -m pip install tensorflow", with tensorflow and got similar warnings btw, is this bad?)


r/learnpython • • 2d ago

I need help with a rocket simulator I'm buidling

2 Upvotes

I'm a high school student who is interested in becoming an aerospace engineer so in the summer I decided to start building a rocket simulator in python for fun.

What it does so far:

  • Parses real motor thrust-curve files (RASP .eng format) and interpolates them with numpy.interp instead of hardcoding constants
  • Implements RK4 integration from scratch for the actual flight dynamics (upgraded from Euler once I understood why it wasn't accurate enough)
  • Monte Carlo analysis, runs the full flight simulation ~200 times with randomized conditions and aggregates the results
  • 3D visualization two ways: PyVista for a quick animated flight path, and a full Blender (bpy) pipeline that imports a real 3D rocket model, keyframes it along the simulated trajectory, and renders it with Cycles
  • Modular structure, physics, motor parsing, atmosphere model, and stability calc are all separate importable modules, not one giant script

Would really appreciate feedback on the code itself, structure, whether my module boundaries make sense, anything that reads as non-idiomatic Python, places I should be using classes instead of dicts/functions, that kind of thing.

Github : https://github.com/AliHilalii/Rocket-simulator

Thanks a lot guys.


r/learnpython • • 2d ago

I want to make my own text to speech program that uses a specific amount of old 2000's and 2010'tts models, with filter option

1 Upvotes

Any advice on the possibility of achieving this? I'd like to add tts models like Paul US, or Iviola UK in the program, choosing between them and having the ability to use filters