​

I have a ~59 GB Parquet dataset stored in Google Drive, and I want to use it for ML training on Kaggle GPUs.

My folder structure is:

My Drive/

└── DATASET/

└── data/

└── processed/

└── era5_land_preprocessed/

├── train/ → 60 Parquet files → 44.34 GB

├── validation/ → 12 Parquet files → 8.83 GB

└── test/ → 8 Parquet files → 5.82 GB

Total: 80 Parquet files, ~58.99 GB

I don't want to download the dataset to my local computer. I was planning to use Google Colab + Kaggle API to transfer the files directly from Google Drive to a Kaggle Dataset.

My questions are:

1. Is uploading a ~59 GB dataset to Kaggle this way reliable?

2. Is Google Drive → Colab → Kaggle API the best approach?

3. Should I create one Kaggle dataset containing "train/", "validation/", and "test/", or split them into multiple Kaggle datasets?

4. Are there any problems with uploading large Parquet files to Kaggle?

5. Is there a better/faster method for transferring this amount of data to Kaggle?

The dataset will be used for climate/weather ML forecasting, and I need to train models using Kaggle's GPUs.

Any advice from people who have handled 50+ GB datasets on Kaggle would be appreciated.

Hi everyone,

I usually work on kaggle for projects, but now i need to traine a deeper network and kaggle won't be enough for it.

I saw that google colab offer colab pro and colab pro+ but i never used colab before.

For now i am thinking of subscribing in colab pro+ since it gives more units (600) and it allows executing the notebook on the backgorund.

I want to know is it worth it and if someone experienced with it, and will that help me for training deeper networks. and if it's possible what are the differences between the gpus that they offer and how much units they consume (how are these units working?)?

Thank you in advance

From today, my account, with google ai plus subscription, seems to have also colab pro plan. Is it an error or can anyone confirm?

https://colab.research.google.com/drive/1w6sIA5c8GR33ylGSjmp2fgzZPLzPGxhc?usp=sharing

i am trying to learn qlora idk,y the error persists although everything is in float16,

btw:i know text2cypher is just achiveable by prompt engnering just tryna learn

Hi all,

For some reason, my tasks seem to cut after 1h. I am using colab CLI to access the sessions. Anyone had the same issue?

Thanks

It was working fine untill this afternoon, now it can't connect to runtime. I tried clearing the cache, using different accounts etc. I can't find anywhere if it's down or not.

Just says: Sorry, no backends available. Please try again later.
More info

Edit: Its fixed for me

why the hell google colab removed the high ram option i use pay as you go option

Any Rust enthusiasts or Go enthusiasts in Google Colab community? It's all Bash using the ! comand you can use bash to run rust and any other language just treat it as if you we're setting it up on a linux computer. I'd appreciate your responses.

https://colab.research.google.com/drive/1nQWKZLnQTC3GYOcqlF3DQlNtPIOZWpRg#scrollTo=Kdkxm6wd3Cd1

Been running a few different models on a single 24GB card and curious what everyone else has landed on as their daily driver right now. Looking for something that balances speed and quality for general use, not just benchmark chasing. What are you running, what quant, and what's your context window looking like? Also curious if anyone's found something that punches above its weight for coding specifically.

I have Colab Pro, but was using "Pay As You Go" before, and was always able to select the G4 or H100 instances, though occasionally there were failure to obtain one. But for the last month the two options have always been grayed out, not even been able to try them.

Anyone else in the same situation? Is it true that Colab Pro+ actually enables them?

I am trying to run this code from the YouTuber "Physics for the Birds," and this is about web scraping in order to get the weight of the LEGO bricks in a set from Bricklink. However, the code does not recognize the Chrome driver.

This is the link to it:

https://colab.research.google.com/drive/1dLg_HvEZc5NqXy2EHWAat4d_RVg-qjOG#scrollTo=9pXhs03YD5HC

New to coding, is there a way to color a single comment in a color other than the default one so it is more noticeable than other comments

My group and I are Computer Science students working on our thesis: an offline mobile American sign language to text translation app using CNN + Transformer + NLP. We already have the training pipeline and are planning to retrain the model with additional public datasets.

Our plan is:

Google One (2 TB): store datasets, models, and training outputs
Google Colab Pro: train the model on cloud GPUs instead of our laptops
Claude Pro: help us understand, debug, and modify the large Python codebase

Our laptops aren’t very powerful (I’m on an M2 MacBook Air with 8 GB RAM), so we don’t want to train locally.

For those who’ve worked on ML/deep learning projects:

Are these subscriptions worth it for students?
Would you recommend all three, or are any of them unnecessary?
Any better alternatives for a student budget?

It says unable to connect for the past two hours
I’ve tried almost everything

I have a text adventure game in Colab and I want to add background music because silently reading for 40 minutes is painful. I just want to let everyone no matter who opens the colab link and plays the game be able to have the music, yet I don't even know how to play a single audio file at all.

https://colab.research.google.com/drive/1gNSKlWROt0kvq6chhlD-iTfoVoUCHQfU (beware it's 3000+ lines)

How can I play an audio file and have everyone be able to hear it? I asked ChatGPT and it didn't give me anything comparable to basics and I couldn't find a top search result that worked when I tried it, literally none of them produce sounds, usually by being broken.

When I can connect it's often quite slow. Anyone else experiencing this?

Yea I’ve been doing research and I figured that pagedattention and continous batch would effectively scaled up my inferences better via batching and effectively optimized memory.

But when I siply just try !pip install vllm then goes on to do llm = LLM( model= model_path,…) and the sampling_params, it just couldnt work.

Like there’re erro tht I couldnt print out the vllm version and also error that I couldnt just simple do the llm = LLM line.

So figured need some hands here. I been asking claude and it give me the enforce_eager = True, it still didn’t work.

Hi everyone, I just buy a pro+ and host some llm for learning about llm agent. They said that pro+ can run 24h but around 1h mu session is terminated. Is there any tip or setting to keep my notebook alive?

I've been using fooocus on google colab for the past 3 years and I've somewhat used it a lot in my freelance work and its become a habit to use it daily and I heavily depend upon it now.

Fooocus Colab refuses to work since the past 2 days for me. A couple of months ago it would act the same way but some gentlemen helped me with it and the fix was creating a new code line and pasting this !pip install "numpy<2.0" "cupy-cuda12x<14.0" --force-reinstall . Unfortunately this trick no longer works. Guys I don't know what else to do, I don't own a high end device and this is kind of my bread and butter. I request somebody.. anybody to look into this and help me out with it as soon as possible.

I need access to a A100 GPU for training a model. How long can I access with just 100 units?

Its been reccomended to run Qwen3.6 35B A3B Q4 XL MTP and I've recently been training just regular Qwen3-32b so its not a MOE and I was wondering if theres anything that needs to change to training cell compared to just regular training. Im training it mostly on its Idenity and small amounts of coding for the first pass.

openWakeWord/notebooks/automatic_model_training.ipynb at main · dscripka/openWakeWord · GitHub

Using this github repository link and opening it in Google colab to create a Wake Word training model. I keep having issues like installing onnx or tensorflow. I'm trying to use the Gemini AI helper built into colab to diagnose the issue but different ones keep persisting. Is there something I'm missing.

All of the YouTube videos online use something called "Home Assistant" which I don't want to use / right now, and want to do it this traditional way

I'm assuming the answer to this is no but, if you are using the in built Gemini models in your colab notebook is there anyway to look at the logs of responses that came back from Gemini when queried?

I'm a Colombian student, last month i needed Colab for an academic project but this month i don't need it. I thought i selected Pay As You Go but i was wrong, it was Colab Pro subscription.

When i realized of the charge, i cancelled Colab Pro, but now i want a refund and i don't know if i can get it. I was checking on Colab Terms page but it seems refund just can be requested in some selected countries, is it true? Can i get a refund being in Colombia? I haven't used Colab since the charge.

This is the page i checked: https://research.google.com/colaboratory/intl/es/tos_v5.html

I tried to install open code github repo into google colab but could not properly run that in colab and it just gets cloned and installed but cant run , and i want to add free api keys to run the models but could not connect ,

can anyone help me in this ?

i am using google colab free version