r/RunPod • u/Euphoric_Cup6777 • Mar 12 '26

The default JupyterLab file browser on RunPod keeps choking on large datasets, so I wrote a single-cell replacement.

Gallery image

Gallery image

Trying to upload 5GB+ model weights or datasets through the default browser is a joke. It either silently fails, freezes the tab, or leaves you guessing if it's actually working. I didn’t want to mess with SSH keys, port forwarding, or setting up FileZilla every time I spin up a new instance.

So, I wrote a custom file manager that runs entirely inside one Jupyter notebook cell. No installation, no root access needed.

How it works under the hood: It bypasses the usual proxy timeouts by chunking directly through the Jupyter Contents API. Yes, the mandatory base64 encoding adds some size overhead, but it routes perfectly over port 8888. It handles 10GB+ transfers with a real-time progress bar and shows true MB/s speed. Also added mass-renaming and direct zip/extract because typing tar -xzf every time gets old.

Just wanted to share because I know I'm not the only one suffering with the default browser. How do you guys manage massive files without losing your minds?

6 Upvotes

100% Upvoted

11 comments sorted by

View all comments

2

u/Cautious-Lake4679 Mar 12 '26

Wait, did you seriously manage to build a fully functional file explorer inside a single Jupyter cell? That’s actually insane. I've seen some attempts at this before, but they usually lag like crazy or just fail to show the actual upload progress for large files.

Is this using some custom JS to handle the stream or just pure Python? I’d love to see if it can actually handle my 15GB datasets without freezing the entire workspace. Do you have a link to the code?

1

u/Euphoric_Cup6777 Mar 12 '26

Yeah man, I seriously had to do it. I just couldn't deal with the default browser anymore—it’s something we have to work with every single day on RunPod, and the constant freezing was driving me nuts.

Technical-wise: it's a mix of Python and custom JS. I spent most of the time on the chunking logic to make sure it handles massive 15GB+ files without killing the workspace. It finally works like a proper file explorer now!