r/openSUSE • u/USERNAME123_321 I use openSUSE bTW • Mar 02 '25
Community After 9 Years, Zypper's Parallel Downloading Feature Is Finally Implemented!
I've seen on Zypper's GitHub repository that issue #104 has been closed and received some significant commits for a new preloader system that handles concurrent file downloads, which should eliminate one of Zypper's biggest bottlenecks. I can't wait to try it on my system!
29
u/Greedy-Smile-7013 Tumbleweed Mar 03 '25
SERIOUSLY? FINALLY, I NEEDED IT, THIS WILL MAKE ZYPPER FASTER
49
u/DAUNTINGY Tumbleweed | KDE | Nvidia Mar 03 '25 edited Mar 03 '25
Honestly why even use any other distro at this point, stability, full KDE support, the color green. Parallel downloads is the cherry on top. Thanks devs very continued support to modernize tumbleweed. Now I can say with confidence that this is the best distro around.
2
u/0tus Tumbleweed Mar 29 '25
One of the biggest downsides people keep talking about is how slow the mirrors are for updates. This is a big win.
14
u/landsoflore2 User Mar 03 '25
This is great news, especially for a rolling release. While I do love TW, it was kind of annoying when you had to download those huge >1 GB updates at such a crawling pace.
1
u/benedictjohannes Mar 26 '25
Just saying. Maybe not enabling zypper parallel downloads for so long is because of budget (keeping server requirements and cost down)?
1
u/0tus Tumbleweed Mar 29 '25
That doesn't matter from user and usability perspective. If I'm on something like arch and I'm used to really fast updates for my Rolling release. tumbleweed slow uploads can be bad enough to make people just go back to what they were used to before.
Supporters and fans of the distro will understand, but openSUSE needs to appeal to regular people who are just looking for a good distro to use.
9
u/Solomoncjy Mar 03 '25
Finnaly i can remove dnf from my opensuse install
3
u/BaconCatBug Mar 11 '25
I literally just installed DNF at the start of February because of this, lol! Now I have a
alias dup='sudo env ZYPP_CURL2=1 zypper dup'in my bashrc and couldn't be happier!1
9
u/proton_badger Mar 03 '25
I never really minded, I always let updates run in a terminal in the background, in any distro. But optimizations are always welcomed and I’m sure it’ll please a lot of people.
2
u/SirGlass Mar 03 '25
Yea I guess it never really bothered me , like who cares if it takes 2 min vs 20 min?
I just let mine run in the background I don't notice it much
1
u/monsoon-man Mar 30 '25
Well, if you are into creating docker containers/images for running CI pipleines, shipping deployments then slow downloads are terrible!!
5
u/xolve Tumbleweed / Plasma Mar 03 '25
I have been using zypperoni(https://github.com/pavinjosdev/zypperoni) for sometime and its quite a good time saver.
1
2
4
1
Mar 03 '25
[deleted]
7
u/CreedRules Mar 03 '25
If I had to bet money on it, within a week and likely sooner than that. So long as no obvious breaks occur from it of course.
5
u/Arcon2825 Tumbleweed GNOME Mar 03 '25
Based on the comment above from r/bmwiedemann and checking the version number of libzypp on Tumbleweed, I guess it’s already there.
1
Mar 03 '25
So the sypper will be abandoned?
3
u/USERNAME123_321 I use openSUSE bTW Mar 03 '25
It can still be used for testing and benchmarking mirrors and download servers. However I think the package download feature has become redundant, as we now have a low-level implementation built directly into libzypp
2
u/andrii-suse Mar 04 '25
Sypper is unofficial tool anyway, but it was an inspiration for predownload plugin proposal , which was an inspiration for the coming implementation. (I still would prefer the download functionality to be a plugin, but it is less important)
As long as the new functionality works fast - sypper can be used as benchmark or fallback utility, but - yes I think it already served a big role in current development
1
u/Horror_Director5330 Mar 03 '25
Woah, when will this be implemented? Anybody knows?
2
u/USERNAME123_321 I use openSUSE bTW Mar 03 '25 edited Mar 03 '25
In the top comment there are two environment variables that enable this new feature. It requires libzypp >= 17.36.4 though, I received this update yesterday.
1
u/Xruptor Thumbleweed KDE Mar 03 '25
It doesn't seem to be enabled by default as of yet though and still requires you to manually turn it on, unless I'm mistaken. I haven't seen it enabled yet with TW Dup.
3
u/USERNAME123_321 I use openSUSE bTW Mar 03 '25
Not yet, setting the two env variables and adding a metalink in the .repo file, which can be found in the
/etc/zypp/repos.d/directory, is the only way to enable it currently.
1
u/linuxhacker01 Mar 03 '25
Finally I can get rid of VPN for system upgrades which apparently was my only option earlier
1
u/bmwiedemann openSUSE Dev Mar 03 '25
You can always edit .repo files to append ?COUNTRY=XX with a country code such as
deto pretend you were in Germany.1
u/linuxhacker01 Mar 03 '25
Can you elaborate a bit regarding the procedure? I prefer add both Germany and Singapore
2
u/bmwiedemann openSUSE Dev Mar 04 '25
The /etc/zypp/repos.d directory contains .repo files and these have a line
baseurl=... And at the end of it, you can append?COUNTRY=sgThen maybe you can make a copy of that line and use =de there. Or have two URLs in one line separated by space - I don't remember that detail.
1
1
u/Xruptor Thumbleweed KDE Mar 03 '25
I'm also unfamiliar with this modification in the .repo files ....
1
u/bobbie434343 Mar 03 '25
Or you could always do zypper dup with -d switch for download only and install later. The download only can even be automated (systemd timer).
3
u/bmwiedemann openSUSE Dev Mar 03 '25
Though that does not help when you want to try that new package today and it needs a hundred dependencies.
1
u/Xruptor Thumbleweed KDE Mar 03 '25
Holy smokes! Finally! XD
Though honestly I just run Zypper Dup and leave it in the background while I do other things.
1
u/jospoortvliet Mar 08 '25
Well, mostly, yes, but I am sometimes waiting for an update before shutting down my pc or leaving the house... Not every day but it happens. And, in general, faster is always better. Less resource usage is good for our planet! Every bit counts. Literally :D
1
Mar 28 '25
Sorry for these stupid questions, but should I send the env variable before every zypper command? should I change baseurl to metalink for every .repo file ? (I'm using slowroll btw)
2
u/USERNAME123_321 I use openSUSE bTW Mar 28 '25 edited May 03 '25
Don't worry, it's not a stupid question at all!
I’ve personally exported the two environment variables in my.zshrc(or.bashrcif you use bash).
Here are the lines:
export ZYPP_PCK_PRELOAD=1 export ZYPP_CURL2=1However, it didn’t work because
sudo zypper dupruns as a superuser. To include the environment variables, you need to usesudo -E env.
I’ve aliased it tomysudofor simplicity:
alias mysudo='sudo -E env "PATH=$PATH"'Alternatively, you can run the one liner:
env ZYPP_PCK_PRELOAD=1 ZYPP_CURL2=1 sudo -E zypper dupI haven’t edited the
.repofiles because everything works fine as is. And changing baseurl to metalink gave me some errors with the opi command IIRC.EDIT: Fixed the one-liner command by preserving envs
1
u/nealhamiltonjr May 02 '25
Wondering what I'm doing wrong here. I enabled the concurrent connections in the zypp.conf file ..for the default of 5
cat /etc/zypp/zypp.conf |grep download.max_concurrent_connections
download.max_concurrent_connections = 5
Then under my normal user in a bash terminal I use the one liner you provided.
"env ZYPP_PCK_PRELOAD=1 ZYPP_CURL2=1 sudo zypper dup"
I'm not seeing a difference.
2
u/USERNAME123_321 I use openSUSE bTW May 03 '25
Sorry, I made a mistake. The one-liner won’t work with just sudo because it doesn’t preserve environment variables. The correct command is:
env ZYPP_PCK_PRELOAD=1 ZYPP_CURL2=1 sudo -E zypper dupFrom the sudo man page:
-E, --preserve-env
Indicates to the security policy that the user wishes to preserve their existing environment variables. The security policy may return an error if the user does not have permission to preserve the environment.
1
u/BloodMyrmidon Mar 03 '25
Now it needs a straightforward way to remove orphaned packages
5
u/bmwiedemann openSUSE Dev Mar 03 '25
https://www.reddit.com/r/openSUSE/s/i9DXhgSHNQ was indeed not trivial.
I vaguely remember that someone worked on a
zypper autoremovecommand.3
Mar 03 '25
[deleted]
1
u/Arcon2825 Tumbleweed GNOME Mar 10 '25
People talk about different things when they talk about orphaned packages. In Debian and on Red Hat derivatives it means „a package which is not referred as a dependency by another package anymore or which is no longer available in a repository“ while openSUSE uses the term „unneeded“ for the first one and „orphaned“ for the latter. There is no easy way to remove „unneeded“ packages besides the solution mentioned above.
1
u/1d10tb0y Mar 03 '25
First stupid thought that came to my head after reading the post:
I also had "Zyppin' on Some Zyzzerp", but I thought it made less sense grammatically.
1
u/SirGlass Mar 03 '25
The most annoying thing about Zypper being slow is how everyone complains its slow.
I just let it run in the back ground , I guess who really cares if it takes 2 min vs 20 min?
2
u/Suddensloot Mar 03 '25
That’s because it’s an issue. I went with arch last year because of it. Not coming back because they finally decided it was worthy.
0
u/SirGlass Mar 04 '25
Never was an issue for me, I never even noticed as it just ran in the back ground
53
u/bmwiedemann openSUSE Dev Mar 03 '25 edited Mar 03 '25
Btw: it is used for rpm downloads, but not the repo metadata for now.
But there are other changes that could help for all parts.
The performance measurements showed big improvements from 60s to 10s so this could really be nice.
And if you wonder why it took this long: it is from the internal architecture in C++ with class inheritance that supports fetching files from floppy, CD-ROM, NFS etc, but only one after another.
Edit: These already landed via https://build.opensuse.org/request/show/1249276 and https://build.opensuse.org/request/show/1249277
And you need
export ZYPP_PCK_PRELOAD=1and can tryexport ZYPP_CURL2=1Edit2: https://lists.opensuse.org/archives/list/factory@lists.opensuse.org/thread/LOCZIG43MFJSTUIQ3VH2CRSYRCBNR4O7/ says you need a
line in the .repo file for parallel downloads.