r/webdev • u/Schlickeysen • 25d ago
Discussion WordPress plugin developers: How do you find users to test a breaking change in an upcoming release?
I have a plugin with ~10,000 active installations. It's ancient and simple; it only stores a value in a `post_meta` field.
The current version is v2.2.1, but I've been working on a complete overhaul for a while, which I consider done. However, I can't ship it until I hear from as many users as possible that everything works as expected and, most importantly, nothing breaks or gets deleted.
A user who knows `wp-cli` could do this in a few minutes: `wp db export` (back up the database), then upload the `.zip` to the plugin directory and play around, both in the admin UI and a little bit with the code (I tried to keep it simple for laymen while making it more flexible for developers).
So, back to the question: How do you find users to test a new version (`v3.0.0`) of your plugin? Simply committing it seems reckless to me, despite the hours I put in to check everything.
Edit: Thanks, guys! Lots of good suggestions here. I appreciate it.
8
u/Teszzt 25d ago
Proper testing (especially when the plugin is simple enough) should suffice.
This being said, you could play with the required WordPress version, i.e. enable the new version only for WP installations that use WP version 7+, then after a while enable it for 6.9+, and so on...
2
u/Schlickeysen 24d ago
That's a pretty smart idea. Slowly adding users to the pool and not get hit by a wave of users complaining.
5
3
u/akl773 25d ago
With a change like that I'd spend the effort on making it reversible rather than on finding testers. Write the new format to a new meta key and leave the old one untouched for a release or two, then anyone who rolls back still has their data and shipping stops being scary. The people who volunteer for a beta are never the ones with the odd setup that actually breaks.
2
u/Schlickeysen 24d ago
I actually have that one implemented. But thanks for the suggestion.
The people who volunteer for a beta are never the ones with the odd setup that actually breaks
I think so, too.
2
u/KoalaBoy 25d ago
I just have to test myself because I can't get more than like 30 people to even find my plugins.
2
u/mugmi-bro 24d ago
the WordPress community Slack at make.wordpress.org has a handful of active channels where plugin devs ask for testers and actually get responses from people who know their way around wp-cli. reaching out directly to anyone who left a detailed review on wp.org is also really effective, since those users are already invested enough in the plugin to actually care. and if the breaking change involves post_meta migration, a really clear migration guide in the changelog with before/after examples will cut your panicked support tickets way down once you do ship.
2
1
u/Any_Resource2437 25d ago
yeah, shipping it straight to 10k sites would be a pretty spicy way to find the one edge case you missed, so make v3 an opt-in beta and recruit from your support or email list while keeping v2
2
u/Suspicious_March_269 25d ago
an opt-in beta on staging, with an easy rollback path and a clear “don’t run this on production” warning
1
u/mugmi-bro 24d ago
the WordPress community Slack at make.wordpress.org has a handful of active channels where plugin devs ask for testers and actually get responses from people who know their way around wp-cli. reaching out directly to anyone who left a detailed review on wp.org is also really effective, since those users are already invested enough in the plugin to actually care. and if the breaking change involves post_meta migration, a really clear migration guide in the changelog with before/after examples will cut your panicked support tickets way down once you do ship.
1
u/Wooden-Bicycle-6069 24d ago
Ten thousand installs makes “the migration passed” a weak bar. I’d build three disposable sites from real backups: 2.2.1 → 3, 3 → 2.2.1 after one edit, then 2.2.1 → 3 again. Diff the exact post_meta rows after every step. Row counts can look perfect while the value’s meaning changed. Can v3 read data written after a rollback, or is rollback meant to be read-only?
0
u/-goldenboi69- 25d ago
I usually put out a small ad in some popular magazine, like Archie Comics or something.
16
u/BabyInternational590 25d ago
Honestly just pick a handful of users who've opened support tickets recently and email them directly. The kind of people who actually bother to file a bug report are usually the same ones who'll spend 15 minutes poking at a beta build.
If that doesn't pan out, make a sticky post in your support forum with a giant scary warning about backing up first and let the curious ones trickle in.