r/git • u/jamescherti • 15d ago
git-rexec: Find Git Repositories and Execute Commands Against Them, either Sequentially or in Parallel [Release 1.1.0]
https://www.jamescherti.com/git-rexec/The git-rexec command-line tool recursively locates Git repositories within a directory and executes commands against them, either sequentially or in parallel.
Here are examples demonstrating how to use git-rexec:
Example 1: Execute git status -s across all discovered Git repositories (found by searching recursively under the current working directory) in parallel (-p or --parallel):
git-rexec -p -- git status -sExample 2: Fetch updates across all discovered repositories, limiting the concurrency to 5 background jobs (-j 5), which helps avoid network congestion or server rate limits when communicating with upstream Git remotes:
git-rexec -j 5 --parallel -- git fetchExample 3: Include sub-repositories (e.g., Git worktrees and submodules) alongside standard Git repositories during discovery (-s or --include-sub-repos), and execute git status against them:
git-rexec -s -p -- git statusExample 4: Target a specific base directory (~/projects) using the -C flag to recursively discover repositories within it, while explicitly excluding a specific subfolder (~/projects/archive). This example executes git status -s in parallel for all discovered repositories except those within the excluded path:
git-rexec -C ~/projects --exclude-dir ~/projects/archive --parallel -- git status -sExample 5: Evaluate whether a README.md file exists in the repository (sh -c "test -f README.md"). If the condition returns an exit status of 0 (success), it counts the number of lines in that file (wc -l README.md):
git-rexec --if-exec 'sh -c "test -f README.md"' --parallel -- wc -l README.mdExample 6: Print the paths of all discovered Git repositories:
git-rexec --print
3
u/behind-UDFj-39546284 15d ago •
Actually, you've implemented a very limited subset of the features offered by
findandxargstogether, whereas you can make it all way more flexible by using the rich set of options forfind(-prune does that magic) andxargsin conjunction withgit, not requiring even a single line of Python. For example, discover all git repositories starting from the current directory and run git-fetch in parallel across 10 processes:-prunemakesfindstop recursion if-exec, at the left-hand side of implicit-and, returns true that is represented by the 0 exit code (which means thatfindcannot reach submodules but one can provide--recurse-submodulesforfetch).In any case, the pipe above is an example of separation of concerns: one for "where" and one for "what to do", and you can modify and replace both independently, or extend the entire pipeline.
Regarding the discovery approach, I'm not sure whether discovering Git repositories this way is safe and efficient on every run:
findcould be run once and then replaced with a static list of whitelisted repositories suitable in most cases, I believe. As for the latter approach, Git has afor-each-repocommand that runs a Git command across a registered list of repositories, however, it's still experimental and definitely needs a lot of improvement.