"Incremental recursion" may be the culprit, and if you disable it, the memory usage will come down.
"Incremental recursion" can be disabled using the "--no-inc-recursive" option or its shorter "--no-i-r" alias. Your "rsync" command should look like the following.
rsync --no-inc-recursive --progress ...
If you are using "--delete" option, it will not disable incremental recursion. --delete-before, --delete-after, --prune-empty-dirs, and --delay-updates disable incremental recursion.
According to the manual of rsync (man rsync),
Some options require rsync to know the full file list, so these options disable the incremental recursion mode. These include: --delete-before, --delete-after, --prune-empty-dirs, and --delay-updates. Because of this, the default delete mode when you specify --delete is now --delete-during when both ends of the connection are at least 3.0.0 (use --del or --delete-during to request this improved deletion mode explicitly). Incremental recursion can be disabled using the --no-inc-recursive option or its shorter --no-i-r alias.
If you use "--no-inc-recursive" option, using "--delete" will not increase RAM usage.