Using FTP (I assume you mean SFTP) will buy you some performance, as would other protocols that are faster and requiring less compute than SMB.
I predict whatever solution you use will only buy you time. Usage is bound to increase so you’ll still hit the performance limits for the hardware platform at some point, unless you can constrain the simultaneous connections. File sizes will impact scalability a lot as well.
You can’t guess this one. You need to test.
tl;dr - I suspect you can’t win.
My torrenting level is very casual and (sry) I only leech. Also my ISP is a small one in the UK. Our Government seems to only force the big ISP to tattle on its users and block pirating sites. At least that’s how it has been for the last 10 years.
I have qbittorrent and Plex on my server. It is tempting to setup a VPN just for qbittorrent just to be sure.