So RPC calls won't work if node stops syncing.. are there too many downsides by using the prunning mode?
There are downside of pruned node, but that depends on which RPC call you need. If you only fetch transaction details of address on your wallet, pruned node should be good enough.