Nice edit.    Glad you figured out your original issue.  Regarding this one, you could run separate instances with a specific GPU selected for each instance.  If you want it all in one instance, you brought up a good reason why it would be useful to choose resources per worker in the command line.  Unfortunately, I don't think it's currently possible (refer to my response to another question in this thread just a few posts up).