Update. I got this actually working while trying to reproduce a bug…
Long story short: you can use multiple USB destinations. That is to say, at least with one supported controller. I’m not sure if it’s generic support, but I’ll test somewhere next week.