Yes you could certainly do that, but the diodes are mainly so the wall wart supply couldn't backfeed power into your PC's usb port and vice versa. Since I removed the PC power passthrough diode it can't run as an unpowered hub anymore which removes that issue. Also you lose a bit of efficiency from the forward voltage drop of the diodes.