You don't have to wait, speed estimates normally take only a few seconds to appear. You only have to wait if you want them to get more precise.
If the speed for one of your workers remains zero, it means that it's not submitting shares, which can happen if it has issues or is not configured properly.