I believe the traffic would affect the speed (I'm not sure how much though). This cool graphic shows a flowchart of how coins actually get transferred within the network.