At those times difference is for sure bigger.
A bigger contrast is seen at times of the day. Go to
https://jochen-hoenicke.de/queue/#1,6m and zoom in (by selecting an area on the graph) on a random week or so in the last few months. Generally when you see a surge in the mempool size, and therefore the average fee, it is usually around 13:00 - 23:00 UTC, which corresponds to day time in most of Europe, and initially the east coast and then all of the Americas. Conversely, the mempool is usually fairly empty, with fees around 1-2 sat/byte, around 01:00 - 06:00 UTC, when the aforementioned continents are generally sleeping and Australia and eastern Asia are awake.
You can use this to your advantage to plan non-urgent transactions, such as consolidating all your small inputs.