Ordinals were paying more than what was necessary, many blocks were filled with artificially increased fees. So, regular Bitcoin user wants to get transaction confirmed in the next few blocks, when 20 and more upcoming blocks are filled with ordinals who pay enormously high transaction fees, you are forced to pay high fee to get it confirmed soon and not to wait for weeks and months.

Fee increase and decrease is not purely only supply and demand game. Ordinals creators were artificially increasing the fee for some reason, they were paying more than it was necessary to get transaction confirmed in the next block, they were paying hundreds of times more than it was necessary and some regular Bitcoin users were just forced to pay high fees because of them.