Yes, Bologna will really be in 5th place because of this defeat.
because Atalanta still has 2 matches that have not been played. Meanwhile, their points this time were 66 points, only 2 points different from Bologna. And of course this will be quite easy for Atalanta to overcome, considering that Atalanta's next two matches shouldn't be a big problem.
Atalanta will compete against Torino and Fiorentina.
At least Atalanta should be able to get 3 points and immediately take over Bologna's position. Yes, in essence, this season, Atalanta will finish in 4th position and Bologna will drop to 5th position.
I agree that Bologna defeat has opened door for Atalanta to overtake them in standings. With two remaining matches and only two point deficit Atalanta has clear path to surpass Bologna. Their upcoming matches against Torino and Fiorentina appear manageable and securing three points in at least one of these games should be achievable. As you have already pointed it out this would propel Atalanta into fourth place bumping Bologna down to fifth. It is looking likely that Atalanta will capitalize on this opportunity and secure coveted Champions League spot leaving Bologna to settle for Europa League berth.