Yes, Bologna will really be in 5th place because of this defeat.
because Atalanta still has 2 matches that have not been played. Meanwhile, their points this time were 66 points, only 2 points different from Bologna. And of course this will be quite easy for Atalanta to overcome, considering that Atalanta's next two matches shouldn't be a big problem.
Atalanta will compete against Torino and Fiorentina.
At least Atalanta should be able to get 3 points and immediately take over Bologna's position. Yes, in essence, this season, Atalanta will finish in 4th position and Bologna will drop to 5th position.
I agree that Bologna defeat has opened door for Atalanta to overtake them in standings. With two remaining matches and only two point deficit Atalanta has clear path to surpass Bologna. Their upcoming matches against Torino and Fiorentina appear manageable and securing three points in at least one of these games should be achievable. As you have already pointed it out this would propel Atalanta into fourth place bumping Bologna down to fifth. It is looking likely that Atalanta will capitalize on this opportunity and secure coveted Champions League spot leaving Bologna to settle for Europa League berth.
Atalanta has not yet exhausted its chances of winning a higher position because it is only 2 points behind Bologna but still has two games in hand (a make-up match with Fiorentina in one week). In the immediate future, Coach Gasperini and his students would certainly like to "celebrate" the Europa League championship title with their home fans by winning the match to welcome Torino in round 38 this weekend. If Atalanta cannot surpass Bologna on the rankings, they must finish outside the top 4 positions. If they finish 5th, Roma will qualify for the Champions League. If they finish in fourth place, Roma will participate in the Europa League. In case Atalanta and Bologna both reach 68 points, Atalanta will rank 5th on the separate rankings (head-to-head performance is worse than Bologna).