The PS5 will stand vertically on its own without the stand, but it can easily be bumped over.
Back in 2016, when the mid-generation hardware upgrades of the Xbox One X and PS4 Pro were still just mere announcements, we publicly wondered if we had seen the last truly distinct break between console generations. Instead of releasing completely new console platforms—with exclusive games and features that just don't work on the previous model—Microsoft and Sony at the time both seemed to be leaning towards a smartphone-style model, with regular releases of more powerful consoles that share a common software platform with what came before.
This seems to be the direction Microsoft is heading with the Xbox Series X and S, two console options that serve as baseline hardware power upgrades in a consistent Xbox ecosystem. But Sony is going for more of a hard break with the PS5. In addition to the usual horsepower boost (and standardization of quick-loading NVMe storage), Sony has put extra effort into a new controller and system-level features that try to make the new console more distinct from PlayStations past.
After a few weeks with the console, there's a lot to like about the PS5's new vision for the PlayStation line. Whether those improvements are worth $500 at the moment, though, is a harder question to answer.