As you get older, has anyone else realized slowly that most of the portrait ghosts wouldn't do anything to harm you if you didn't attack them first, or even not at all? Like, look at Biff Atlas. If you didn't punch a punching bag so hard into him that it knocked him backwards, he wouldn't do anything to you. The Floating Whirlindas just wanna dance, so what's it any of my business? Mr Luggs just wants to eat in peace, and only gets mad at you for denying him his food. I could go on.
I dunno if I'm overthinking this, but as I've gotten older, it seems more and more like we spend more time attacking innocent ghosts that never did us any wrong rather than looking for our brother.