As I see it, there are several factors that, together, help to create this "don't get involved" mindset in people.
Conditioning.
We are conditioned from our early years to play by the rules, go along with the program, and always be polite. Nice people don't fight. If you have a problem,take it to the authority figure (teacher, parent, cop, etc.) and they will handle it for you. Most people (in my experience) are not taught that there are times when it is good and proper to fight. Because of this most do not learn how to physically defend themselves effectively (although this has changed somewhat since the eighties with movies like Karate Kid popularizing the martial arts). So, this conditioning leads to
Lack of Training, which results in
Fear.
This factor is one I become aware of as I got into more advanced training. I believe people do not get involved mostly out of fear; fear that they cannot handle it (lack of training), fear of getting hurt, fear of getting in trouble with The Law (especially given the political nature of prosecutors), and fear of lawsuits. I would add, in some cases fear of extra-legal retribution. The thing about this level of fear is some of it is unconscious. We rationalize that we aren't getting involved because "good" people don't fight, and "it's a job for the authorities who are trained for it". A corollary to this is that you don't really know if your motivation for not getting involved is as pure as you think it is because you don't really have a choice. If you don't have the option to fight due to lack of training or tools, it becomes very easy to rsee your lack of involvement as taking the moral high ground. In other words, there is no virtue in not hurting someone if you lack the ability to do so anyway! So, if you have no other choice than not to get involved, it becomes the default action even when it is the wrong action.
Lack of Commitment
Lastly, is the lack of commitment to really hurt someone. A real fight, the way I learned to do it, is not pretty. It's not about fancy moves or trading blows with someone like a boxing match. It's definitely not about being fair. It's about doing what you must to survive. It's about taking the attacker out in the quickest, most efficient, and brutal way you can. That means making their fingers, knees, or elbows go backwards until they break. It means gouging eyes, chokes and blows to the throat, and slamming them down on the pavement or into a wall. A normal person has a natural aversion to this purposeful kind of violence. That is good. That's why we try to walk away, but when the time comes to fight, we need to commit.