There's about as much risk as there is of putting a round through your butt cheek while carrying a J frame, in a quality holster, behind the hipbone. Basically, I figure if your draw is that sloppy, then, maybe, carrying a gun isn't for you until you can master something as basic as being able to safely draw and holster a firearm.
Ammunition doesn't spontaneously go off. Either there's something is already wrong with the gun, something in the gun breaks or a human interferes and screws up.
As for where it's pointed, I worry where a holstered gun is pointed almost as much as I worry about walking past unoccupied, parked cars in a parking lot. Cars don't just turn themselves on and run over people. Guns don't just randomly shoot people either.