My advice would be to make sure you not only hold it, but actually shoot it. Take it for a test drive. Make sure all the controls are where you want them, that it handles correctly for you, etc. Just sitting in the car on the lot doesn't tell you whether you will really want to drive it.
The Ruger LCP for example fits a lot of small hands, disappears into a pocket, and delivers a .380ACP bullet (not my FIRST choice, but it can work!). As great as it feels standing at the gun counter, some at the range are uncomfortable with the recoil. Its slim line that is its advantage for concealment is a narrow wedge driven into the hand, and its extremely light weight which is its advantage for carry makes for a real snap in the wrist (plus, the small grip which is great for concealment means most people can't get the pinky finger on it and that also means the perceived recoil is greater).
Bottom line... beyond simple fit to your hand, you have to find the right balance between concealment and controllability and power of: potency, size, and weight. Keep in mind that a gun you don't like to shoot may not get practiced with enough to stay proficient.
That said, my wife tried the LCP and after two shots set it down, saying "Ouch, that hurt; why would I want to do that?"; then took my XD and, after two magazines, said "This, I can do"...
Like others in this forum, I LOVE the XD, myself. Great trigger on it - er, FANTASTIC trigger on it; powerful, but controllable; VERY accurate; small in the sub-compact size...