There are two approaches to consider:
1. State Law. Some states specifically address the issue while others are silent. If your state's law regarding firearms at church is specific, you have no choice but to follow the law. In many states however, firearms in church is a topic on which state law is silent -- essentially leaving it up to the property owner. If your state's law is silent, then you can typically carry UNLESS the church has put out the word in its church bulletins, newsletter, has a sign on the front door and/or has made an announcement from the pulpit.
2. Discretion. Although I practice both open and concealed carry, I believe that discretion and common sense dictates concealed carry at church. Use common sense. If and when the time ever comes that you need to display your firearm because some nut job has arrived and is ready to conduct a mass killing, then you'll need to render your firearm out in the open as a life-saving tool. But until then I believe discretion requires concealment.