If the pastor or elder you ask doesn't know him well, he can guide you to a trustworthy source that knows him better.
If you know the man well or at least better than what I've just described, but you are not sure whether you are interested in him, I'd encourage you to at least take some time to get to know him before giving an unequivocal "no." Keep in mind that this is different from feigning interest when there isn't any.
The idea is to remove that period of confusion or vulnerability for the woman by being forthright about what level of intention or commitment exists (a la 1 Thessalonians 4).
You probably won't know at this stage how things are going to ultimately turn out regarding marriage (that's why you date), so you need not communicate that right away.
Because this sort of (perhaps unintentional) deception is a particular temptation in a dating context, we need to be deliberate about avoiding it.
That's where the following practical suggestions come in.
If a man initiates with you, ladies, think and pray and seek counsel before simply dismissing him.
Do you have enough information to know that you could not marry this man?
If you don't have even information at that level, feel free to tell him that you want some time to think and pray about it (that is, if you're not sure at that point that you're not interested).
Then — in addition to actually thinking and praying about it — ask one of your pastors or elders whether he knows him and what he thinks.
If you know the woman from church, if you've seen her interact in a group, observed her with others, maybe worked with her as a part of some ministry, that input should be enough for you to think through the decision of whether initiation of a relationship is the right thing.
Remember, your intent at this point is not necessarily marriage — and that's not what either of you are committing to at this stage.