I agree that saying no to a date becomes more complicated when the one who is inviting you to go out happens to be a close friend. In this case, you have to be extra careful not to hurt his feelings (as if turning him down is not gonna hurt).
But seriously, refrain from such expressions as “Dude, are you serious?” Just imagine the embarrassment he is already feeling internally just by asking you to go out with him. Tell him how you enjoy being around him then explain why you are not interested in him romantically, in a kind and respectful way. Who knows, you might even salvage your friendship after that.
On the other hand, do not be pressured into going out with a date just because you don’t want to hurt your friend’s feelings. Only do it if you really want to know him more intimately. Otherwise, it is better to be honest and turn him down now than to give him false hope and then hurt him in the future.