In Domino 6.x, you can define a Mail Rule in your Messaging Configuration that lets the server reject any SMTP mail that contains a specific filename or extension. In Domino 5.x, you had to use a pre-delivery agent to do this, but this was after the mail was accepted into the mail routing system.