Announcement

Collapse
No announcement yet.

Reverese NDR attack - change IP/MX records

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Reverese NDR attack - change IP/MX records

    It's been quite a while since I monkeyed around with Exchange so keep that in mind with what follows.

    I recently inherited a split-site network for the next month or two, until they sort out a permanent in-house IT solution.

    There is a 100MB Ethernet link (BT Short Haul Data Services) joining the two sites.

    8 Windows 2003 Standard servers and one Exchange 2003 Enterprise box serve about 100 clients.

    Breakout to the internet is currently a leased line which is soon to be discontinued.

    ADSL at the main branch will take over - most traffic is LAN based file sharing and printing; internet use it very low, so ADSL should suffice at a fraction of the cost of the leased line.

    Last week the Exchange box was subject to what I think was a reverse NDR attack.

    I sorted the flooded queues in Exchange, triple checked that the server was not an open relay, ran various virus and spyware scanners and all the other usual nonsense we have to do when this happens.

    I also had to arrange de-listing from some blacklists on Wednesday last week.

    In this enlightened age though, Yahoo, Gmail, Microsoft et al run their own blocks and getting off their lists is a major event.

    Given that our current public (leased line) IP address is not blacklisted there ought to be no problems but mails are being rejected by Yahoo, Gmail, Microsoft, so something needs to be done.

    Migrating Exchange to the ADSL public IP seems a promising solution to deal with this - I wouldn't normally consider it but since we were going to shift from leased line to ADSL anyway...

    So, long story short, I need to refresh myself and make sure the plan I have is sound.

    Reverse DNS has been requested from the ADSL provider and this will be in place by tomorrow.

    In no order of preference:
    • Change current A record to new public IP address
    • Add new public IP address(s) as MX records
    • Wait and think lofty thoughts while global DNS sorts itself out
    • Meanwhile, back at the ranch, stop the current Default SMTP Virtual Server after making a new one (could just tweak the existing one but I tend to do it this way)
    • Go though internal DNS and overcheck any records referencing the legacy public IP addresses
    • Configure new firewall on the DSL line with all required ports forwarded for Exchange, OWA etc


    I know the above is a gross oversimplification and not necessarily listed in order of priority, but can anyone see any screaming howlers that I might have overlooked ?

    If I pull the trigger on this late on Friday evening I should have all the kinks worked out by start of play on Monday morning so that staff can come in and get on as usual.

    Thoughts ?

  • #2
    Re: Reverese NDR attack - change IP/MX records

    Do you have any mail scanning in place externally or internally? settings such as recipient lookup can help with an NDR attack and activating Tar pitting.

    You can see this via the SMTP virtual server natively within Exchange as well, so worth considering.

    Comment


    • #3
      Re: Reverese NDR attack - change IP/MX records

      Thanks for your reply - the only mail scanning done at present by a corporate Kaspersky product (Total Space Security I think, from memory) and there was also an Untangle gateway in front of Exchange that was dealing with SPAM.

      Untangle died (hardware failure) and I intend shoving a pfsense box or something similar in for combined mail scanning, SPAM filtering and possibly proxying to replace it.

      I will check later today to see if there are any other elements I have missed but I think that's about it. It appears to be a simple, even rudimentary setup.

      I'm in two minds whether or not to shove a inbound/outbound SMTP Mail Relay in place using one of the other servers as an additional layer between Exchange and the outside world. There is a member server and also a DC that both sit almost completely idle for most of each day, so there is scope without investment to do this.

      Looks like my weekend is going to be fun...

      EDIT:

      Just found that the Exchange server is configured for port 26 so that port 25 can be used by Mailwasher Enterprise...
      Last edited by NDR; 4th July 2012, 10:59.

      Comment


      • #4
        Re: Reverese NDR attack - change IP/MX records

        Well before you begin, (not sure if you did already or this is a plan) drop the TTL on all public DNS records that you will be updating. This will reduce the change time, when you actually need them to change quick, then raise them back up to a 1 day or so, once the drama subsides.

        <strike>Also if you SPF in place you might not have had to deal with the banning in the first place, since there is a mix of domain AND ip to get on the big mailers blacklists. Yahoo is the worst......</strike>

        EDIT: Woops never mind SPF wont help, got confused
        Last edited by Lior_S; 4th July 2012, 14:20.
        "...if I turn out to be particularly clear, you've probably misunderstood what I've said” - Alan Greenspan

        Comment


        • #5
          Re: Reverese NDR attack - change IP/MX records

          If Exchange isn't answering the inbound email from the internet, then you need to look at what is. Mailwasher was the cause of the problems, because by the time that Exchange would reject the email, it was too late.

          As a minimum I expect all antispam solutions to do recipient validation. If they do not, and they expect to answer SMTP traffic rather than Exchange (or whatever is hosting email) then they don't even get considered. I have sites that drop 80% of email, where it is addressed to invalid recipients.

          Tarpit and Recipient Validation on Exchange can and does work, but Exchange needs to be answering the email.

          SPF records, as already corrected, would have zero effect on this. SPF records are for others receiving email from you, they do next to no good for your own incoming email.

          Simon.
          --
          Simon Butler
          Exchange MVP

          Blog: http://blog.sembee.co.uk/
          More Exchange Content: http://exchange.sembee.info/
          Exchange Resources List: http://exbpa.com/
          In the UK? Hire me: http://www.sembee.co.uk/

          Sembee is a registered trademark, used here with permission.

          Comment


          • #6
            Re: Reverese NDR attack - change IP/MX records

            Mailwasher was installed post NDR attack, or so I am told.

            Prior to that the Untangle box was filtering SPAM with Exchange still on port 25.

            Untangle died (blown mainboard) the day after the reverse NDR attack.

            'Facts' are a little thin on the ground since they have no onsite IT and everyone is pitching in with their version of what they think happened.

            To be frank, I am bouncing the idea around of putting a new Exchange box in by re-purposing an existing server, set it up from scratch properly and work with a known quantity, move the mailboxes and so on.

            I hate following along after a mess because you never really know what damage has been done, or where.

            I will be happier with Exchange back on port 25, Mailwasher uninstalled, an AD aware gateway device of some sort for SPAM filtering before the emails hit Exchange and the shiny new A record and MX record IPs resolving to the FQDN

            One plus about the coming weekend - there's a great 7-day-a-week coffee shop two minutes away from where I'll be tussling with this...

            Comment


            • #7
              Re: Reverese NDR attack - change IP/MX records

              I haven't really found anything open source that can do recipient validation that I am happy with. If you really want to put something in front of Exchange, then IIS with a copy of Vamsoft ORF works well, although I will often run Vamsoft ORF directly on the Exchange 2003 server. I find it protecting most Exchange 2003 servers that I support and it works well - plus it is cheap.

              Simon.
              --
              Simon Butler
              Exchange MVP

              Blog: http://blog.sembee.co.uk/
              More Exchange Content: http://exchange.sembee.info/
              Exchange Resources List: http://exbpa.com/
              In the UK? Hire me: http://www.sembee.co.uk/

              Sembee is a registered trademark, used here with permission.

              Comment


              • #8
                Re: Reverese NDR attack - change IP/MX records

                You could just do spamhaus zen alone in 2003, that should help. No cost at all.....
                "...if I turn out to be particularly clear, you've probably misunderstood what I've said” - Alan Greenspan

                Comment


                • #9
                  Re: Reverese NDR attack - change IP/MX records

                  Originally posted by Lior_S View Post
                  You could just do spamhaus zen alone in 2003, that should help. No cost at all.....
                  If you don't mind someone unaccountable deciding what email you can receive...

                  Simon.
                  --
                  Simon Butler
                  Exchange MVP

                  Blog: http://blog.sembee.co.uk/
                  More Exchange Content: http://exchange.sembee.info/
                  Exchange Resources List: http://exbpa.com/
                  In the UK? Hire me: http://www.sembee.co.uk/

                  Sembee is a registered trademark, used here with permission.

                  Comment


                  • #10
                    Re: Reverese NDR attack - change IP/MX records

                    Originally posted by Sembee View Post
                    If you don't mind someone unaccountable deciding what email you can receive...
                    Simon,
                    I was hoping you would say that ,

                    From Vamsoft website:
                    Employing the extremely effective combination of automatic tests like DNS blacklists and SURBLs (including Spamhaus ZEN and uribl.com), the reverse DNS test, the HELO/EHLO domain test, the SPF test and Greylisting......
                    Vamsoft seems to use spamhaus as well. If you had an issue, how fast would they refer you to spamhaus to remove the blocked IP...? They seem to have a winning combination of spam filtering, I like that, just saying that RBL are probably the most effective first level spam control, and you can get that alone at no cost.

                    I know you don't like it, but you seem to be using it.
                    "...if I turn out to be particularly clear, you've probably misunderstood what I've said” - Alan Greenspan

                    Comment


                    • #11
                      Re: Reverese NDR attack - change IP/MX records

                      Originally posted by Sembee View Post
                      ... If you really want to put something in front of Exchange, then IIS with a copy of Vamsoft ORF works well, although I will often run Vamsoft ORF directly on the Exchange 2003 server. I find it protecting most Exchange 2003 servers that I support and it works well - plus it is cheap.

                      Simon.
                      Testing the product for an couple of hours or so today on my own server (the things we do to our own environments) and I'm very impressed on all fronts.

                      Very lightweight, no noticeable impact at all on system resources, zero false positives so far and that price can't be argued with.

                      Thanks for the suggestion - this is definitely going into the box of tricks and I will probably look to install it this weekend on the previously mentioned Exchange server.

                      A large part of the problem appears to be that Google, Microsoft, Yahoo, Sky and similar are ignoring blacklists in favour of their own 'block' lists.

                      Having spent hours jumping through their respective hoops trying to get them to unblock this particular server the only reply I got came from Yahoo and it basically said no, go away and look at our bulk sender policy.

                      All of this from a reverse NDR attack...

                      The really annoying thing is that small companies who get tripped up like this may sometimes have almost nowhere to go for rapid resolution.

                      As it happens this company has an ADSL line that only went live a week ago, so there is somewhere to shift things across to.

                      BT have set up reverse DNS PTR for the two of the new public IP addresses on the ADSL line so this weekend should see things back to where they should be, more or less.

                      I noticed some squirrelly internal DNS though - looks like some missing records are causing nslookup to fail on the Exchange box, so some additional jiggery pokery to do there.

                      Thanks once again for the product recommendation - very much appreciated.

                      Comment


                      • #12
                        Re: Reverese NDR attack - change IP/MX records

                        Originally posted by Lior_S View Post
                        Simon,
                        I was hoping you would say that ,



                        Vamsoft seems to use spamhaus as well. If you had an issue, how fast would they refer you to spamhaus to remove the blocked IP...? They seem to have a winning combination of spam filtering, I like that, just saying that RBL are probably the most effective first level spam control, and you can get that alone at no cost.

                        I know you don't like it, but you seem to be using it.
                        Vamsoft CAN use blacklists, but you don't have to use them, and I don't. Almost every antispam product I have seen can use them.

                        I do use the Blacklists that Vamsoft can produce for you, based on the email delivery attempts that are seen by your server, but I have 100% control over those.

                        I have never been a fan of blacklists like Spamhaus and the like, simply because I don't have control and if you get on one of their blacklists and they say "tough", there is nothing you can do about it. For a mission critical app like email, that simply isn't good enough.

                        Simon.
                        --
                        Simon Butler
                        Exchange MVP

                        Blog: http://blog.sembee.co.uk/
                        More Exchange Content: http://exchange.sembee.info/
                        Exchange Resources List: http://exbpa.com/
                        In the UK? Hire me: http://www.sembee.co.uk/

                        Sembee is a registered trademark, used here with permission.

                        Comment


                        • #13
                          Re: Reverese NDR attack - change IP/MX records

                          Originally posted by NDR View Post
                          Testing the product for an couple of hours or so today on my own server (the things we do to our own environments) and I'm very impressed on all fronts.

                          Very lightweight, no noticeable impact at all on system resources, zero false positives so far and that price can't be argued with.

                          Thanks for the suggestion - this is definitely going into the box of tricks and I will probably look to install it this weekend on the previously mentioned Exchange server.

                          A large part of the problem appears to be that Google, Microsoft, Yahoo, Sky and similar are ignoring blacklists in favour of their own 'block' lists.

                          Having spent hours jumping through their respective hoops trying to get them to unblock this particular server the only reply I got came from Yahoo and it basically said no, go away and look at our bulk sender policy.

                          All of this from a reverse NDR attack...

                          The really annoying thing is that small companies who get tripped up like this may sometimes have almost nowhere to go for rapid resolution.

                          As it happens this company has an ADSL line that only went live a week ago, so there is somewhere to shift things across to.

                          BT have set up reverse DNS PTR for the two of the new public IP addresses on the ADSL line so this weekend should see things back to where they should be, more or less.

                          I noticed some squirrelly internal DNS though - looks like some missing records are causing nslookup to fail on the Exchange box, so some additional jiggery pokery to do there.

                          Thanks once again for the product recommendation - very much appreciated.
                          Run the product for a few days without actually blocking anything, but white listing outbound email. In my experience most users email the same core set of recipients all of the time. Therefore the auto white list will ensure that most of that email flows without interruption.
                          I rave about this product quite a bit - check my blog for some of the results I have seen with it. There is a new version soon, which if you buy now you will get, a few more checks but the same core functionality.
                          For most clients, Recipient Validation, DHA and then setting up honeypots based on the most common non-valid users has pretty much knocked out the spam.

                          Simon.
                          --
                          Simon Butler
                          Exchange MVP

                          Blog: http://blog.sembee.co.uk/
                          More Exchange Content: http://exchange.sembee.info/
                          Exchange Resources List: http://exbpa.com/
                          In the UK? Hire me: http://www.sembee.co.uk/

                          Sembee is a registered trademark, used here with permission.

                          Comment


                          • #14
                            Re: Reverese NDR attack - change IP/MX records

                            touché.

                            But spamhaus is not like the rest. Its the only one I would ever use, 30 min TTL....

                            The others can be a nightmare. Most notably sorbs, dont ever use them. They suck so much I want them to fail and go away. Unfortunately they are making loads of money by holding people hostage, they are not a reputable RBL and spoil it for the rest of them.........but many use them

                            Anyway, NDR sorry for hijacking your thread a bit.
                            "...if I turn out to be particularly clear, you've probably misunderstood what I've said” - Alan Greenspan

                            Comment


                            • #15
                              Re: Reverese NDR attack - change IP/MX records

                              Update:

                              Everything went more or less as planned over the weekend and normal service was resumed by Sunday afternoon, ready for start of play this morning.

                              Staff were dismayed, or so I am told, that they could get on with things and not use email server problems as an excuse or shield...

                              One anomaly remains: Google is still bouncing messages but, curiously, it is bouncing them with the correct FWDN of the Exchange server, but the wrong (old) public IP.

                              This is affecting GMail - several companies this client deals with host their services with Google.

                              Sending to [email protected] is fine, but GMail or businesses who use Google's hosted services are adversely affected.

                              I spent hours last week jumping through the hoops involved in requesting de-list from Yahoo, Google and Microsoft's 'block' lists. Yahoo said no (paraphrased), Microsoft said no, get your act together (paraphrased) and Google failed to respond at all.

                              So, new public IP that was triggered on Friday evening and went globally stable yesterday and Google still bounces emails from the correct FQDN bound to the old public IP address.

                              This probably just goes to show something...



                              I am astonished that Google is (mis)handling emails like this, and that there is no method of addressing the problem with them.

                              Comment

                              Working...
                              X