Announcement

Collapse
No announcement yet.

3620 ICMP Destination unreachable for outbound DNS replies

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • 3620 ICMP Destination unreachable for outbound DNS replies

    I've got a 3620 that doesn't like DNS replies. I'm hosting a DNS server on my network that serves DNS queries for my domain. I'm running NAT, and have set up the TCP and UDP rules to get DNS queries to my DNS server.

    However, every time my DNS server sends a reply, my router send back "ICMP Destination unreachable" and my DNS replies never make it back to the requester.

    Below is a snip from my packet capture, taken by tshark on the DNS server.
    439.282534 172.16.3.11 -> 76.193.82.126 DNS Standard query response CNAME freon.ljb2of3.net A 207.155.37.246
    439.285486 172.16.3.1 -> 172.16.3.11 ICMP Destination unreachable (Host unreachable)

    This happens right after every query response.

    I know I can reach the remote host, because I am doing remote desktop to there to test DNS!!

    I've ruled out the config of the DNS server, since I've tried my new DNS server, which was working before I put in the new router, and my old DNS server, which worked for a very long time before that.

    I've attached my router config. If anybody has any idea how I can get DNS up and running on my network again, I'd much appreciate help.

    Thanks, Landy
    Attached Files

  • #2
    Re: 3620 ICMP Destination unreachable for outbound DNS replies

    Anybody? I still haven't been able to get this working.

    -ljb2of3

    Comment


    • #3
      Re: 3620 ICMP Destination unreachable for outbound DNS replies

      Landy,

      Are there any other devices in front of your 3620, (i.e a firewall)? If so what type? What is the output when you do a "show ip nat translation"? Can you even see if the router is actually trying to perform the nat translation?

      Ryan

      Comment


      • #4
        Re: 3620 ICMP Destination unreachable for outbound DNS replies

        Ryan,

        In front of the router is my DSL modem, which is actually a router in it's own right. However, the cisco router is set as the DMZ host in the modem, so all traffic goes that direction.

        Below is the snippet from tshark. It shows that the packets are in fact making it through the router to the dns server.

        Code:
         30.458855 12.156.42.194 -> 172.16.3.5   DNS Standard query A www.ljb2of3.net
         30.461412   172.16.3.5 -> 12.156.42.194 DNS Standard query response CNAME freon.ljb2of3.net A 207.155.37.246
         30.464309   172.16.3.1 -> 172.16.3.5   ICMP Destination unreachable (Host unreachable)
        Below is the output from my show ip nat translations.

        Code:
        Cisco-3620#show ip nat translations tcp
        Pro Inside global         Inside local          Outside local         Outside global
        tcp 192.168.254.5:1235    172.16.4.3:1235       205.188.210.165:5190  205.188.210.165:5190
        tcp 192.168.254.5:1201    172.16.4.3:1201       64.12.24.177:5190     64.12.24.177:5190
        tcp 192.168.254.5:139     172.16.3.3:139        192.168.76.1:1156     192.168.76.1:1156
        tcp 192.168.254.5:143     172.16.3.5:143        12.156.42.194:49768   12.156.42.194:49768
        tcp 192.168.254.5:1178    172.16.4.3:1178       216.155.193.174:5050  216.155.193.174:5050
        tcp 192.168.254.5:4861    172.16.4.14:4861      72.14.207.99:80       72.14.207.99:80
        tcp 192.168.254.5:1372    172.16.4.3:1372       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1373    172.16.4.3:1373       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1381    172.16.4.3:1381       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1384    172.16.4.3:1384       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1386    172.16.4.3:1386       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1022    172.16.3.5:22         ---                   ---
        tcp 192.168.254.5:3390    172.16.3.3:3389       ---                   ---
        tcp 192.168.254.5:3392    172.16.4.3:3389       ---                   ---
        tcp 192.168.254.5:3393    172.16.3.3:3392       ---                   ---
        tcp 192.168.254.5:1110    172.16.4.14:1110      62.146.40.167:443     62.146.40.167:443
        tcp 192.168.254.5:1175    172.16.4.3:1175       216.239.51.125:5222   216.239.51.125:5222
        tcp 192.168.254.5:1459    172.16.4.14:1459      77.247.177.111:80     77.247.177.111:80
        tcp 192.168.254.5:1901    172.16.4.3:1901       129.244.3.130:143     129.244.3.130:143
        tcp 192.168.254.5:1179    172.16.4.14:1179      212.188.147.226:80    212.188.147.226:80
        tcp 192.168.254.5:3476    172.16.4.3:3476       ---                   ---
        tcp 192.168.254.5:4590    172.16.4.3:4590       64.191.203.30:80      64.191.203.30:80
        tcp 192.168.254.5:5846    172.16.4.3:5846       204.2.243.65:80       204.2.243.65:80
        tcp 192.168.254.5:25      172.16.3.5:25         124.120.237.176:17462 124.120.237.176:17462
        tcp 192.168.254.5:5800    172.16.4.3:5800       ---                   ---
        tcp 192.168.254.5:1239    172.16.4.13:1239      67.133.150.12:80      67.133.150.12:80
        tcp 192.168.254.5:4751    172.16.4.14:4751      192.204.11.32:80      192.204.11.32:80
        tcp 192.168.254.5:25      172.16.3.5:25         190.42.145.206:1727   190.42.145.206:1727
        tcp 192.168.254.5:4049    172.16.4.14:4049      205.188.7.148:443     205.188.7.148:443
        tcp 192.168.254.5:4056    172.16.4.14:4056      64.12.165.108:443     64.12.165.108:443
        tcp 192.168.254.5:5900    172.16.4.3:5900       ---                   ---
        tcp 192.168.254.5:1219    172.16.4.3:1219       207.46.108.25:1863    207.46.108.25:1863
        tcp 192.168.254.5:8080    172.16.4.3:8080       ---                   ---
        tcp 192.168.254.5:59298   172.16.3.4:59298      ---                   ---
        tcp 192.168.254.5:4033    172.16.4.3:8084       ---                   ---
        tcp 192.168.254.5:4050    172.16.4.14:4050      205.188.248.153:443   205.188.248.153:443
        tcp 192.168.254.5:4879    172.16.4.14:4879      75.125.81.10:80       75.125.81.10:80
        tcp 192.168.254.5:38897   172.16.3.3:38897      ---                   ---
        tcp 192.168.254.5:4916    172.16.4.14:4916      209.8.115.113:80      209.8.115.113:80
        tcp 192.168.254.5:65535   172.16.3.10:3389      ---                   ---
        tcp 192.168.254.5:25      172.16.3.5:25         ---                   ---
        tcp 192.168.254.5:53      172.16.3.5:53         ---                   ---
        tcp 192.168.254.5:80      172.16.3.5:80         ---                   ---
        tcp 192.168.254.5:143     172.16.3.5:143        ---                   ---
        tcp 192.168.254.5:139     172.16.3.3:139        223.1.1.128:6072      223.1.1.128:6072
        tcp 192.168.254.5:4051    172.16.4.14:4051      205.188.9.185:443     205.188.9.185:443
        tcp 192.168.254.5:3392    172.16.4.3:3389       12.156.42.194:42095   12.156.42.194:42095
        
        
        Cisco-3620#show ip nat translations udp
        Pro Inside global         Inside local          Outside local         Outside global
        udp 192.168.254.5:11      172.16.3.2:123        80.96.120.249:123     80.96.120.249:123
        udp 192.168.254.5:14      172.16.3.6:53         193.0.0.196:53        193.0.0.196:53
        udp 192.168.254.5:123     192.168.254.5:123     204.15.208.61:123     204.15.208.61:123
        udp 192.168.254.5:13      172.16.3.6:53         202.12.27.33:53       202.12.27.33:53
        udp 192.168.254.5:15      172.16.3.6:53         192.12.94.30:53       192.12.94.30:53
        udp 192.168.254.5:16      172.16.3.6:53         202.12.29.59:53       202.12.29.59:53
        udp 192.168.254.5:59298   172.16.3.4:59298      ---                   ---
        udp 192.168.254.5:38897   172.16.3.3:38897      ---                   ---
        udp 192.168.254.5:53      172.16.3.5:53         ---                   ---
        udp 192.168.254.5:12      172.16.3.6:53         216.97.160.4:53       216.97.160.4:53
        FYI: 172.16.3.5 is a virtual ip address, 172.16.3.6 and 172.16.3.7 are a two node cluster that has 172.16.3.5 when they are the master node.

        -Landy

        Comment


        • #5
          Re: 3620 ICMP Destination unreachable for outbound DNS replies

          Landy,

          Thanks for extra information... I don't think it is the router per se. Take a look at the following:

          udp 192.168.254.5:53 172.16.3.5:53 --- ---

          and then take at these:

          udp 192.168.254.5:13 172.16.3.6:53 202.12.27.33:53 202.12.27.33:53
          udp 192.168.254.5:15 172.16.3.6:53 192.12.94.30:53 192.12.94.30:53
          udp 192.168.254.5:16 172.16.3.6:53 202.12.29.59:53 202.12.29.59:53

          It looks like the real inside local address is responding to dns request instead of your virtual inside local address of 172.16.3.5. That's why the response gets back to the requestor before the router throws the port unreachable message. What would be interesting is if you could post the detailed icmp unreachable message. When you get any type of icmp error part of the error includes the IP header that produced the error. If you want to work on this more please feel free to contact me on IM.

          Ryan

          Comment


          • #6
            Re: 3620 ICMP Destination unreachable for outbound DNS replies

            Ryan,

            I tried making Bind listen only on the virtual IP address. No change. Attached is the full output from wireshark. The capture was done on the dns server itself, so the capture is from its perspective of the network.

            -Landy

            PS I may hit you up on IM later... wanted to go ahead and post here for the benefit of others who may come up with a similar problem in the future.
            Attached Files

            Comment


            • #7
              Re: 3620 ICMP Destination unreachable for outbound DNS replies

              Landy,

              This is definitely a good one Is there any reason why that DNS is setup in a cluster instead of doing the traditional primary secondary? I really do think that the issue is has something to do with your clustered setup. Maybe try running BIND on the loopback address on both servers. I am not quite sure because I have never run any type of DNS service on a cluster. I am still willing to help but I will have to do a little research on this.

              Ryan

              Comment


              • #8
                Re: 3620 ICMP Destination unreachable for outbound DNS replies

                Ryan,

                The cluster is a heartbeat cluster, only one node is actually active at a time. DNS is on the cluster simply because thats what I've got handy. The cluster runs my email and web server as well.

                I just tried having bind run on the loopback address. It didn't like that one bit. Even local dns requests timed out then.

                Before I completely took down my old dns server (from before it was on the cluster) I had tried sending dns traffic to it. It was a normal sort of dns server, and I still had the same issues. However, I will set up bind on a test machine in the next few days to try it out again to be sure.

                Thanks very much for attempting to help me, I appreciate it.

                -Landy

                Comment


                • #9
                  Re: 3620 ICMP Destination unreachable for outbound DNS replies

                  Landy,

                  You are more than welcome! I love a good challenge. Besides that is what a community forum is all about. Please let me know if I can further assist. One last thing; I did find this link on BIND HA clustering. It may help it may not but I will post it anyway for your reading pleasure.

                  http://wiki.samba.org/index.php/7.0._BIND_DNS

                  Ryan

                  Comment


                  • #10
                    Re: 3620 ICMP Destination unreachable for outbound DNS replies

                    Hey i know this sounds really really elementary but the one thing i've learned from a good friend of mine who's a security admin on citrix servers is that it's always something elemtary when it comes to server issues so i'm going to suggest two things that no-one else has yet.

                    first make sure that you're not blocking icmp from passing through on the new router if security is enabled it most likely is.

                    secondly many isp's block dns requests and other types of traffic from traveling on common ports due to security risks for example most isp's now block smtp traffic on port 25 it may take some investigation but it is altogether possible that you need to change from the default ports.

                    I don't mean to make it sound like the people here are not intelligent many of you most likely have much more experience then I however there's one thing i've learned and that is the more experience you have the more quickly ppl tend to jump to solutions which are often times more complex then they need to be. Since none of us but the person with the problem is able to actively touch this server and configuration I suggest this merely as a coverall.

                    Comment

                    Working...
                    X