Announcement

Collapse
No announcement yet.

Please help in S.C & VMotion VLAN Setup

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Please help in S.C & VMotion VLAN Setup

    Hello Folks,

    I want to configure my ESX Servers to works with VLAN under Nortel Switches 4542 GT in Stack-mode.

    Server Configuration:
    2- DL380 G5, each with Single Port HBA, 6 pNICs, 2 pCPU Dual-Proc.
    2- BL460 G1, Each with Dual-port HBA, 6 pNICs, 1 pCPU Dual-Proc.

    Setup:
    vSwitch0 = ESX Networks: Service Console "172.16.20.0/24" && VMotion "10.1.0.0/24" using VLANs.
    vSwitch1= Production Network: 128.104.0.0/16
    vSwitch2 = DMZ Network: 192.168.1.0/24

    Private Network for ESX:
    vSwitch0 with 2 pNICs connected vmnic0 & vmnic1 Teamed on the vSwitch Level.
    2 Portsgroup.
    1 Service Console
    1 VMotion

    In the portgroup Setting for S.C --> Nic Teaming is vmnic0 Active and vmnic1 Standby
    In the portgroup setting for VMotion --> Nic Teaming is the vmnic1 Active and vmnic0 Standby.

    vmnic0 connected to pSwitch on port configured with VLAN 2
    vmnic1 connected to pSwitch on port configured with VLAN 3

    Production Network:

    vSwitch1 with 2 pNICs connected vmnic2 & vmnic3 Teamed on the vSwitch Level.
    1 Portgroup.
    Production VMs

    vSwitch2 with 2 pNICs connected vmnic4 & vmnic5 Teamed on the vSwitch Level.
    1 Portgroup
    DMZ VMs

    ================================================== =============================

    If
    I assign an IP Address to the S.C with the same IP which is configured
    on the VLAN, "Without Assigning the ((VLAN ID)) in the portgourp,
    through pServer, i can reach to other ESX Host Service Console, because
    both of them are on the same VLAN.

    As soon as I assign the
    ((VLAN ID)) on the portgroup of S.C, i lost the connectivity to the
    server, and I started troubleshooting the vswif0 to create another
    Service Console Network in order to access it the ESX Host. "And the
    same applies on the VMotion Network".

    The same goes to the
    VMotion network as well. From the pSwitch, both the VLANs are reachable
    to 172.16.20.0/24 Service Console, and 10.1.0.0/24 for VMotion Network.

    I want the Service Console Network, can talk to the VMotion Network and vice versa to get the VMotion works.

    Service Console:
    IP:172.16.20.2/24
    D.G: 172.16.20.1
    DNS: 172.16.20.57 "This host is connected to the same VLAN where the ESX hosts connected". It's a VC and DNS Server.

    VMkarnal:
    IP:10.1.0.2/24
    D.G: 10.1.0.1

    From,
    within the ESX Host, I'm unable to reach to the Default Gateway of the
    VMotion Network using vmkping. Nor the Service Console able to reach to
    the VMotion Network.

    Moreover, I wanted to reach to the Service
    Console Network 172.16.20.0 via 128.104.0.0 Network to do my
    Administrative Task. In this case, do I have to add a Static Route in
    the Service Console, in order for the VI Clients reach from Production
    Network?


    Further Testing:

    have UnTagged the ports for both VLANs. I setup both PortGroups S.C & VMKernel without VLAN ID.
    I
    got one host can ping VMkernel PortGroup on another host via COS ping.
    Also from the same host tried vmkping S.C IP and D.Gateway. It's
    successul.

    However, from the another host I can reach the
    first host S.C IP but not VMkernel. Nor the VMKernel able to reach it's
    D.Gateway.

    Since the both VLANs are reachable within the pSwitch. Do I have to use a port Trunking, and assign different VLAN ID "The Trunked vLAN" in each Portgroup *S.C & VMotion*?

    In additional to what i have mentioned to earlier regarding the NIC Teaming.
    Both, vmnic0 & vmnic1 assigned to vSwitch0, in the NIC Teaming
    Setting of vSwitch0, both vmnic0 & vmnic1 as Active/Active. And
    within each PortGroup, S.C = vmnic0 Active & vmnic1 Standby. And
    VMkernel = vmnic1 Active & vmnic0 Standby. Is this Setting may
    confusing the VLAN to work properly?

    Furter troubleshooting I'm going to make;
    1. Remove the Nic Teaming from the PortGroups.
    2. Configure the vSwitch0 with only vmnic0 on both hosts, assuming a pNIC failure.
    3. Test both hosts can ping each other S.C and D.Gateway 172.16.20.1
    4. Cofigure VMkernel with the prospetive VLAN IP schema & test vmkping whether it can reach S.C IP & it's D.Gateway.
    5. If it's success, then will configure the same on the other host and test the connectivity between the hosts.

    If not, do I have to configure a Trunking on the pSwitches and make both VLANs 3 & 4 members of the Trunked VLAN?


    Further Testing:

    In the pSwitch on port 5 where the vmnic0 connected, i have set the port ot TagAll.

    Result:
    1. I lost the connectivity to the vswif0 "Service Console IP". But within this vSwitch0, I have VM Network Portgroup, and one of the Virtual Machines IP's is set to the same VLAN of the Service Console, it's reachable

    This without VLAN ID specified on any PortGroup.

    2. While the the pSwitch Port 5 is set to TagAll, I have specify a VLAN ID to both the Service Console & VM Network Portgroups, I got the connectivity back up on the Service Console as well as on the VM Network.

    Now I have the vmnic0 connected to port 5 "VLAN 3 - ip: 172.16.20.0" on the pSwitch and vmnic0 linked to vSwitch0. Also, vmnic1 is connected to port 6 "VLAN 4 - ip: 10.1.0.0" on the pSwitch and vmnic1 is Linked to vSwitch0 as well.

    Question: How to get the VMotion works since Service Console is setting on the different Network and VMotion on Different Network?

    Do I have to specify a Static Route in the ESX Server in order for the VMKernel Network sees the Service Console Network?

    Thanks,
    ================================
    HND: Higher National Diploma in
    Computer Science(IT)


    Passed:
    MCSA+Security 2003, VCP3, VCP4
    Done:VMware DSA
    ================================[/COLOR]

  • #2
    Re: Please help in S.C & VMotion VLAN Setup

    This is a long post - and it seems like a well thought out plan as to how you would like your setup...

    The way it works, is like this

    If your ports on the physical switch are trunked (i.e. will allow more than one vlan to pass over it) then you would have to set the vlan id on the SC interface or the VMKernel interface

    If the vmkernel interface needs to be passed to a default gateway that is different to the SC's gateway, then you will have to set the vmkernel gateway address.

    There should be no need to add any additional static routing to the ESX
    Last edited by Maish; 2nd December 2008, 10:01.
    Maish
    ----------------------------------------------------------
    Technodrone|@maishsk|Author of VMware vSphere Design
    VMware vExpert 2013-2010,VCAP5-DCA/DCD,VCP
    MSCA 2000/2003, MCSE 2000/2003
    A proud husband and father of 3 girls
    ----------------------------------------------------------
    If you find the information useful please don't forget to give reputation points sigpic.

    Have a good one!!

    Comment


    • #3
      Re: Please help in S.C & VMotion VLAN Setup

      Hello Maish,

      The problem is i'm not a networking engineer, I'm just an Admin & Infrastructure guy. I have brought a Networking Engineer, and he said sorry, I don't know how to solve this issue , i thought the only way is to dig down on the Nortel Switch and try whatever possible could be archived by me


      Setup is consist of 2 pSwitches in Stack-Mode.
      I have on the pSwitch port5 is on VLAN 172.16.20.0/24 Network and port6 is on VLAN 10.1.0.0/24 Network.

      If I made the Trunking Ports for both ports "VLAN" I have to set the port Tagging to UnTagAll or to TagAll?

      If I made UnTagAll or TagAll on the ports, both PortGroups must have the VLAN ID? Or only one PortGroup?

      The Default Gateway of the Service Console is the VLAN IP 172.16.20.1/24 and the Default Gateway of the VMkernel is the VLAN IP 10.1.0.1/24


      Is there any configuration in the Switch side I have to make?

      Thanks,
      Habibalby
      ================================
      HND: Higher National Diploma in
      Computer Science(IT)


      Passed:
      MCSA+Security 2003, VCP3, VCP4
      Done:VMware DSA
      ================================[/COLOR]

      Comment


      • #4
        Re: Please help in S.C & VMotion VLAN Setup

        Thing is I am also not a network guy, so I cannot be of much assistance to you here..


        Maybe here ??
        Maish
        ----------------------------------------------------------
        Technodrone|@maishsk|Author of VMware vSphere Design
        VMware vExpert 2013-2010,VCAP5-DCA/DCD,VCP
        MSCA 2000/2003, MCSE 2000/2003
        A proud husband and father of 3 girls
        ----------------------------------------------------------
        If you find the information useful please don't forget to give reputation points sigpic.

        Have a good one!!

        Comment


        • #5
          Re: Please help in S.C & VMotion VLAN Setup

          Hello,

          The problem has been solved by the following steps


          1. Disabled the Trunking ports in the pSwitches
          2. In the pSwitch I have set the VLAN3 and VLAN4 member of each other.
          3. Set the VLAN ID in the Service Console and VMkernel Portgroups.
          4. In the pSwitch, I have set the Tagging to TagAll for both ports 5 & 6 in both pSwitches.
          5. PVID are the same on both ports on both pSwitches.
          6. For the second host, did the same steps entering the VLAN ID in the correspondence portgroup.

          Result:

          * Both hosts they can reach other.
          * Both hosts they can reach the configured Default Gateway.
          * Both hosts they can reach the VMkernel IP Address of each host using vmkping command.
          * Test vmnic0 failure = Still I can reach other host IP Address & hostname, and the Default Gateway as well as the VMkernel IP Address of other host in the Cluster
          * Test vmnic1 failure = Still I can reach other host IP Address & hostname, and the Default Gateway as well as the VMkernel IP Address of other host in the Cluster.

          Now, when I tried to re-configure the VMware HA, I got an error while I configuring the VMware HA, I got an errors:

          HA agent on hosts.esx.local in cluster ESX in Development Network has an error: No Active Primaries Found "Means no Primary Host found in the Cluster"
          CMD startagent failed: Internal AAM Error - Agent could not start

          Steps followed to overcome this problem:
          1. Put the hosts in Maintenance Mode.
          1. Disconnected the hosts from the Virtual Center.
          1. Removed the hosts from the Virtual Center
          1. Restarted the Virtual Center, just in case to get everything cleared.
          1. Added both hosts to the Cluster, and enabled VMware HA. WAW, everything works as expected successfuly.



          Regards,
          Habibalby
          ================================
          HND: Higher National Diploma in
          Computer Science(IT)


          Passed:
          MCSA+Security 2003, VCP3, VCP4
          Done:VMware DSA
          ================================[/COLOR]

          Comment


          • #6
            Re: Please help in S.C & VMotion VLAN Setup

            happy you could sort it out...
            Maish
            ----------------------------------------------------------
            Technodrone|@maishsk|Author of VMware vSphere Design
            VMware vExpert 2013-2010,VCAP5-DCA/DCD,VCP
            MSCA 2000/2003, MCSE 2000/2003
            A proud husband and father of 3 girls
            ----------------------------------------------------------
            If you find the information useful please don't forget to give reputation points sigpic.

            Have a good one!!

            Comment


            • #7
              Re: Please help in S.C & VMotion VLAN Setup

              Good Early Morning,

              Yeah, I was insisting to know how to configure the pSwitch VLANs. And it's good practice for my learning and experience as well.


              But, most of the forums thread and blogs regarding the ESX VLAN, they are mentioning that the VLANs must be trunked in order to aheive the VLANing works in ESX.

              In my setup, i didn't make Trunking, but i made a VLANs membership of each other and it works like a charm

              Thanks for you support,
              Habibalby
              ================================
              HND: Higher National Diploma in
              Computer Science(IT)


              Passed:
              MCSA+Security 2003, VCP3, VCP4
              Done:VMware DSA
              ================================[/COLOR]

              Comment

              Working...
              X