Announcement

Collapse
No announcement yet.

RE:How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • RE:How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

    First off Excellent article, no need for negitive comments here, you addressed a needed niche, we're right in the middle of testing our VC3 Infrastructure deployment and before I order my c-Class Blade and iSCSI SAN I was looking for a way to test the full functions of VM. Thank you!

    Now if only you could find a way to turn a WD My Book (Networked attached) in to a sedo iSCSI I'd really have it made

  • #2
    Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

    You could connect your book to openfiler or a Windows iSCSI machine to be used as storage, but performance wouldn't be real good fo running more than 1 or 2 VMs concurrently.
    VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
    boche.net - VMware Virtualization Evangelist
    My advice has no warranties. Follow at your own risk.

    Comment


    • #3
      Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

      Hi.

      -> jasonboche
      OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
      Why, what are the bottleneck?

      Have you tried to use VMotion, did it work?


      -> GeoffM
      There are also FreeNas and sw based StarWind, both are good and easy to set up.

      Thanks.
      Matz

      Comment


      • #4
        Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

        I tried FreeNAS and OpenFiler and didn't have much luck. I friend turned me onto DSS and it rocks. We are running Vmotion and HA w/o any issue. It runs on a USB thumb drive and they have a 1TB version for free that we use.

        http://www.open-e.com/dss_lite/data_...ow=description

        Comment


        • #5
          Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

          Originally posted by Matz View Post
          Hi.

          -> jasonboche
          OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
          Why, what are the bottleneck?

          Have you tried to use VMotion, did it work?


          -> GeoffM
          There are also FreeNas and sw based StarWind, both are good and easy to set up.

          Thanks.
          Matz
          The limitation isn't iSCSI, it's the disk you'd be runnig it on. Because you'd be running VMs on a single IDE or SATA drive in a USB "book" drive, disk I/O would start to show serious contention and bottleneck probably after the 2nd VM.

          VMotion works just great on iSCSI. Been using it and Openfiler for close to 2 years.
          VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
          boche.net - VMware Virtualization Evangelist
          My advice has no warranties. Follow at your own risk.

          Comment


          • #6
            Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

            Hi

            Thanks for great input.

            -> jasonboche
            Put together some old hw.
            Installed OpenFiler on a HP DL380 with 2x36GB for sys and 4x 300GB for the data.
            1xGbit nic.

            1. So you mean, you like OpenFiler and have used it successfully to use VMotion between 2 ESX (3,5) server(s)?

            2. Any tweaks adjustment you did have to do?

            3. What about performance?

            *******************************
            I have also installed DSS-Lite to see if that is a great iScsi application
            Have you or anyone else tesed it?
            What about the $$ version - Open-E Data Storage Server / Open-E iSCSI-R3 ?

            *******************************

            Thanks again for all input.
            Matz

            Comment


            • #7
              Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

              Originally posted by Matz View Post
              Hi

              Thanks for great input.

              -> jasonboche
              Put together some old hw.
              Installed OpenFiler on a HP DL380 with 2x36GB for sys and 4x 300GB for the data.
              1xGbit nic.

              1. So you mean, you like OpenFiler and have used it successfully to use VMotion between 2 ESX (3,5) server(s)?
              I have been using Openfiler as an iSCSI target for ESX 3.0.x and 3.5.x for the past 2 years which allows me to use and test all the enterprise features in virtual infrastructure such as VMotion, DRS, HA, DPM (experimental), etc. Shared storage really does provide a lot of flexibility for maintenance. Once you go shared storage, you'll never want to go back.

              Originally posted by Matz View Post
              2. Any tweaks adjustment you did have to do?
              Well there's the Openfiler setup which is pretty straight forward. Then there's the configuration in Openfiler to present the disk(s) to ESX. It is pretty simple and can be done relatively quickly. Openfiler is a very fast build. As far as any specific settings or hardware you might want, I"ll speak to them below.

              Originally posted by Matz View Post
              3. What about performance?
              Openfiler will install on a very wide variety of x86 and x64 hardware (if you have x64, make sure you are using the x64 compiled version of Openfiler!). As such, performance can vary greatly depending on your hardware setup. You can go as cheap as you want, recycling old hardware and minimum disk spindles which will result in a shared storage solution, but it may not perform the best. Alternatively, you could spend some money on ebay or at your local electronics store and make subtle improvements in your Openfiler hardware which will enhance the performance of your box resulting in better performing VMs and more capacity for more running VMs.

              Let's go over a few of the key bottlenecks you want to address with iSCSI.

              1. iSCSI is disk over network. Naturally for your iSCSI box you will want a Gigabit network card which your iSCSI traffic will traverse. A big bottleneck here would be going with a Fast Ethernet 10/100 NIC (or wireless!!), or an improperly configured Gb NIC. Naturally, you will want the other end of that Gb cable plugged into a Gb switch that supports Gb networking. Taking it a step further, you will want your ESX NICs to be Gb and also plugged into a Gb switch! Hopefully you are getting the point that sticking a 10/100 NIC anywhere in the mix will result in a less than stellar performing iSCSI datastore. While we're on the subject of NICs, I want to point out that there does exist a NIC bonding driver which rPath Openfiler Linux will support which means you can multihome your Openfiler box and bond the NICs to aggregate traffic throughput resulting in 2Gb+ speeds. This is a highly complex tweak and there is some fine print and caveats that you will have to take a look at but the option is there for you. I can provide you with the documentation on this if you'd like to explore it further.

              2. Processor(s): You will find out by running TOP on your Openfiler box that the processor will become loaded when throwing a lot of iSCSI traffic at it. That said, the faster the processor, the better. Openfiler supports SMP so if you can find a multi proc box, definitely go that route. My Openfiler box is currently a 4 way P4 2.x GHz.

              3. RAM: Doesn't appear to come into play much so you can rest easy in this category. My box has 9.5GB of RAM of which I don't think I'm even getting close to using a gig of it..

              4. Array controller, disk spindle count, and disk spindle speed. I saved the best for last. By far the greatest improvements (and most expensive) you can make to your iSCSI box are increasing your disk spindle count to the most possible (or affordable) and going with an array controller that will be able to keep up with the number of disk spindles and the I/O reads/writes. My server has a Compaq SMART Array 6400 array controller and 12x 146GB 10KRPM hot pluggable SCSI drives. While I can't afford 15KRPM SCSI drives, that would make another great improvement in my setup. Maybe someday.

              That's all for now, I'm about to get off the bus here. Good luck!!

              I came back to tack on a concluding statement now that I have a little more time. I wanted to point out that even the fastest of iSCSI hardware or the average fiber channel SAN will flounder a bit when overloading it with disk I/O requests from the ESX hosts. Examples usually include doing something with multiple VMs at the same time which are disk I/O intensive. For instance, powering on several VMs at the same time. Installing a large service pack on several VMs at the same time, etc. This is where you'll see your Gb NIC shine, but your disks will be working overtime trying to keep up with all the requests. If you run TOP, you'll also likely see the processors on your Openfiler box are buried. Another thing to remember is that if you're using the swISCSI initiator in ESX, there's some overhead in that guy too on the ESX side. You'll see this by running ESXTOP on your ESX hosts when your iSCSI box is working hard. Let me clarify - it's not that your VMs will start failing as the Openfiler box works hard, they will just be slower as it churns through the disk queues. Any shared storage solution has its limitations and assuming you cover all areas, your bottlenecks will usually be disk I/O so focus on disk I/O throughput when building an iSCSI box for ESX.

              Originally posted by Matz View Post
              *******************************
              I have also installed DSS-Lite to see if that is a great iScsi application
              Have you or anyone else tesed it?
              What about the $$ version - Open-E Data Storage Server / Open-E iSCSI-R3 ?

              *******************************

              Thanks again for all input.
              Matz
              Last edited by jasonboche; 12th June 2008, 14:32.
              VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
              boche.net - VMware Virtualization Evangelist
              My advice has no warranties. Follow at your own risk.

              Comment


              • #8
                Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

                Originally posted by Matz View Post
                Hi.

                -> jasonboche
                OpenFiler, do you mean you not use more than 2 VM "running" on the of iscsi?
                Why, what are the bottleneck?

                Have you tried to use VMotion, did it work?


                -> GeoffM
                There are also FreeNas and sw based StarWind, both are good and easy to set up.

                Thanks.
                Matz
                Thanks Matz I'll look over both (along with DSS) very througly - I provide some SMB consaulting as well and got to thinking this would be a great cost point solution when combined with a Western Digital Networked MyWorlBook (Or simular) they have a 1 TB and a redundent 1TB (or 2 TB's if not used in redundentcy mode) and while they have a 1 gbs Network interface I do understand it's not a true iSCSI with I/O and other performance capabilities.
                Last edited by GeoffM; 1st July 2008, 19:03.

                Comment


                • #9
                  Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

                  Hi,

                  Just a word of caution.

                  I was surprised to read the article and see that Openfiler was used in this setup in combination with ESX's Enterprise features. I therefore assume that it's connected to more than one ESX host and the hosts are sharing the same iScsi targets.

                  I had a similar setup about a year ago - three ESX hosts and a 4TB shared target on Openfiler. The problem was that ESX would contunally get corrupt vmfs volumes, volume sizes would literally fluctuate during use, VM's woud disappear etc. So I ended up calling VMWare support (which did a great job, btw) - only to find out that Openfiler is not supported as a iScsi solution for the ESX's software redirector.

                  I said "yeah, so?" - but was informed that it's not only not supported - but that, like with most other non-certified iScsi solutions, it actually lacks the ability to use a few SCSI calls - which ESX DOES use. Hence the problems I was running in to.

                  In the end I just upgraded the exisiting "real" SAN with more storage (yup, spending MAJOR money). Right now we're using the Openfiler machine as a dedicated backup target - with huge disks and perfect performance. So all is well in the end anyway

                  But: if anybody has any news about ESX and Openfiler working together WELL in a PRODUCTION environment I'd be happy to hear about it!

                  Comment


                  • #10
                    Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

                    I have been using Openfiler in my home production environment for 2 years or more with 3 ESX hosts. No problems.

                    Not many are going to suggest using a free opensource product like Openfiler for your work production environment. Spend a little money on a shared storage solution that you'll at least get support for if you have a problem.

                    Jas
                    VCDX3 #34, VCDX4, VCDX5, VCAP4-DCA #14, VCAP4-DCD #35, VCAP5-DCD, VCPx4, vEXPERTx4, MCSEx3, MCSAx2, MCP, CCAx2, A+
                    boche.net - VMware Virtualization Evangelist
                    My advice has no warranties. Follow at your own risk.

                    Comment


                    • #11
                      Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

                      Originally posted by jasonboche View Post
                      I have been using Openfiler in my home production environment for 2 years or more with 3 ESX hosts. No problems.

                      Not many are going to suggest using a free opensource product like Openfiler for your work production environment. Spend a little money on a shared storage solution that you'll at least get support for if you have a problem.

                      Jas
                      Jas -

                      Good to see you are using openfiler for 2 years. Even I am trying to setup a VMware setup at my home to practice.

                      I have installed and configured new ESX 3.5. Now I struck in configuring openfiler my esx server.

                      I was able to install opeinfiler and configured volume using following document

                      Http://www.applicationdeelivery.co.uk/blog/leew/how-to-set-up-a-free-iscsi-or-nas-storage-system-for-vmware-esx-using-openfiler

                      After installation and configuration open filer and ESX server – Configuration- Storage adapters – vmhba32

                      It shows model, iSCSI name, iSCSI Alias & Discovery methods: Send Targets

                      But it is not showing IP address & Targets: (Target is showing 0) due to this I am not able to view my iSCSI storage.

                      I did all my possibilities. I struck over here need help… J

                      Comment


                      • #12
                        Re: How to create an inexpensive iSCSI SAN for VMware ESX by David Davis

                        Hi I am looking forward to see this topic I will try open filer soon

                        Comment

                        Working...
                        X