No announcement yet.

Copy very large files across the internet...

  • Filter
  • Time
  • Show
Clear All
new posts

  • Copy very large files across the internet...

    Hi everyone,

    I'm need of a service, or application, that can allow me to copy very large files (in excess of 3 GB) from my network to an offsite location. Basically, this is a large database file that gets zipped ( 3 GB or larger), and this file needs to be transferred to my offsite location nightly.

    Currently I'm using a secure FTP server to transfer the file, but anytime my internet link 'blinks' for any reason - the transfer fails. I've tried using an FTP server that allows for upload retries, but I just need a more reliable solution.

    Can anyone give me some ideas or point me to a proven and solid solution.


  • #2
    Re: Copy very large files across the internet...

    There are a few thoughts that come to my mind after reading your post:
    1. Why is your internet connection "blinking"? General ISP unreliability? What your network's size and importance is (home, small business or large enterprise) will determine how much effort you would want to put into tracking that problem down. For instance, if it's a T1, you have definite grounds for complaining to your provider. A flakey network connection always worries me.
    2. FTP / SFTP with resume capabilities was what came to my mind after reading your first sentence. Your third sentence rained on that parade. What exactly is unreliable about ftp resumes? I would think it would work well... and if it doesn't that possibly portends pernicious peril.
    3. Finally, if it were me, I'd look into creating my own private torrent which would seem to include reliable download/upload retries by nature.
    Wesley David
    LinkedIn | Careers 2.0
    Microsoft Certifications: MCSE 2003 | MCSA:Messaging 2003 | MCITP:EA, SA, EST | MCTS: a'plenty | MCDST
    Vendor Neutral Certifications: CWNA
    Blog: || Twitter: @Nonapeptide || GTalk, Reader and Google+: [email protected] || Skype: Wesley.Nonapeptide
    Goofy kitten avatar photo from Troy Snow:


    • #3
      Re: Copy very large files across the internet...

      Just a thought: Is there a way to export the records that are new or have changed, into a set of plaintext SQL statements, that will create a small text file of only the changed data, thus eliminating the need to transfer the entire database (much of which stays the same) every night.

      Or, split the zip into smaller chunks (but you would still end up sending the entire database, albeit made up of smaller files).
      Best wishes,
      MCP:Server 2003; MCITP:Server 2008; MCTS: SBS2008


      • #4
        Re: Copy very large files across the internet...

        What database app are you using?
        Joined: 23rd December 2003
        Departed: 23rd December 2015


        • #5
          Re: Copy very large files across the internet...

          If it's a one time only and your a bit in a hurry you might create smaller rar/zip files.
          You might even use Quickpar as addition to fix the rar files on the remote site.

          However in the end I would find out why it looks like you internet connection is failing.
          Technical Consultant

          MCITP(EA, SA), MCSA/E 2003:Security, CCNA, SNAF, DCUCI, CCSA/E/E+ (R60), VCP4/5, NCDA, NCIE - SAN, NCIE - BR, EMCPE
          "No matter how secure, there is always the human factor."

          "Enjoy life today, tomorrow may never come."
          "If you're going through hell, keep going. ~Winston Churchill"