Announcement

Collapse
No announcement yet.

wrinting 1000000 files in one directory

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • wrinting 1000000 files in one directory

    Hi, I have this user at our university who has a program taking 20 pictures (7-8k jpg) per second. THe 1st hour, the system (win XP Pro) is ok. After 1 hour, it slows down to the pointot 1 pic/second or even less. All pics are in the same dir. Can somebody tell me why it is like that ? Why is the system ok for a while, and then slows up ? Are there limitations in the # of ilfes written in a directory ?
    Many thanks in advance

    Simon

  • #2
    Re: wrinting 1000000 files in one directory

    Does rebooting the workstation help?

    Is Windows fully up to date on the machine?

    In any case, I suspect the process is just chewing through RAM. As less RAM becomes available, the system will run a lot slower.

    It's also possible that the disk is becoming fragmented, although I wouldn't expect that to slow the machine down so much.
    Gareth Howells

    BSc (Hons), MBCS, MCP, MCDST, ICCE

    Any advice is given in good faith and without warranty.

    Please give reputation points if somebody has helped you.

    "For by now I could have stretched out my hand and struck you and your people with a plague that would have wiped you off the Earth." (Exodus 9:15) - I could kill you with my thumb.

    "Everything that lives and moves will be food for you." (Genesis 9:3) - For every animal you don't eat, I'm going to eat three.

    Comment


    • #3
      Re: wrinting 1000000 files in one directory

      Originally posted by gforceindustries View Post
      Does rebooting the workstation help?

      Is Windows fully up to date on the machine?
      Yes it is

      In any case, I suspect the process is just chewing through RAM. As less RAM becomes available, the system will run a lot slower.

      It's also possible that the disk is becoming fragmented, although I wouldn't expect that to slow the machine down so much.
      I had an update lately, windows XP stores the file names also with the 8.3 pattern, and as there are a LOT of files, the 8.3 names are all the same, and XP cannot deal well with this stuff. So we will try to give a very short name to the files, and we will see what's happening. Thanks for your answer anyway. I'll keep the answers coming here.
      Simon

      Comment


      • #4
        Re: wrinting 1000000 files in one directory

        It may be using 8.3 filenames because of the program generating the files, rather than because of something to do with Windows. Definately worth trying with shorter names to see if that helps.
        Gareth Howells

        BSc (Hons), MBCS, MCP, MCDST, ICCE

        Any advice is given in good faith and without warranty.

        Please give reputation points if somebody has helped you.

        "For by now I could have stretched out my hand and struck you and your people with a plague that would have wiped you off the Earth." (Exodus 9:15) - I could kill you with my thumb.

        "Everything that lives and moves will be food for you." (Genesis 9:3) - For every animal you don't eat, I'm going to eat three.

        Comment


        • #5
          Re: wrinting 1000000 files in one directory

          Does the application release memory once its written the file??

          Does the machine have enough disk space to handle the amount of files being written to the folder??

          Comment


          • #6
            Re: wrinting 1000000 files in one directory

            Originally posted by wullieb1 View Post
            Does the application release memory once its written the file??

            Does the machine have enough disk space to handle the amount of files being written to the folder??
            Yes, and Yes, when the app is running, I fired task manager, the memory is low and the proc is ok, even when it's beginning to slow up.

            Comment


            • #7
              Re: wrinting 1000000 files in one directory

              Have you tried any of the performance monitor (Perfmon) physicalDisk and Memory counters?
              Caesar's cipher - 3

              ZKHQ BRX HYHQWXDOOB GHFLSKHU WKLV BRX ZLOO UHDOLVH LW ZDV D ZDVWH RI WLPH!

              SFX JNRS FC U6 MNGR

              Comment


              • #8
                Re: wrinting 1000000 files in one directory

                No I didn't fire perfmon, but the problem seems to be solved now, and it had to do with the fact that even with XP, the system stores the filenames in 8.3 as well. Now that the files are named with just one single letter (with incremental numbers automatically after), all goes well, for compatibility issues, see this article here : http://tinyurl.com/6osunu
                The names before were some like Swiss1998000001.jpg, then Swiss1998000002.jpg but XP retained the alias 8.3 like Swiss~01.jpg, and after a lot of files couldn't handle all the names. I don't know though if there is a way of disabling this "feature" in XP.

                Many thaks to all who replied.

                Simon

                Comment


                • #9
                  Re: wrinting 1000000 files in one directory

                  Thanks for posting back with a fix for your problem.

                  This helps others if they come across the same problem.

                  Comment


                  • #10
                    Re: wrinting 1000000 files in one directory

                    Originally posted by wullieb1 View Post
                    Thanks for posting back with a fix for your problem.

                    This helps others if they come across the same problem.
                    No prob, I think it's normalto do it...

                    Comment

                    Working...
                    X