Q about Coro file::find

[Marc Lehmann]

On Tue, Jan 31, 2012 at 07:49:49PM +0100, Aleksandar Lazic <al-anyevent at none.at> wrote:
> I want to use more then one process which goes sequentially thru the
> directories.

Coro gives you threads, not processes, and furthermore, they all block when
perl blocks on I/O (e.g. via File::Find).

However, you can use IO::AIO, which offers aio_readdir and aio_stat,
which will be done in parallel. There is also an example program
"treescan" installed that walks a directory hierarchy and optionally stats
everything.

You can try:

   treescan -sq /NFS-SHARE/dir1

This will more or less do the same work as "du -s", but will not actually
display anything. It allows you to assess any possible speed improvement
though.

If your cache is big enough (also depending on nfs version and write
activity...), you could also try:

   treescan -sq /NFS-SHARE/dir1
   du -sk /NFS-SHARE/dir1

This first populates the cache with info about all files, so a subsequent
du might not have to ask the server again.

Coro comes with a companion module "Coro::AIO" which offers the same
requests as IO::AIO, but the block the current thread (and only the
current thread). That can make coding easier.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\