ftpcopy

NAME

ftpcopy - mirror or copy files from a FTP server

SYNOPSIS

ftpcopy ftp://host[:port]/directory [local-directory] or ftpcopy host[:port] /directory [local-directory]

DESCRIPTION

ftpcopy copies a FTP site recursivly. It afterwards deletes all files in the local directory tree which were not found on the remote site. local-directory defaults to "." - the current working directory - if the -n (--no-delete) option is used. local-directory is not needed if the --interactive option is used. Otherwise you must provide a local-directory argument.

OPTIONS

bsection


: Use NAME to login on the ftp server. The default is `anonymous'. Use an empty name to force to not log in.
: Use PASS as password to login on the ftp server. The default is `anonymous@invalid.example'. Use an empty password to enforce not sending any password.
: Send ACCOUNT as the account name using the ACCT ftp command if the ftp servers asks for one.
: ACCOUNT in ftp speak is some kind of sub account in some other terminologies. If you don't understand what it means you have a good chance to never need this option anyway. If you think you need it please try the -u option first.
: Number of tries to connect and log in. The default is 1, meaning that ftpls will give up after the first error.
: This option was added in version 0.3.0.
: ftpcopy will sleep for NUMBER seconds after a try to connect or login has failed. The default is 5, a zero is treated as one, and abuse (especially together with --tries) is likely to annoy the server administrator.
: This option was added in version 0.4.5.
: Only use IPv4, even is IPv6 is available. This option effectively disallows the use of IPv6, except for DNS queries.
: This option was added in version 0.6.0.
: Only use IPv6, even is IPv4 is available. This option effectively disallows the use of IPv4, except for DNS queries.
: This option was added in version 0.6.0.

bsection


: this controls the amount of logging done: 0: nothing except warnings and error messages 1: downloads and deletes 2: links/symlinks created, files we already got 3: useless stuff
: The default is 1.
: Log transfer rates.
: This option was added in version 0.3.9.
: Report progress to the standard error output (file descriptor 2).
: This will print a report of the download every second: a short form of the file name, the bytes got and expected and the percentage received.
: This option was added in version 0.6.0.

bsection


: Download only files modified in the last NUMBER days. Locally existing copies of the not downloaded files will be kept.
: Download only files up to BYTES bytes length. Locally existing copies of overlong files will be deleted during the clean-up step.
: This option was added in version 0.5.1.
: Exclude files and directories matching WILDCARD (note that this means shell style wildcards, not regular expression like those of grep).
You can repeat this option as often as you want. You can intermix it with the -i option.
See below, for more information.
: This option was added in version 0.3.0.
: Include files and directories matching WILDCARD. You can repeat this option as often as you want. You can intermix it with the -x option.
See below, for more information.
: This option was added in version 0.3.0.
: Do not compare file sizes when checking the remote file has to be downloaded.
: This option was added in version 0.4.4.
: Do not compare file modification times when checking the remote file has to be downloaded. This option may be combined with --ignore-size, in which case a file will never be downloaded regardless of changes in file size or modification time. In other words: ftpcopy will not download any updates.
: This option was added in version 0.4.4.
: Descend at most LEVEL directories deep. 0 means do not enter sub directories at all, 1 means enter sub-directories, but not their sub-directories, and so on. The default is 2^32-1, meaning enter all.
: This option was added in version 0.5.2.

bsection


: Do not delete files. This influences the cleanup step at the end, when ftpcopy removes files not found on the server. It doesn't stop ftpcopy from deleting files when it detects something in it's way during a download.
: Do not delete more then COUNT files. This option may be useful to limit the impact of a tempoary loss of files on the server. The default is 0, meaning unlimited. This, too, only influences the cleanup step.
: This option was added in version 0.4.5.

bsection

Only create the directory hierarchie, do not download files. Any file in the tree will be deleted unless the -n option is also given. Note: if the -s option is also given then even symbolic links to files will be created (without the files, of course).

Do not copy or delete files: Only show what would be done.

This option was added in version 0.3.6.

Timeout to use for network read/write and connect operations. The default is 30 seconds and is usually sufficient.

This option was added in version 0.3.8.

Limit the transfer rate of file downloads to about that many bytes per seconds. The implementation is crude and simple, by sleeping up to one second between network reads, and therefore does not even try to limit the rate exactly to that number. On the other hand it usually works and is unlikely to break things by causing timeouts.

This option was added in version 0.4.7.

This option tells ftpcopy to ignore any directories given on the command line, and to read commands from the standard input. Each command consists of two lines, the first being a directory on the remote server, and the second a local directory.

ftpcopy will print an END-OF-COPY line after each operation.

This option was added in version 0.3.6.

bsection

or "things you may need in a less than perfect world":

Do directory listings in ASCII mode instead of binary mode. Use this option if the FTP server is unable to correctly list directories in binary mode, for example, if you see a message like this (outside this manual page this usually comes on one line):

  ftpcopy: fatal: received unwanted answer to LIST:
  426 Data connection: Illegal seek.

This option was added in version 0.5.2.

Add OPTS to LIST command.
This allows to pass arbitrary options to the FTP servers LIST command. Note that ftpcopy does not cope well with recursive directory listings.

This option was added in version 0.3.0.

Deal with symbolic links. This is only useful to mirror sites which create listings through /bin/ls. This will fail if a file name in a link contains a ` -> ' sequence.

Do not use the poll() system call even if it's available, but use select(). This allows ftpcopy to be used together with the runsocks program from the socks5 reference implementation. Please note that you'll need a directly reachable name server anyway, as the DNS library in use does not support SOCKS (you can always use IP addresses).

This option was added in version 0.3.8.

Use the MDTM FTP command to determine file modification time.

The default is to take the times from the directory listings. This doesn't work if the server implements an inferior listing format (most do) and doesn't send time stamps in universal coordinated time (UTC).

This option makes ftpcopy send a MDTM command for any file it might want to download. The drawback is that this eats performance: ftpcopy usually sends just one command for a complete directory its traverses. With the --mdtm option it has to send an additional command for any file.

This option was added in version 0.3.10.

Allow data connections to the address IP4.

Normally ftpcopy only accepts data connections to the addresses of the host it connected to. The list of addresses contains the IP addresses of the host, as returned by the DNS query, or the IPv4 address of the host, in case an IPv4 address was given on the commandline.

This option allows to add one or more IP addresses, separated by commas, to the internal list. It may be given more than once.

You may want to use this option if you are behind some kind of machine (gateway) doing NAT (network address translation) or maquerading and ftpcopy prints an error message containing "illegal redirect by FTP server".

Note: If you are not behind a NAT gateway, then do not use this option without thinking: FTP redirects may be used to launch denial of service attacks against innocent targets.

The environment variable FTPCOPY_ALLOW_PASV_IP has the same function.

This option was added in version 0.6.1.

Do not try to resume downloads.

The REST command, needed to resume a failed download, is badly specified and is likely to be misinterpreted and -implemented. Use this option in case of trouble.

This option was added in version 0.6.0.

Change all local file names to lower case.

Use this only if you are absolutely sure that the remote side doesn't contain any files or directories whose names collide with each other. Otherwise this will waste bandwidth.

This option was added in version 0.3.8.

REMOVED OPTIONS



: These options have been added in version 0.3.0 and removed in 0.3.7. --keep-dir made ftpcopy keep a local directory even if it had been replaced by a link to an other directory or a file, --remove-dir did exactly the opposite. --keep-dir was the default for historical reasons.
The current behaviour is like that of --remove-dir: ftpcopy creates a copy of the remote ftp site.

IN/EXCLUDE

In- and exclude lists are internally mixed together, keeping the order in which they were given. The list starts with an implicit `include *'. ftpcopy honors the last match. The wildcard matching is done against the full remote path of the file. The / character has no special meaning for the matching: it is treated like any other. Note: you have to include top level directories of files or directories you want to include. Something like this will not work:
--exclude "*" --include "/w/h/e/r/e/file.c" You need to include /w, /w/h and so on.

EXAMPLES

bsection

        ftpcopy  \
        --exclude '*.cdb'  \
        --exclude '*software/precompiled*' \
        cr.yp.to / /private/file/0/mirror/cr.yp.to

This means: i'm not interested in .cdb files. precompiled stuff is also not downloaded. the host to connect to is cr.yp.to. the remote directory is /. and /private/file/0/mirror/cr.yp.to is the local directory.