Hi,

I have a problem with a program that tries to download files with curl,
with some coming from sourceforge. "dl.sourceforge.net" is an alias
for a list of mirrors, as if it was a machine with multiple IP
addresses. Curl nicely tries the addresses until it finds one that
accepts a connection.

The problem I have is that several sourceforge mirrors seem poorly
maintained: some don't have all the files, some have a different
directory layout. For instance one (heanet.dl.sourceforge.net) has
host/sourceforge, most of the others have host/pub/sourceforge. Thus
curl downloads fail as soon as that broken server gets picked up. My
script retries a few times, but it always gets this rotten FTP server,
so eventually it just fails to get the file, and it's all downhill from
there on. If I try some minutes later, it works as it finds another
FTP server.

What would be the best way to make curl select a mirror that actually
works? I tried looking for an option that would tell curl to keep on
trying different addresses associated with the host name until one of
them succeeds, even if one of them allows login but fails actual
download, but couldn't find any such thing. In the past I used to
hardcode a specific sourceforge mirror instead of the generic host
name, but then I discovered that a lot of the servers go up and down
like yoyos or bounce connections semi-randomly. Not good when you are
writing a script that's "just supposed to work"...

Should I start playing tricks with "dig" and tell curl which mirror
specifically it should try? I'd rather avoid that, but at the moment
it looks like there's not much of a choice...

Any help would be greatly appreciated!

Lassi

-------------------------------------------------------
This SF.net email is sponsored by OSDN's Audience Survey.
Help shape OSDN's sites and tell us what you think. Take this
five minute survey and you could win a $250 Gift Certificate.
http://www.wrgsurveys.com/2003/osdntech03.php?site=8
Received on 2003-12-02