We see this issue on scrapping data from movie providers (like The Movie Database).

I don't think that would be any big data, for example that could be a 'file' we would try to get:

http://api.tmdb.org/3/search/movie?api_key=57983e31fb435df4df77afb854740ea9&query=Batman%20The%20Dark%20Knight%20Returns%2c%20Part%201&year=2012&language=en

Because of the amount of data there, I am not sure if slow/fast transfer would matter.

Log:

16:24:16 T:140013845079808 DEBUG: CurlFile::Open(0x7f574c8eabc0) http://api.tmdb.org/3/search/movie?api_key=57983e31fb435df4df77afb854740ea9&query=Batman%20The%20Dark%20Knight%20Returns%2c%20Part%201&year=2012&language=en
<stuff...>
16:24:26 T:140013845079808 ERROR: CCurlFile::FillBuffer - Failed: Timeout was reached(28)
16:24:26 T:140013845079808 ERROR: CCurlFile::CReadState::Connect, didn't get any data from stream.

but that data shouldn't take 10 seconds to load, as you can see loading it in a browser is quick.

I'll leave the rest to fristch as I probably will start to spew nonsense after that.

Thanks,
John

---
** [bugs:#1426]  curl v7.38.0 does not work with xbmc - cannot scrape or download**
**Status:** open
**Created:** Sat Sep 20, 2014 12:01 PM UTC by graysky
**Last Updated:** Mon Sep 22, 2014 08:28 AM UTC
**Owner:** Daniel Stenberg
Xbmc v13.2 does not work properly with curl v7.38.0. I cannot scrape anything now nor download anything through xbmc. The log indicates timing out to curl. I rebuilt xbmc against this new version of curl, but still, no downloads are possible.  Downgrading to v7.37.1 of curl solves the problem, so it seems to be related to this new release.
I have verified that I can download files form the shell with the current version of curl.  The xbmc dev group closed my ticket against xbmc with the comment, "report to curl."
Here are the lines from xbmc's log file showing the timeout:
ERROR:CCurlFile::FillBuffer-Failed:Timeoutwasreached(28)
ERROR:CCurlFile::CReadState::Connect,didn'tgetanydatafromstream.
ERROR:Open-failedtoopensource<http://mirrors.xbmc.org/addons/gotham/addons.xml.md5>
ERROR:CCurlFile::FillBuffer-Failed:Timeoutwasreached(28)
ERROR:CCurlFile::CReadState::Connect,didn'tgetanydatafromstream.
ERROR:Open-failedtoopensource<http://mirrors.xbmc.org/addons/frodo/addons.xml.md5>
ERROR:CCurlFile::FillBuffer-Failed:Timeoutwasreached(28)
ERROR:CCurlFile::CReadState::Connect,didn'tgetanydatafromstream.
ERROR:Open-failedtoopensource<http://mirrors.xbmc.org/addons/gotham/addons.xml|Encoding=gzip>
ERROR:CCurlFile::FillBuffer-Failed:Timeoutwasreached(28)
ERROR:CCurlFile::CReadState::Connect,didn'tgetanydatafromstream.
ERROR:Open-failedtoopensource<http://mirrors.xbmc.org/addons/frodo/addons.xml|Encoding=gzip>
ERROR:RepositoryXBMC.orgAdd-onsreturnednoadd-ons,listingmayhavefailed
OS: Arch Linux x86_64
% curl -V
curl 7.38.0 (x86_64-unknown-linux-gnu) libcurl/7.38.0 OpenSSL/1.0.1i zlib/1.2.8 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp 
Features: AsynchDNS IPv6 Largefile GSS-API SPNEGO NTLM NTLM_WB SSL libz TLS-SRP 
---
Sent from sourceforge.net because curl-tracker@cool.haxx.se is subscribed to https://sourceforge.net/p/curl/bugs/
To unsubscribe from further messages, a project admin can change settings at https://sourceforge.net/p/curl/admin/bugs/options.  Or, if this is a mailing list, you can unsubscribe from the mailing list.