Hi,
I am facing a weird problem here. I have an application that uses two
components to handle web services. One is gsoap and other is component
built using libcurl.
1. If I provide a URL to Curl component. It does not connect to URL.
2. If I access the network using gsoap component first, it works and then
If i connect using libcurl component to any other url it connects fine.

The server to which the curl connects is local server only.
http_proxy is set to null.
Work environment is windows.

I have following code in libcurl component functions
1. void Connect(const std::string& url)
{
assert(0 != sessionData.get());
sessionData->Clear();
sessionData->url = url;
//trim the input URL. Leading and trailing whitespaces can cause problem
boost::algorithm::trim(sessionData->url);
std::string protocol = sessionData->url.substr(0,
sessionData->url.find(PROTOCOL_DELIMITER));

//if protocol is empty or there is no "://" in the url then throw exception
if (protocol.empty() || (protocol == sessionData->url))
{
throw UnsupportedProtocolException("The Protocol could not be retrieved");
}

if (!boost::algorithm::iequals(protocol, HTTP_STRING))
{
throw UnsupportedProtocolException("The Protocol not Supported.");
}
//Clear up the data associated with the old connection
responseHeader.Clear();
//We remove the easy handle from the multi handle when we want to configure
on the easy handle.
CURLMcode multiStatus =
curl_multi_remove_handle(sessionData->curlMultiHandle,
sessionData->curlEasyHandle);
if (CURLM_OK != multiStatus)
{
throw NetworkException("Could not remove easy handle from multi stack.");
}
sessionData->curlEasyStatusCode =
curl_easy_setopt(sessionData->curlEasyHandle,
CURLOPT_URL, sessionData->url.c_str());
if (CURLE_OK != sessionData->curlEasyStatusCode)
{
throw NetworkException("Could not set the URL in connect.");
}
//The previous URL might have set the Seek() offset. This will create a
//Partial request in Connect. To avoid this, we must resume transfer from
beginning.
int offset = 0;
sessionData->curlEasyStatusCode =
curl_easy_setopt(sessionData->curlEasyHandle,
CURLOPT_RESUME_FROM,
offset);
if (CURLE_OK != sessionData->curlEasyStatusCode)
{
throw NetworkException("Could not resume from beginning in CONNECT.");
}

multiStatus = curl_multi_add_handle(sessionData->curlMultiHandle,
sessionData->curlEasyHandle);
if (CURLM_OK != multiStatus)
{
throw NetworkException("Could not add easy handle from multi stack.");
}

//Keep calling ReceiveStreamContent() in a loop till we get some data from
the server.
//Since initial calls will get the Http headers and once the data callback
is called
//by CURL, internalBufferSize changes, which means we have received some
data. In this
//process, if anywhere the transfer aborts/end of media occurs, we consider
it as failure to
//connect. We do not expect connection to close in CONNECT state.
//Also we have a select time out of 1 sec so that we dont get blocked when
having some
//interrupt
int attempts = 0;
while ( (sessionData->internalBufferSize == 0) && (attempts <
MAX_CONN_ATTEMPTS) ) // MAX_CONN_ATTEMPTS = 60
{
ReceiveStreamContent(CONNECTIONSTATE_CONNECT);
++attempts;
{
boost::mutex::scoped_lock interruptThreadLock(guardInterrupt);
if (isInterrupted)
{
sessionData->endOfMedia = true;
sessionData->internalBufferUsed = 0;
isInterrupted = false;
throw ConnectionInterruptedException("Interrupted");
}
}
}
//This will fill the ResponseHeader structure.
ParseHeader();
if (attempts == MAX_CONN_ATTEMPTS)
{
sessionData->endOfMedia = true;
sessionData->internalBufferUsed = 0;
throw NetworkException("Could not connect - Exceeded max retry
attempts !!");
}

//If connection fails due to any reason then throw exception
if (sessionData->endOfMedia)
{
throw NetworkException("Connection closed in Connect()");
}
} //end of Connect()

2. void ReceiveStreamContent(ConnectionState state) const
{
fd_set read_fds;
fd_set write_fds;
fd_set ex_fds;
int runningHandles = 0;
struct timeval timeOut;
timeOut.tv_sec = 0;
timeOut.tv_usec = 0;

//if ReceiveStreamContent is called from Connect() then the timeout is more
than
//the time out for ReceiveStreamContent called from GetData()
if (CONNECTIONSTATE_CONNECT == state)
{
timeOut.tv_sec = SELECT_CONNECT_TIMEOUT; //60 sec
}
else if (CONNECTIONSTATE_DATA == state)
{
timeOut.tv_usec = DATA_TIMEOUT; //90 ms
}

if (sessionData->curlMultiStatusCode == CURLM_OK)
{
FD_ZERO(&read_fds);
FD_ZERO(&write_fds);
FD_ZERO(&ex_fds);
int max_fd;
sessionData->curlMultiStatusCode =
curl_multi_fdset(sessionData->curlMultiHandle,
&read_fds, &write_fds, &ex_fds, &max_fd);
if (CURLM_OK != sessionData->curlMultiStatusCode)
{
throw NetworkException("could not retrieve multi fd_set.");
}

if (-1 < max_fd)
{
// If we don't set timeout for select (select hangs indefinetely),
// we wonx`t be able to knew that we aren't receiving data though we
connected to a url.
int ret = select(max_fd+1, &read_fds, &write_fds, &ex_fds, &timeOut);
switch(ret)
{
case 0 : //If timeout has occured in CONNECT state then we are not able to
connect to URL
break;
case -1: //Some error has occured on the socket.
sessionData->endOfMedia = true;
sessionData->internalBufferUsed = 0;
throw NetworkException("select() Failed");
break;
}
}
}
sessionData->curlMultiStatusCode =
curl_multi_perform(sessionData->curlMultiHandle,
&runningHandles);

// If no handles are running, it implies that connection with the server
has been lost.
// If the transfer is complete/ or connection is closed then runnnig
handles become 0
if (0 == runningHandles)
{
sessionData->endOfMedia = true;
sessionData->internalBufferUsed = 0;
}
}

Description about the code :
=========================
1. sessionData->internalBufferSize gets modified in Data callback of
libcurls easy handle.
2. attempts < MAX_CONN_ATTEMPTS is required to exit the loop incase we have
tried connecting for 60 seconds. (1 sec timeout 60 times).

Now in case of case 1. 1. If I provide a URL to Curl component. It does not
connect to URL.
There are other functions in which the easy and multi handles are created
and properly configured. The problem I face is select call returns with
valur 1. That means some data has been received/sent or some exception has
occured. Now I have enabled verbose using setopt. The output of verbose is

*Expiry cleared
*Expiry cleared

And my value of attempts becomes 60. That mean I am running over the loop
60 times without receiving any data. This does not happen if I access the
URL with some other application or with other gsoap component first and
then using libcurl. The URL is local url only.

The problem is 60 times select call is not enough for libcurl to get even
the headers from server. But for same URL it is able to get headers and
data much before 60 attempts if some other component has accessed the
network.

Please suggest what could be wrong in my code?

Thanks and Regards
Shivanand

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-12-08