cURL / Mailing Lists / curl-users / Single Mail

curl-users

RE: Commonalities between cURL instances

From: David Burley <BurleyD_at_rct.dyndns.org>
Date: Thu, 7 Mar 2002 11:51:39 -0500

The issue:

I am connecting to a web site that detects connections and will block you if
you try to connect more then once. I have found that if I run cURL in a loop
using a shell script it runs forever without them caring, as you only have
one connection.

If you start multiple instances, they detect you and display that you have
connected more then once. However, if you start up another browser (Internet
Explorer, as I am running it in Cygwin), they only detect the instances of
cURL as having more than one on their site and treat the instance of
Internet Explorer independently of cURL. If I was to open multiple instances
of IE they detect it and display that you are connected more then once.

I know from looking at what I am posting to them "curl -v" that the post
data, the unique values anyways, are unique for each and every instance. The
cookie files are also being stored independently of one another and are
being deleted after each round of requests.

Thus, I come to the conclusion that something must be common between the
instances of cURL. Else, I would think that they would detect multiple
instances with one copy of cURL and one copy of Internet Explorer open and
they do not. Could the commonalities be coming from the SSL implementation
(in other words... not cURL or libcURL's problem)?

Is there an easy hack to force it to create and kill connections with each
request. I'll try to hack on that tomorrow and see if I can't code it... I
haven't dealt much with this kind of programming but I can probably figure
it out.. maybe you can tell me if it is possible and then I'll try to
implement it for myself. Would be good practice either way.

Thanks for the help,

David

-----Original Message-----
From: Daniel Stenberg [mailto:daniel_at_haxx.se]
Sent: Wednesday, March 06, 2002 6:15 PM
To: David Burley
Cc: 'curl_at_contactor.se'
Subject: Re: Commonalities between cURL instances

On Wed, 6 Mar 2002, David Burley wrote:

> I am facing an issue with cURL that I am attributing to "sharing" of
> data/connections between cURL instances.

You should probably present the problem to us as well, as it might help us
understand what you're after or possibly recognize something we might know
how to fix.

> One thing that I think I found in the cURL code, which I am not to
familiar
> with, is that it seems cURL caches the SSL connection information for
> future use.

The SSL session ID is cached to reduce re-connection time (the SSL
handshake), should a subsequent request be made to the same host after the
connection has been closed.

But libcurl also caches connections, which makes it attempt to keep
connections alive to be able to re-use it in a subsequent request. If a
connection is re-used, the SSL session ID isn't needed as it doesn't have to
redo the SSL handshake.

> So, if I ran to copies of cURL both to the same site at about the same
> time... I believe they would use the same SSL connection.

No. The sharing is only made within a single curl easy handle, and it can
only be used sequentially by a single program.

> Presuming I explicitly gave each instance their own cookie file and post
> data, what else might they share?

Two separate curl processes don't share anything.

> Do they even share the SSL connection that I think they are?

No.

> If they are sharing data, how do I get it to stop?

Two separate curl processes will never share data.

> If I use libcURL will I get the same results as with cURL itself?

If you use libcurl the right way and the same way the curl tool does, you
will get the same results indeed.

Using libcurl is often slightly more error-prone than using the curl tool,
as
people tend to do more mistakes when they program than when they simple use
a
command line tool.

I hope this helps. If you have specific libcurl issues, please take them on
the libcurl mailing list.

-- 
    Daniel Stenberg -- curl groks URLs -- http://curl.haxx.se/
Received on 2002-03-07