> From: curl-library-bounces_at_cool.haxx.se [curl-library-bounces_at_cool.haxx.se] on
> behalf of Daniel Stenberg [daniel_at_haxx.se]
>
> On Tue, 24 Jul 2012, George Rizkalla wrote:
>
> > We are currently looking at contributing to libcurl's pipelining
> > implementation, and we were hoping to get your feedback on some areas we'd
> > like to help with.
>
> Awesome. You are most welcome here and I'm sure I'm not the only one who will
> appreciate this!
>
> Let me give you a word of advice already now: work on making sure we can
> exercise most of the pipelining code and features in the test suite.
> Pipelining is tricky already by default, if we can't verify functionality and
> track down problems with the test suite things often tend to fade and break
> while we're not paying attention. I believe the existing pipelining support
> suffers a bit from this.

I'm working on unit tests for the auth callbacks right now, and learning how the test suite works as I go, so I should be in good shape to make use of it for pipelining when the time comes.

> This sounds fine too. I'm a bit doubtful about the ncesssity for the
> CURLMOPT_*_PENALTY_SIZE options though. They seem like options that simply
> nobody would touch if we would claim the defaults are somewhat sensible. Or do
> you see any use-case in your world where you will actually change these
> according to some scheme or acquired knowledge in the application?

We're going to want to do a lot of performance profiling to find the optimal default (probably refining it between releases), and it would be convenient if we could just update one setting in the app with the results rather than use a custom build of curl. I suspect this setting might be very dependent on use case, but I won't really know until we do performance profiling. Conceivably we might find one optimal value and then update curl to use it as the default.

Again this is something that could be expanded later if it turns out to be necessary, since adding an option won't affect existing apps that only use the default. But if you're worried about keeping the number of variables down for the unit tests, hard-coding this won't really help because we'll want the flexibility to change the compiled-in default as we do performance testing, and we don't want that to invalidate the unit tests. So we need to make sure the unit tests don't end up with too many assumptions/dependencies on this value, whether it's officially variable with an option or not.

> > -Pipeline-able Sockets
>
> > In addition to the obvious checks for the use of an HTTP/1.1 server,
> > befo
> > assuming that pipelining is possible on a particular socket, libcurl could
> > check for older/incompatible IIS servers before deciding to pipeline.
>
> Ouch. How would that be done? Scanning for certain magic keywords in response
> Server: headers?

That's how Mozilla does it: checks for the following list.

   252 "Microsoft-IIS/4.",
   253 "Microsoft-IIS/5.",
   254 "Netscape-Enterprise/3.",

> > If the first response to a request on a socket is marked as HTTP/1.0, or
> > an
> > older IIS server version is used, or the site is black-listed (see below),
> > the socket should be characterized as CAN_PIPELINE = false.
>
> This makes the blacklist basically always just grow. Will there be any means
> of purging old entries or don't you see this as a problem?

Well, it means that socket can't be used for pipelining for the rest of its life, but it's not like the server characteristics are going to change while we're connected to it, so that's unavoidable. When another socket is connected to the same server, we'd check the blacklist again. Setting CAN_PIPELINE = false due to HTTP/1.0 or whatever would not update the blacklist - it's just a non-persistent, per-connection setting.

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2012-07-25