On 04/06/2010 11:33 AM, Daniel Stenberg wrote:
> On Mon, 5 Apr 2010, Ben Greear wrote:
>
>>> Easy: libcurl only gets one resource per request, so you'd call it
>>> three times to get three mails.
>>
>> Ahh, so you don't want to support something like this?
>>
>> imap://tester:passwd_at_mail.candelatech.com/INBOX/;uid=1:3
>
> I didn't say that.
>
> If that is a valid URL then I think it is worth doing an effort to make
> that URL supported. However, everything is a ballance and features come
> at a price. I'm not prepared to sacrifise everything just to do more
> protocol features.
>
> If the above URL can be supported without the source becoming a maze,
> then I'm all for it.

That URL example is straight out of the RFC for IMAP URLs.
This will currently be a pain for users to interact with since we need some
way to tell the users where one message stops and another starts. We'd
probably need new start-msg/stop-msg callbacks or something like that.

With my current logic, specifically calling the Transfer
is what makes multiple msgs work. I think that call to Transfer is a
small thing...even if we remove that, the bulk of the other
logic is going to remain relatively complex because we have to
be able to deal with anything from a partial to multiple + partial
messages in the pp->cache.

The ugliness is at least mostly confined to the imap.c code, aside
from the trivial patch to make the Transfer logic public.

If we explicitly do not support more than a single message, then
I think we can use the transfer logic as you wish it used. It
would make it less efficient to download lots of messages with
CURL, but maybe that's not a big deal.

>>> Considering more than one file then enters the "FTP wildcard" concept
>>> area, but it would still technically be made by doing multiple
>>> individual requests internally even if you can hide that as looking
>>> like one big request from the outside the way the FTP wildcard patch
>>> does.
>>
>> I don't think I'll be writing such logic, but I will remove the
>> current support in my patch for multiple email if you prefer that.
>
> The thing is, your current logic is twisted and isn't following the curl
> ways so we cannot have it that way. If you're not prepared to take a
> step back and redo the functionality in a way that is more curlish, then
> yes I believe removing the support for that feature is the only option.

I'm willing to try..I just don't see how to do it in any clean manner.

>> Ok, so after we either download the complete thing in pp->cache OR set
>> up the transfer with Curl_setup_transfer, we always call: state(conn,
>> IMAP_STOP) in the fetch_body response, and then some part of the
>> pingpong statemachine (or something else?) will cause the Transfer to
>> happen?
>
> pp->cache is only for keeping headers, and headers are only headers. I
> think we should strive to minimize abusing the header buffers to get
> data with.

When I started hacking on imap the pp->cache contained headers + data.
I don't see any way to avoid this while still using pingpong logic.

> Content that is downloaded or uploaded is meant to be done in the
> TRANSFER phase, and it is not supposed to be limited to fixed sizes like
> header lengths or similar.
>
> Downloads are started in the current code by the imap_state_fetch_resp()
> function. It calls Curl_setup_transfer() and then ends the state
> machine, and as that is part of the DO phase state-machine, the next
> logical step within libcurl after DO is: TRANSFER.

Ok, provided we limit to a single message per request, I think I
understand how to make this work now.

>>>> * SEARCH [25k of data]
>
>>> Since pingpong won't deal with that, I can only imagine that we need
>>> to consider the search results to be the transfer here, so DO would
>>> issue the search command and the TRANSFER would receive the results.
>>
>> We need to watch for the response result line (first line that doesn't
>> start with "* SEARCH"
>
> However hideous that is, I can't see how we can avoid it. But we already
> do a similar thing for POP3 which always scans for \r\n.\r\n to signal
> end of transfer, so it should be fairly easy to make a similar thing for
> IMAP SEARCH. See Curl_pop3_write().

I'll look at that pop3 code as soon as I get time to work on this some
more.

Thanks,
Ben

-- 
Ben Greear <greearb_at_candelatech.com>
Candela Technologies Inc  http://www.candelatech.com
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html