curl is at curl.se / Bugs / #1456 Curl SMTP client CRLF and dot stuffing doesn't work

Steve Holme - 2014-11-25

Hi,

Thank you for your report.

I think you have found a bug in the documentation!

Looking at the code in transfer.c that performs the LF to CRLF conversion it is contained within the else block of:

if(conn->handler->protocol & PROTO_FAMILY_SMTP) { result = Curl_smtp_escape_eob(conn, nread); if(result) return result; } else

The dot stuffing happens in Curl_smtp_escape_eob() which checks for CRLF. replacing it with CRLF..

I will need some clarification from other members of the curl development team, as to which protocols are supposed to support the conversion but I don't believe SMTP does given the above and my experience with curl in this area.

If you remove your usage of CURLOPT_CRLF and transfer a file that contains CRLF you should find that dot stuffing works.

Kind Regards

Steve
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- bra - 2014-11-26
  
  Hi,
  
  First of all, thanks for taking care of this.
  I think CRLF conversion is a must for SMTP. For example, all of (at least what I know) LMTP/IMAP/POP servers which use the maildir format store the messages in unix (LF) format. Sending these out with SMTP requires conversion in my application, which (converting to and from wire or network format) should be the task of the network library -here the SMTP client, curl.
  If the infrastructure is already there, I can't see a good reason not to do this. :)
  
  Thank you very much.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-25

In the meantime I have updated the documentation to reflect what the code does.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-26

Yet another update...

This was annoying me a little so I have spent the last 40 minutes working on a fix!

I will perform some final testing tomorrow, review and push if all is good.

Kind Regards

Steve

Last edit: Steve Holme 2014-11-26

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-26

assigned_to: Steve Holme
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-26

I have a pushed my implementation of this as commit 4bd860a001

It is a little different from last nights quick fix as I found a memory leak in my code and fixed a few other minor issues in the next commit.

I have also updated the documentation accordingly.

I have a new test case for this as well but will push that in the morning after some shut eye ;-)

If you build libcurl yourself you can apply the git format patch by following the link above (and appending .patch to the url to download the associated .patch file). Alternatively the next nightly archive will contain this.

Last edit: Steve Holme 2014-11-26

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-26

I will close the issue if you are able to confirm it works for you / once I have pushed my test case and verified the result.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-26

status: open --> open-confirmed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-27

I pushed the additional test case (test941) this morning as commit c5399c827d which looking at the next ran autobuild verifies the fix.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-27

Thanks again for your report.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-27

status: open-confirmed --> closed-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- bra - 2014-11-28
  
  Hi,
  
  Thanks for the quick fix for the CRLF issue, that works now.
  BTW, the example program still can't send the e-mail, because the dot stuffing doesn't work.
  
  Here's the wireshark output from the session:
  DATA
  354 End data with <CR><LF>.<CR><LF>
  Date: Mon, 29 Nov 2010 21:54:29 +1100
  To: recipient@example.org
  From: sender@example.org(Example User)
  Message-ID: dcd7cb36-11db-487a-9f3a-e652a9458efd@rfcpedant.example.org
  Subject: SMTP example message
  
  .The body of the message starts here.
  
  .
  It could be a lot of lines, could be MIME encoded, whatever.
  Check RFC5322.
  .
  250 2.0.0 Ok: queued as B82944109EF
  502 5.5.2 Error: command not recognized
  502 5.5.2 Error: command not recognized
  502 5.5.2 Error: command not recognized
  QUIT
  
  Could you please fix that too?
  
  Thanks,
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
  - Steve Holme - 2014-11-28
    
    I'm a little confused as the dot stuffing has worked for a long time (See test 901) and I haven't changed that code (apart from the different internal buffer it now looks at).
    
    The reason it wasn't working for you previously is that the dot stuffing looks for CRLF. and not LF. as was happening before I added support for --crlf to perform the conversion. The dot stuffing now happens after that conversion so intenally your mail will be CRLF. and the dot stuffing should work ;-)
    
    I don't run the test suite personally as I run under Windows but my own test files are working okay ;-) I have dot stuffing with CRLF and dot stuffing with LF as well as plain emails without any dot stuffing.
    
    However, I will investigate this further tonight to see if there are any other issues in the dot stuffing code that could prevent the example working for you.
    
    As such I will re-open this issue.
    
    Last edit: Steve Holme 2014-11-28
    
    If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
    - Steve Holme - 2014-11-29
      
      Just a quick update...
      
      I don't build against libcurl for use in an application at home but instead build curl so my testing tonight has been a little different from yours.
      
      I used the following command line:
      
      curl -v -ssl -k smtp://smtp.example.com -T C:\curl\mail-lf.txt --crlf --mail-rcpt recipient@example.org --mail-from sender@example.org
      
      Whilst debugging transfer.c and smtp.c I can see my strings are as follows (I have removed the headers and body separator for clarity):
      
      As read from my file:
      
      ".The body of the message starts here.\n.\nIt could be a lot of lines, could be MIME encoded, whatever.\nCheck RFC5322.\n.\n"
      
      After LF to CRLF conversion:
      
      ".The body of the message starts here.\r\n.\r\nIt could be a lot of lines, could be MIME encoded, whatever.\r\nCheck RFC5322.\r\n.\r\n"
      
      After calling Curl_smtp_escape_eob() to do the dot stuffing:
      
      "..The body of the message starts here.\r\n..\r\nIt could be a lot of lines, could be MIME encoded, whatever.\r\nCheck RFC5322.\r\n..\r\n"
      
      Do you have the same issues if you use curl?
      
      Kind Regards
      
      Steve
      
      Last edit: Steve Holme 2014-11-29
      
      If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-11-28

status: closed-fixed --> open-confirmed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-12-12

After some discussion on the libcurl mailing list, we think we have found the issue, which would only affect libcurl based applications that a) used a read function and b) where the EOL character(s) were at the end of each buffer.

As such I have pushed commit f0ecdd04d3

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-12-12

status: open-confirmed --> closed-fixed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steve Holme - 2014-12-12

The fix doesn't appear to broken the existing dot stuffing tests - although it would be good if we could add a libcurl test to also verify this.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Curl SMTP client CRLF and dot stuffing doesn't work

curl is at curl.haxx.se

Searches

Help

#1456 Curl SMTP client CRLF and dot stuffing doesn't work

Discussion