Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The option «--dump-header» does not work correctly. #10074

Closed
u20221022 opened this issue Dec 10, 2022 · 9 comments
Closed

The option «--dump-header» does not work correctly. #10074

u20221022 opened this issue Dec 10, 2022 · 9 comments
Labels
cmdline tool

Comments

@u20221022
Copy link

u20221022 commented Dec 10, 2022 •
edited

I did this

Start PowerShell 7.3.0 under the MS Windows and execute commands:
PS >

.\curl.exe --progress-bar `
    --remote-name `
    --dump-header "Header32.txt" `
    "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win32-mingw.zip" `
--next `
    --remote-name `
    --dump-header "Header64.txt" `
    "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win64-mingw.zip" `
--next `
    --remote-name `
    --dump-header "Header96.txt" `
    "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win64a-mingw.zip"

32, 64, 96 | %{(Get-Content -LiteralPath Header$_.txt).Count}

################################################################################ 100,0%
################################################################################ 100,0%
################################################################################ 100,0%
22
22
22

Repeat all above:
22
44
44

Repeat again:
22
66
66

And so on. All but the first header dump has incorrect content.

I expected the following

All header dumps contain an equal number of lines (22). Always.

curl/libcurl version

Build: 7.86.0_2
[curl -V output]
curl 7.86.0 (x86_64-w64-mingw32) libcurl/7.86.0 OpenSSL/3.0.7 (Schannel) zlib/1.2.13 brotli/1.0.9 zstd/1.5.2 WinIDN libssh2/1.10.0 nghttp2/1.51.0 ngtcp2/0.11.0 nghttp3/0.7.1 libgsasl/2.2.0
Release-Date: 2022-10-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe TLS-SRP UnixSockets zstd

operating system

MS Windows 10 last version with all updates
@gvanem
Copy link
Contributor

gvanem commented Dec 10, 2022

All header dumps contain an equal number of lines (22). Always.

I get 21 lines in all headers when using another shell. A Powershell issue?

@u20221022
Copy link
Author

u20221022 commented Dec 10, 2022 •
edited

Repeat without manual deleting of dumps. Read more carefully: repeat some times. Repeat. Is the PowerShell clearing the file content? PowerShell does start external program and send options. Nothing more.

Console of Cmd.exe:

curl.exe --progress-bar --remote-name --dump-header "Header32.txt" -y 20 -Y 1024 "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win32-mingw.zip" --next --remote-name --dump-header "Header64.txt" -y 20 -Y 1024 "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win64-mingw.zip" --next --remote-name --dump-header "Header96.txt" -y 20 -Y 1024 "https://curl.se/windows/dl-7.86.0_2/curl-7.86.0_2-win64a-mingw.zip"
Dir Header??.txt

10.12.2022 14:25 802 Header32.txt
10.12.2022 14:26 4 818 Header64.txt
10.12.2022 14:27 4 807 Header96.txt

@u20221022
Copy link
Author

I get 21 lines in all headers when using another shell. A Powershell issue?

Using another arithmetics. An arithmetics issue? Empty line as end of dump you can not calculate? 1+20+1=22. The cURL does report, that num_headers=20.

@bagder
Copy link
Member

bagder commented Dec 10, 2022 •
edited

This line (brought by #4762) looks wrong:

curl/src/tool_operate.c

Line 972 in 99f72c0

newfile = fopen(config->headerfile, per->prev == NULL?"wb":"ab");

It assumes that the same file is used repeatedly and thus appends in all but the first fopen() calls.

@jay
Copy link
Member

jay commented Dec 10, 2022

Yes this one is a bug. @emilengler

@emilengler
Copy link
Contributor

See #4753

emilengler added a commit to emilengler/curl that referenced this issue Dec 11, 2022
@emilengler
tool: overwrite instead of append when using -D
98a1a48 
This commit fixes a bug, in which the tool appends to a text file
instead of overwriting it, when a previous transfer took place.

See curl#4753
See curl#4762
Fixes curl#10074
Closes #XXXX
@emilengler emilengler mentioned this issue Dec 11, 2022
Closed
emilengler added a commit to emilengler/curl that referenced this issue Dec 11, 2022
@emilengler
tool: overwrite instead of append when using -D
21fabf4 
This commit fixes a bug, in which the tool appends to a text file
instead of overwriting it, when a previous transfer took place.

See curl#4753
See curl#4762
Fixes curl#10074
Closes curl#10079
@emilengler
Copy link
Contributor

I looked at the regarding PRs and issues from 3 years ago.

#4753 (comment)
#10074 (comment)

@jay: If I follow your logic correctly, I see a contradiction in these two comments. Could you elaborate?

@u20221022
Copy link
Author

If you add, then to everyone. Why not add to the first?

@bagder
Copy link
Member

bagder commented Dec 11, 2022

@emilengler there's a difference:

This should put the headers from both transfers into the same file:

curl -D file $URL1 $URL2

This should (probably) not append the headers to the files if they exist before the command line is invoked:

curl -D file1 $URL1 --next -D file2 $URL2

emilengler added a commit to emilengler/curl that referenced this issue Dec 11, 2022
@emilengler
tool: determine the correct fopen option for -D
dee2883 
This commit fixes a bug in the dump-header feature regarding the
determination of the second fopen(3) option.

See curl#4753
See curl#4762
Fixes curl#10074
Closes curl#10079
emilengler added a commit to emilengler/curl that referenced this issue Dec 11, 2022
@emilengler
tool: determine the correct fopen option for -D
50e7d4d 
This commit fixes a bug in the dump-header feature regarding the
determination of the second fopen(3) option.

See curl#4753
See curl#4762
Fixes curl#10074
Closes curl#10079
@bagder bagder closed this as completed in 8b1e5df Dec 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cmdline tool
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tool: determine the correct fopen option for -D emilengler/curl
5 participants
@bagder @gvanem @jay @emilengler @u20221022