Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws-sigv4 generates extra X-Amz-Date header (possibly different) #11738

Closed
apparentorder opened this issue Aug 25, 2023 · 5 comments
Closed

aws-sigv4 generates extra X-Amz-Date header (possibly different) #11738

apparentorder opened this issue Aug 25, 2023 · 5 comments
Labels
authentication

Comments

@apparentorder
Copy link

I did this

curl -v --aws-sigv4 aws:amz:foo --header 'X-Amz-Date: 20230813T061344Z' --user foo:bar https://httpbin.org/post

I expected the following

The provided X-Amz-Date is used for signature calculation and is emitted once.

But actually it's emitted twice:

> GET /post HTTP/2
> Host: httpbin.org
> authorization: AWS4-HMAC-SHA256 Credential=foo/20230825/foo/httpbin/aws4_request, SignedHeaders=host;x-amz-date, Signature=bcecfc18afb2e19fe031bfd2bbaa68faffb3ff3cc3e990b014217422f422860a
> x-amz-date: 20230825T202910Z
> user-agent: curl/7.81.0
> accept: */*
> x-amz-date: 20230813T061344Z

On some curl versions (tested: 7.81.0), the first X-Amz-Date is actually generated with a different date (the current date), and that current date is used for signature calculation instead of the X-Amz-Date given on the command line.

Other versions (tested: 8.1.2) emit the given X-Amz-Date twice as well, but both with the given value; in that case, the signature is calculated as expected.

As this is a rare / weird use-case, maybe it should just be added to the list of known bugs (?)

curl/libcurl version

curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.14

curl 8.1.2 (x86_64-apple-darwin22.0) libcurl/8.1.2 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.51.0

operating system

curl 8.1.2 on macOS 13.5, or
Darwin xxx 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:21:56 PDT 2023; root:xnu-8796.141.3~6/RELEASE_X86_64 x86_64

curl 7.81.0 on Ubuntu 22.04.2, or
Linux ip-10-0-0-148 5.19.0-1025-aws #26~22.04.1-Ubuntu SMP Mon Apr 24 01:58:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
@jay
Copy link
Member

jay commented Aug 25, 2023

Is there a reason to support user supplied X-Amz-Date?

/cc @outscale-mgo

@apparentorder
Copy link
Author

apparentorder commented Aug 26, 2023 •
edited

Reasons I can think of are

  • testing, i.e. achieve a stable signature value to check against – this is what I was trying to do (in a libcurl client), and this worked on 8.x and I was kinda surprised when 7.x produced different results
  • security experiments to see how a server behaves with offset and malformatted dates

@bagder
Copy link
Member

bagder commented Aug 27, 2023

Is there a reason to support user supplied X-Amz-Date?

curl supports custom headers of any kind, outputting two like this is at least a bug.

Not using the user provided date header seems like an oversight, as exactly like @apparentorder did, I would have expected curl to work with it.

@outscale-mgo
Copy link
Contributor

It's clearly a bug, I was pretty sure I was handling the case in the code, but I guess I didn't test this enough, sorry for the bug, I'll look at it asap.

This is what AWS "standard" say about date:

X-Amz-Date

    The date that is used to create the signature. The format must be ISO 8601 basic format (YYYYMMDD'T'HHMMSS'Z'). For example, the following date time is a valid X-Amz-Date value: 20120325T120000Z.

    Condition: X-Amz-Date is optional for all requests; it can be used to override the date used for signing requests. If the Date header is specified in the ISO 8601 basic format, X-Amz-Date is not required. When X-Amz-Date is used, it always overrides the value of the Date header. For more information, see [Elements of an AWS API request signature](https://docs.aws.amazon.com/IAM/latest/UserGuide/signing-elements.html) in the IAM User Guide.

and then:

Specifies the date and time of the request. Including the date and time in a request helps prevent third parties from intercepting your request and resubmitting it later. The date that you specify in the credential scope must match the date of your request.

The time stamp must be in UTC and use the following ISO 8601 format: YYYYMMDDTHHMMSSZ. For example, 20220830T123600Z. Do not include milliseconds in the time stamp.

You can use a date or an x-amz-date header, or include x-amz-date as a query parameter. If we can't find an x-amz-date header, then we look for a date header.

outscale-mgo added a commit to outscale-mgo/curl that referenced this issue Aug 28, 2023
@outscale-mgo
lib: fix aws-sigv4 having date header twice on some cases
592e2b9 
fix curl#11738

When the user was providing the header X-XXX-Date, the header
was re-added during signature computation,
and we had it twice in the request.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
@outscale-mgo outscale-mgo mentioned this issue Aug 28, 2023
Closed
outscale-mgo added a commit to outscale-mgo/curl that referenced this issue Aug 28, 2023
@outscale-mgo
lib: fix aws-sigv4 having date header twice on some cases
af67c6a 
fix curl#11738

When the user was providing the header X-XXX-Date, the header
was re-added during signature computation,
and we had it twice in the request.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
outscale-mgo added a commit to outscale-mgo/curl that referenced this issue Aug 29, 2023
@outscale-mgo
lib: fix aws-sigv4 having date header twice on some cases
a66ef8d 
fix curl#11738

When the user was providing the header X-XXX-Date, the header
was re-added during signature computation,
and we had it twice in the request.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
outscale-mgo added a commit to outscale-mgo/curl that referenced this issue Aug 29, 2023
@outscale-mgo
lib: fix aws-sigv4 having date header twice on some cases
a364446 
fix curl#11738

When the user was providing the header X-XXX-Date, the header
was re-added during signature computation,
and we had it twice in the request.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
outscale-mgo added a commit to outscale-mgo/curl that referenced this issue Aug 29, 2023
@outscale-mgo
lib: fix aws-sigv4 having date header twice on some cases
063f145 
fix curl#11738

When the user was providing the header X-XXX-Date, the header
was re-added during signature computation,
and we had it twice in the request.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
@jay jay closed this as completed in b137634 Aug 30, 2023
@jay
Copy link
Member

jay commented Aug 30, 2023

Thanks

ptitSeb pushed a commit to wasix-org/curl that referenced this issue Sep 25, 2023
@outscale-mgo @ptitSeb
lib: fix aws-sigv4 having date header twice in some cases
1301d66 
When the user was providing the header X-XXX-Date, the header was
re-added during signature computation, and we had it twice in the
request.

Reported-by: apparentorder@users.noreply.github.com

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>

Fixes: curl#11738
Closes: curl#11754
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
authentication
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix aws sigv4 date twice outscale-mgo/curl
4 participants
@bagder @jay @apparentorder @outscale-mgo