Buy commercial curl support. We
help you work out your issues, debug your libcurl applications, use the API,
port to new platforms, add new features and more. With a team lead by the
curl founder Daniel himself.
Re: Feature request: data stream upload after server response
- Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ] [ by messages with attachments ]
From: fungs via curl-users <curl-users_at_lists.haxx.se>
Date: Mon, 10 Jun 2024 11:26:15 +0000
Hi,
I was waiting for other answers, but let me explain a little more:
- I proposed a feature to make curl file upload work with existing HTTP endpoints (not mine) on existing infrastructure (cloud providers with managed reverse proxies and edge routers).
- Nowadays, it is common that requests are delegated, for instance to an authentication proxy, before data is uploaded to an endpoint. This is just how modern cloud and microservices work.
- With those setups, it does not make sense to start a data upload until an end to end connection has been authenticated and established, because the data to be passed among systems and needs to be cached in the process of routing. Such intermediate systems need to handle many requests and have storage and time limits.
- The solution I see and which seems to work for me is to wait for the response before starting to upload or stream data, which ensure that all the infrastructure negotiation has succeeded.
- The solution would work well with the async upload option in curl, as far as I can tell.
I understand, that from a traditional perspective on the HTTP protocol and its
operations, the problem seems to be with the infrastructure providers
not handling the requests properly. However, HTTP today is used in many
complex situations. A global edge router simply cannot know how a
request is going to be used in the following layers. Therefore, the
workaround presented to me seems like a useful (non-standard) addition
to curl. The alternative would be to provide alternative HTTP upload
clients for the operations (like the shell workaround I posted before).
The is what I would like to avoid.
Happy to discuss it!
Best
Johannes
Date: Mon, 10 Jun 2024 11:26:15 +0000
Hi,
I was waiting for other answers, but let me explain a little more:
- I proposed a feature to make curl file upload work with existing HTTP endpoints (not mine) on existing infrastructure (cloud providers with managed reverse proxies and edge routers).
- Nowadays, it is common that requests are delegated, for instance to an authentication proxy, before data is uploaded to an endpoint. This is just how modern cloud and microservices work.
- With those setups, it does not make sense to start a data upload until an end to end connection has been authenticated and established, because the data to be passed among systems and needs to be cached in the process of routing. Such intermediate systems need to handle many requests and have storage and time limits.
- The solution I see and which seems to work for me is to wait for the response before starting to upload or stream data, which ensure that all the infrastructure negotiation has succeeded.
- The solution would work well with the async upload option in curl, as far as I can tell.
I understand, that from a traditional perspective on the HTTP protocol and its
operations, the problem seems to be with the infrastructure providers
not handling the requests properly. However, HTTP today is used in many
complex situations. A global edge router simply cannot know how a
request is going to be used in the following layers. Therefore, the
workaround presented to me seems like a useful (non-standard) addition
to curl. The alternative would be to provide alternative HTTP upload
clients for the operations (like the shell workaround I posted before).
The is what I would like to avoid.
Happy to discuss it!
Best
Johannes
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-users Etiquette: https://curl.se/mail/etiquette.htmlReceived on 2024-06-10