Hi.

Testing out a curl cmd in a test py script (centos)

The basic url
http://www.bkstr.com/webapp/wcs/stores/servlet/booklookServlet?bookstore_id-1=047&term_id-1=Fall+2017&crn-1=12493

works with no issue in the browser.

As far as I can tell, the server is compressing/encoding the returned data.

The test

curl -vvv -sS -A "Mozilla/5.0 (Windows; U; Windows NT 6.1; )
AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3
Safari/533.19.4" --compressed -L
"http://www.bkstr.com/webapp/wcs/stores/servlet/booklookServlet?bookstore_id-1=047&term_id-1=Fall+2017&crn-1=12493"

works from the cmdline.

However, running the test curl from a test py generates a malformed curl error!!

test py script::
  user_agent ="Mozilla/5.0 (Windows; U; Windows NT 6.1; )
AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.3
Safari/533.19.4"

  url1="http://www.bkstr.com/webapp/wcs/stores/servlet/booklookServlet?bookstore_id-1=047&term_id-1=Fall+2017&crn-1=12493"

  cmd='curl -sS '
  cmd=cmd+'-A "'+user_agent+'"'
  cmd=cmd+' --compressed '
  cmd=cmd+' -L "'+url1+'" '
  proc=subprocess.Popen(cmd, shell=True,stdout=subprocess.PIPE)
  s=proc.communicate()[0].strip()

Is there something else in terms of headers I can try instead of the
"compressed"??

Is there something that someone can see that I can try to deal with
the malformed issue!

Without the --compressed, the returned content is "garbled". I've also
tried to use '-H Accept-Encoding: deflate,gzip;' which results in
garbled content as well.

Comments/Thoughts?...

Thanks
-----------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-users
Etiquette: https://curl.haxx.se/mail/etiquette.html
Received on 2017-08-22