RE: curl getting different html

2025-01-10 Thread Dick Brooks via curl-users
I don't know if this will help, but I had to set the curl option FOLLOWLOCATION in order to retrieve binary data from GitHub. This ensure that redirects are followed. Thanks, Dick Brooks Active Member of the CISA Critical Manufacturing Sector, Sector Coordinating Council - A Public-Private P

Re: curl getting different html

2025-01-10 Thread toby via curl-users
Thanks Hans for the code and your help i don't get the gz'ed page with curl at all (no matter what i send - headers/no header -L/no -L etc) - just the 'needs javascript and cookies' page' a line from 'band of brothers' comes to mind here https://www.youtube.com/watch?v=cQE1YIJDNWo substitue 'clo

Re: curl getting different html

2025-01-10 Thread Hans Henrik Bergan via curl-users
>can curl give me a gzipped page if i use the correct header? (hint hint - can >you tell me what to use for curl) if you want to witness the compressed version first-hand, run curl --header 'Accept-Encoding: gzip, deflate, br, zstd' -o - https://www.podchaser.com/podcasts/crypto-corner-bitcoin-a

Re: curl getting different html

2025-01-10 Thread toby via curl-users
On Wed, 8 Jan 2025 22:09:41 +0100 Hans Henrik Bergan via curl-users wrote: > >i don;t get the 'are you a human' page with any of them > > lucky you: cloudflare's automated systems deems it unnecessary to > challenge you. A variation of a "trustworthy IP range" and > "correct-looking headers" > a

Re: curl getting different html

2025-01-10 Thread toby via curl-users
On Wed, 8 Jan 2025 12:27:00 -0800 Dan Fandrich via curl-users wrote: > On Wed, Jan 08, 2025 at 10:59:15AM -0700, toby via curl-users wrote: > > the page is actually gzipped - dillo can gunzip but curl can't by itself > > (found out using w3m) > > curl will uncompress if given the --compressed opt