[nSLUG] wget & interrupted fetch

Mike Spencer mspencer at tallships.ca
Sat Sep 24 04:10:35 ADT 2016


If a file transfer using wget(1) is interrupted, it will restart the
download where it left off if the data is being saved to a file.  The
-c option can force this in a new session as well.

Is there a technical reason why this doesn't work with
--no-check-certificate and an HTTPS URL?

In such a case, wget will automatically restart an interrupted
transfer but it will start with byte 0 of the data.  It does *not*
delete the local file resulting from the "-O filename" option each
time but may restart a dozen times before the data is complete.

Is there something I'm overlooking in the manpage or some hack for a
workaround? 

I'm aware the RFC 2616 (14.35.2 Range Retrieval Requests) says, "A
server MAY ignore the Range header."  Is it the case that serving data
referenced by an HTTPS URL always ignores a range header that says,
essentially, "Got >this-much<, send me the rest"?

There are a couple of sites -- SlashDot is one -- that *almost always*
send RST before completing transfer -- possibly something to do with
dialup speed being detected and evaluated by the server as failure.  A
browser will just cancel the transaction.  With wget, if I wait long
enough, for enough retries, I can usually get the data in a local
file.  It's a PITA, made worse by wget's restarting at byte 0 for each
retry.

Sorry I missed the cafe meet.  I had a reason to be in Hfx that day
but it fell through.  Maybe the next one.


- Mike

-- 
Michael Spencer                  Nova Scotia, Canada       .~. 
                                                           /V\ 
mspencer at tallships.ca                                     /( )\
http://home.tallships.ca/mspencer/                        ^^-^^


More information about the nSLUG mailing list