[nSLUG] wget & interrupted fetch

George N. White III gnwiii at gmail.com
Mon Sep 26 11:35:28 ADT 2016

On Sat, Sep 24, 2016 at 4:10 AM, Mike Spencer <mspencer at tallships.ca> wrote:

> If a file transfer using wget(1) is interrupted, it will restart the
> download where it left off if the data is being saved to a file.  The
> -c option can force this in a new session as well.
> Is there a technical reason why this doesn't work with
> --no-check-certificate and an HTTPS URL?

What version of wget?  Is there a firewall involved?

> In such a case, wget will automatically restart an interrupted
> transfer but it will start with byte 0 of the data.  It does *not*
> delete the local file resulting from the "-O filename" option each
> time but may restart a dozen times before the data is complete.
> Is there something I'm overlooking in the manpage or some hack for a
> workaround?
> I'm aware the RFC 2616 (14.35.2 Range Retrieval Requests) says, "A
> server MAY ignore the Range header."  Is it the case that serving data
> referenced by an HTTPS URL always ignores a range header that says,
> essentially, "Got >this-much<, send me the rest"?

In my experience, "wget -c" works reliable with some sites and not
with others.  Some sites that work do use https, and in fact there
are many more interrupted  transfers using https so without
continuation it can be nearly impossible to download a large file
on a system with an unreliable network connection.

> There are a couple of sites -- SlashDot is one -- that *almost always*
> send RST before completing transfer -- possibly something to do with
> dialup speed being detected and evaluated by the server as failure.  A
> browser will just cancel the transaction.  With wget, if I wait long
> enough, for enough retries, I can usually get the data in a local
> file.  It's a PITA, made worse by wget's restarting at byte 0 for each
> retry.
> Sorry I missed the cafe meet.  I had a reason to be in Hfx that day
> but it fell through.  Maybe the next one.
> - Mike
> --
> Michael Spencer                  Nova Scotia, Canada       .~.
>                                                            /V\
> mspencer at tallships.ca                                     /( )\
> http://home.tallships.ca/mspencer/                        ^^-^^
> _______________________________________________
> nSLUG mailing list
> nSLUG at nslug.ns.ca
> http://nslug.ns.ca/mailman/listinfo/nslug

George N. White III <aa056 at chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nslug.ns.ca/pipermail/nslug/attachments/20160926/d08ee61d/attachment.html>

More information about the nSLUG mailing list