Shannon Dealy <de...@deatech.com> writes:
> On Thu, 4 Dec 2014, Nikolaus Rath wrote:
>
>> Shannon Dealy <de...@deatech.com> writes:
> [snip]
>> Unfortunately Linux does not provide an easy way to distinguish between
>> an unavailable DNS server, and an un-resolvable host name. To
>> distinguish these cases, S3QL/Dugong attempts to resolve a number of
>> "test" hostnames. If these resolve, but the S3 hostname does not,
>> S3QL/Dugong concludes that this hostname is not resolvable and
>> terminates. Otherwise it assumes that the DNS server is currently not
>> reachable and retries.
>>
>> Attempting to resolve hostnames on your system frequently fails
>> (sometimes 3 times in a row), and sometimes it's apparently sufficiently
>> flaky that
>>
>> 1. server-external-2014-11-21-deatech-com-s3ql.s3.amazonaws.com cannot
>> be resolved
>>
>> 2. Any of google.com, iana.org, or root-servers.net can be resolved
>>
>> 3. server-external-2014-11-21-deatech-com-s3ql.s3.amazonaws.com cannot
>> be resolved
>>
>> in this order, and without any waiting times.
>>
>>
>> At this point, S3QL thus assumes that your bucket ceased to exist and
>> terminates.
>>
>> To avoid this, you'll have to fix the DNS resolution issues on your
>> system. Maybe install a caching proxy nameserver like dnsmasq?
>
> While I can try dnsmasq as a solution, I was not having the degree of
> problems I am seeing under the old version of s3ql, and this is not
> the only network I connect to where I am seeing these problems. This
> could simply mean one or more of:
>
> - the network(s) have become more unstable (certainly possible)
> - that S3QL's test doesn't work quite the same way as before
> - there is a bug somewhere (S3QL or python libraries - I had to upgrade
> python to install the new S3QL) that makes it appear that DNS is still
> broken after it recovers
> - or something else is going on.
The relevant code in S3QL 2.2 was last touched in version 2.2, so I
don't think that's it. However, relevant code in Dugong was last
modified in version 3.2. Prior to 3.2, *any* DNS problem was considered
temporary. In 3.2 and later, DNS problems are only considered temporary
of *no* hostname can be resolved.
The problem with the prior behavior was that it would permanently retry
even in situations like this:
mkfs.s3ql s3://mybucket.aws.s3.cmo
Having this command block for any significant amount of time in the hope
that the wrong hostname becomes resolvable was rather surprising, and
people complained that their scripts would just hang instead of properly
reporting an error.
It seems for you the situation is the other way around...
> I see in the log I sent you that a few failures were reported 10
> minutes before S3QL gave up, but it is not clear to me if S3QL was
> able to resolve DNS names between these. Am I correct in assuming
> that there a timer as well as the three time in a row failure limit
> you mentioned?
There's no limit on the number of retries. After the 3rd retry, S3QL
starts to emit log messages. The time interval between the retries is
increasing over time.
However, all this happens only if the DNS server is unreachable. If the
DNS server is apparently reachable, but unable to resolve the hostname,
no retrying is done.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
