Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Lucas Rolff
> The cache is pretty big and I want to limit unnecessary requests if I can. 30gb of cache and ~ 400k hits isn’t a lot. > Cloudflare is in front of my machines and I pay for load balancing, firewall, > Argo among others. So there is a cost per request. Doesn’t matter if you pay for load balanci

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Quintin Par
Hi Peter, Here are my stats for this week: https://imgur.com/a/JloZ37h . The Bypass is only because I was experimenting with some cache warmer scripts. This is primarily a static website. Here’s my URL hit distribution: https://imgur.com/a/DRJUjPc If three people are making the same request, t

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Peter Booth via nginx
Quintin, Are most of your requests for dynamic or static content? Are the requests clustered such that there is a lot of requests for a few (between 5 and 200, say) URLs? If three different people make same request do they get personalized or identical content returned? How long are the cached r

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Quintin Par
Hi Lucas, The cache is pretty big and I want to limit unnecessary requests if I can. Cloudflare is in front of my machines and I pay for load balancing, firewall, Argo among others. So there is a cost per request. Admittedly I have a not so complex cache architecture. i.e. all cache machines

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Lucas Rolff
Can I ask, why do you need to start with a warm cache directly? Sure it will lower the requests to the origin, but you could implement a secondary caching layer if you wanted to (using nginx), so you’d have your primary cache in let’s say 10 locations, let's say spread across 3 continents (US, E

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Quintin Par
Hi Maxim, Thank you for this. Opened my eyes. Not to sounds demanding, but do you have any examples (code) of proxy_store bring used as a CDN. What’s most important to me in the initial cache warming. I should be able to start a new machine with 30 GB of cache vs. a cold start. Thanks once

Re: Avoiding Nginx restart when rsyncing cache across machines

2018-09-12 Thread Maxim Dounin
Hello! On Tue, Sep 11, 2018 at 04:45:42PM -0700, Quintin Par wrote: > I run a mini CDN for a static site by having Nginx cache machines (in > different locations) in front of the origin and load balanced by Cloudflare. > > Periodically I run rsync pull to update the cache on each of these > mach