Hello Rich,
> On 01 Mar 2016, at 01:44, Rich Jones wrote:
>
> I think the interesting things to explore would be:
> - Loading apps in parallel
> - "Pre-loading" apps before app deployment, then loading that cached state
> at runtime. I guess I'd need to know more about what it means to "load
In my opinion, those latencies are still a bit high, how much is really
used on python/lambda code? On a project of mine without hitting the
database and django-rest-framework my times were around 1-4ms excluding any
network latency. Debug and loggers might have high impact on your times if
it
Ah, interesting! Thanks for tracking that down.
In chat we basically discovered that the intercontinental latency and
shoddy wifi connections were responsible for a lot of the confusion.
Testing from a US-based fiber connection, we got results of ~40ms in both
scenarios.
---
https://arb9clq9
I think I have found your issue, if I'm not wrong, django won't initialize
twice if you call it twice (which was your previous behaviour) at least not
completely, since apps registry has a "ready" flag.
Check this line:
https://github.com/django/django/blob/master/django/apps/registry.py#L66
So
Hm. This is the downside of cloud services - we cannot look under the hood.
Since I think that since this is something we _want_ cached, and because it
will make the function being executed shorter in length - it is a good code
decision to make. Thank you for the idea! However, it looks like the
Rich, I have just performed a real test, with a simple lambda and a
datetime.now() defined at the top of the module as I said, and out of 100
requests, this was the result:
{u'2016-03-01T00:37:30.476828': [43], u'2016-03-01T00:36:51.536025': [58]}
Where the date is the datetime.now() defined at t
That certainly could have something to do with it - there isn't very much
transparency about how API Gateway works. It's super new and pretty janky,
TBQH. However, I think that the behavior describing is not what's expected
- the caching seems to be for the assets of the whole environment, not o
I have repeated the test this time through API Gateway, and out of many
calls I only got two different dates that were instantiated at module
level, meaning my module was only imported twice. I fail to see why it
doesn't behave the same with your code.
El lunes, 29 de febrero de 2016, 21:33:05
That's quite odd, I recall testing this once, where I created a lambda
which had a datetime.now() at the top, and just returned that value. Out of
a few calls, it returned two different results, meaning the module was re
used "most" of the time. This was tested calling the lambda from the AWS
T
As I suspected, moving setup() outside of the handler had a negligible
effect - in fact the test showed a slight drop in performance. :(
Testing from httping. From Berlin to US-East-1:
Before:
---
https://arb9clq9k9.execute-api.us-east-1.amazonaws.com/unicode/json_example/
ping statistics ---
I'm almost sure that right now you are calling setup() with django already
initialized in some cases where the enviorment is reused, I'm amazed django
doesn't complain when setup() is called twice.
El lunes, 29 de febrero de 2016, 21:07:32 (UTC-3), Rich Jones escribió:
>
> Haven't tried that! I
Haven't tried that! I don't _think_ that'll work.. but worth a shot (I
don't think they only cache the handler.. I think they cache the whole
environment). Will report back! And as you're mentioning, we really
shouldn't be doing it every request, so if there were even a way to cache
that manual
Sorry if this looks like a retarded question, but have you tried the setup
calls to the top of the lambda_handler module? I'm not sure why you need
the settings data as an argument to the lambda handler, but if you find a
way to move those 4 lines near setup(), you will only load the whole djang
For those who are still following along, these are the lines in question:
https://github.com/Miserlou/django-zappa/blob/master/django_zappa/handler.py#L31
https://github.com/Miserlou/django-zappa/blob/master/django_zappa/handler.py#L68
It's very possible there are ways of significantly improving
Hey all!
Let me clarify a few of the terms here and describe a bit how Django
operates in this context.
"Serverless" in this contexts means "without any permanent infrastructure".
The server is created _after_ the request comes in, and it dies once the
response is returned. The means that we n
Hi Collin,
On Friday, February 26, 2016 at 10:56:33 PM UTC+1, Collin Green wrote:
>
> Ugh. As a strong advocate of both django and zappa, I'd love it if we
> could keep the conversation on target without degenerating into stack
> attacks. If you don't want to weigh in, please feel no obligation
Ugh. As a strong advocate of both django and zappa, I'd love it if we could
keep the conversation on target without degenerating into stack attacks. If
you don't want to weigh in, please feel no obligation to do so.
I do agree that we could pare down the profile into more actionable
sections. I
Rich,
I believe you know a lot way more than me about AWS Lambda since you have
made such a great project, and I'm really interested to know how it really
works since theyr documentation is a bit superficial.
On their FAQ this is what they state:
*Q: Will AWS Lambda reuse function instances?*
On Friday, February 26, 2016 at 9:53:00 PM UTC+1, Rich Jones wrote:
>
> @Aymeric
> > In my opinion, the first concrete step would be to measure how much time
> is spent executing Django code rather than importing Python modules.
>
> Over 70% of the total request time is spent in django.setup() -
@Aymeric
> In my opinion, the first concrete step would be to measure how much time
is spent executing Django code rather than importing Python modules.
You can find a complete profile of a Django request as it goes through the
complete request/response loop here:
https://github.com/Miserlou/dj
If with "serverless" you are talking deployments such as Amazon Lambda or
similar, I don't think setup is called on every request, at least for AWS
Lambda, the enviorment is cached so it will only happen once each time it
needs to scale up. Are there any other issues?
El viernes, 26 de febrero
Hi Rich,
On 26 févr. 2016, at 16:36, Rich Jones wrote:
> I'd love to discuss ideas for performance optimizations to Django's setup
> method.
In my opinion, the first concrete step would be to measure how much time is
spent executing Django code rather than importing Python modules. That’s t
That might be true in theory.. but in practice?
Do any of the core/contrib/top100 apps actually depend on loading orders?
I've never encountered that problem before, but I don't know.
It seems like we could easily get a 10x performance improvement by doing
this in parallel by default, and have
I'm not sure we can load apps in parallel -- they are loaded in the order
in which they appear in INSTALLED_APPS so the process is consistent. Maybe
it would work for some setups, but I guess it would introduce subtle bugs
in others.
https://docs.djangoproject.com/es/stable/ref/applications/#ho
24 matches
Mail list logo
