Hello, Scrapy users!
We are happy to announce that we just shipped Scrapy 1.1 release candidate
1, which brings beta Python 3 support to Scrapy!
The team has been working hard to bring this highly anticipated feature and
we are very grateful to all developers and users who contributed to it --
thank you! =)
This version also comes with a lot of new features and bug fixes. Here is a
quick summary highlighting the most relevant ones:
Hot new features:
Item loaders now support nested loaders.
FormRequest.from_response improvements.
Added setting AUTOTHROTTLE_TARGET_CONCURRENCY and improved AutoThrottle
docs.
response.text to get response body as unicode.
Anonymous S3 connections are now supported.
Downloader middlewares now can return deferreds.
HTTP caching now follows RFC2616 more closely.
Selectors were extracted to the parsel library. This means you can use
Scrapy Selectors without Scrapy and also upgrade the selectors engine
without needing to upgrade Scrapy.
Behavior changes and fixes that may require your attention:
Scrapy doesn't retry bad requests (HTTP 400) by default. If you need
the old behavior, add 400 to RETRY_HTTP_CODES.
Fix shell files argument handling. If you try scrapy shell index.html
it will try to load the URL http://index.html, use scrapy shell
./index.html to load a local file.
Robots.txt compliance is now enabled by default for newly-created
projects. Scrapy will also wait for robots.txt to be downloaded before
proceeding with the crawl. If you want to disable this behavior, update
ROBOTSTXT_OBEY in settings.py file after creating a new project.
Exporters now work on unicode, instead of bytes by default. If you use
PythonItemExporter, you may want to update your code to disable binary mode
which is now deprecated.
Accept XML node names containing dots as valid.
Check out other new features and fixes in the full release notes at:
http://doc.scrapy.org/en/stable/news.html#id1
Please give Scrapy 1.1.0rc1 a try, check out the cool new features and
Python 3 support, feel free to report any problems at:
https://github.com/scrapy/scrapy/issues
Install Scrapy 1.1.0rc1 via pip:
$ pip install Scrapy==1.1.0rc1
Thank you, happy scraping!
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
