On Thu, Oct 15, 2009 at 09:42:40PM +0300, Jody Fanning wrote: > On 15.10.2009 10:57, Simon Pickering wrote: > >> Why is it necessary to do a "webscrape" of Amazon. Amazon has plenty > >> of simple APIs for getting data freely and easily. The REST based ones > >> are extremely simple and you get back an XML document with everything > >> you need. > >> > > It quite possibly isn't necessary, this is certainly not my area of > > expertise, just something I wanted to get working ASAP. When I looked at > > how to do this I found a mixture of web scraping tools and ones that use > > Amazon Web Services. AFAIK AWS requires that you register to obtain a > > key, and I'm not sure of the exact terms and conditions, but I presume > > they will not want me to share my key, but I may be wrong about this. > > > I'm pretty sure you are able to share the key. I do have an AWS account > and have used it for my own stuff (in Java though). But all those > applications around must have a key embedded since they never ask you to > register yourself. There must be some other open source applications > around that fetch data from Amazon using AWS.
Calibre, for one. The key is embedded right there in the sources.
I don't know if they're allowed to do that since I was too lazy to
actually read the AWS agreement. (Shh! Don't tell anyone.)
I applied for an AWS key, got two (one called "access key", the other
called "secret access key"), did some experiments with pyaws (a Python
library for accessing the AWS). I couldn't get it to work with just the
AWS key, I needed to supply both. I don't know how Calibre manages to
do its thing with just one key (or maybe it's broken now -- I couldn't
figure out the user interface and ran away screaming to invent my own
library management system).
> Amazon did make things a little more complicated a few months ago
> though. All requests must be signed,
This might mean that Calibre was using old unsigned requests at the time
I looked at its source code.
> but they generally provide code
> samples for how to do everything. One thing is to check the licensing of
> the interfaces. There was something recently about different licensing
> between mobile devices and other uses that came up with the iPhone. But
> the advantage is that you won't have any problem with the page layout
> breaking things. Plus their APIs provide pretty much every detail about
> an item you could think of.
Marius Gedminas
--
Give a man a computer program and you give him a headache, but teach him to
program computers and you give him the power to create headaches for others for
the rest of his life...
-- R. B. Forest
signature.asc
Description: Digital signature
_______________________________________________ maemo-developers mailing list [email protected] https://lists.maemo.org/mailman/listinfo/maemo-developers
