Re: [Tutor] Unable to retreive the stock code
On 16Nov2015 15:41, Crusier wrote:
I am currently trying to download the stock code. I am using Python
3.4 and the code is as follows:
[...]
for link in soup.find_all("a"):
stock_code = re.search('/d/d/d/d/d', "1" )
print(stock_code, '', link.text)
[...]
I am trying to retrieve the stock code from here:
1
or from a href.
Well it looks like you have all the needed libraries. You're doing a few things
wrong above. Firstly, to match a digit you need "\d", not "/d". Secondly, your
use of "re.search" searches the literal string "1" instead of, presumably,
the value of link.text. Thirdly, the return from "re.search" is not a stock
code but a "match object"; I would not call it "stock_code" but
"stock_code_match". That will contain a reference to the stock code; since
your regexp matches the stock code then stock_code_match.group(0) will return
the actual matched text, the stock code.
Fix that stuff and see where you are.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Unable to retreive the stock code
Crusier wrote:
> Dear All,
>
> I am currently trying to download the stock code. I am using Python
> 3.4 and the code is as follows:
>
> from bs4 import BeautifulSoup
> import requests
> import re
>
> url =
> 'https://www.hkex.com.hk/eng/market/sec_tradinfo/stockcode/eisdeqty.htm'
>
> def web_scraper(url):
> response = requests.get(url)
> html = response.content
> soup = BeautifulSoup(html,"html.parser")
> for link in soup.find_all("a"):
> stock_code = re.search('/d/d/d/d/d', "1" )
> print(stock_code, '', link.text)
> print(link.text)
>
> web_scraper(url)
>
> I am trying to retrieve the stock code from here:
> 1
>
> or from a href.
>
> Please kindly inform which library I should use.
The good news is that you don't need regular expressions here, just
beautiful soup is sufficient.
Have a look at the html source of eisdeqty.html in a text editor,
and then use the interactive interpreter to get closer to the desired result:
Python 3.4.3 (default, Oct 14 2015, 20:28:29)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> from bs4 import BeautifulSoup
>>> url =
>>> 'https://www.hkex.com.hk/eng/market/sec_tradinfo/stockcode/eisdeqty.htm'
>>> soup = BeautifulSoup(requests.get(url).content)
[snip some intermediate attempts]
>>> soup.html.body.table.table.table.table.tr.td
STOCK CODE
>>> stock_codes = [tr.td.text for tr in
>>> soup.html.body.table.table.table.table.find_all("tr")]
>>> stock_codes[:10]
['STOCK CODE', '1', '2', '3', '4', '5', '6', '7',
'8', '9']
>>> stock_codes[-10:]
['06882', '06886', '06888', '06889', '06893', '06896', '06898', '06899',
'80737', '84602']
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] os.popen - using commands and input %
Hi Guys,
OS = SuSE Enterprise Linux
Python V2.7
My code is as follows
# this list contains system process ID's
pidst=[1232, 4543, 12009]
pmap_str=[]
command="pmap -d %s | grep private |awk '{print $1}' | awk -FK '{print $1}'"
for i in range(len(pids)):
pmap_str.append(os.popen("(command) % pidlist_int[i])")) # <--
this is where I need help, please
As I'm sure you can see, I'm trying to output the os.popen output to a new
list.
On the shell console I can run the pmap command as follows
pmap -d | grep private |awk '{print $1}' | awk -FK '{print $1}'
Output will be a single number such as 485921.
My error is on the line of code shown above. Please assist.
Kind Regards
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.popen - using commands and input %
The following code seems to be pointing me to the right direction, BUT, my
list has 0's instead of the output generated.
>>> for i in range(len(pids)):
... final.append(subprocess.call(["sudo pmap -d %s | grep private |awk
'{print $1}' | awk -FK '{print $1}'" % pids[i]], shell=True))
...
60772
106112
3168
13108
14876
8028
3328
8016
139424
6037524
5570492
4128
144364
154980
154980
>>> pmap_str
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
I;m assuming the zero's are exit codes, which then populate the list, which
is not what I'm after. .
On Mon, Nov 16, 2015 at 2:17 PM, Vusa Moyo wrote:
> Hi Guys,
>
> OS = SuSE Enterprise Linux
> Python V2.7
>
> My code is as follows
>
> # this list contains system process ID's
> pidst=[1232, 4543, 12009]
>
> pmap_str=[]
> command="pmap -d %s | grep private |awk '{print $1}' | awk -FK '{print
> $1}'"
>
> for i in range(len(pids)):
> pmap_str.append(os.popen("(command) % pidlist_int[i])")) # <--
> this is where I need help, please
>
> As I'm sure you can see, I'm trying to output the os.popen output to a new
> list.
>
> On the shell console I can run the pmap command as follows
>
> pmap -d | grep private |awk '{print $1}' | awk -FK '{print $1}'
>
> Output will be a single number such as 485921.
>
> My error is on the line of code shown above. Please assist.
>
> Kind Regards
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.popen - using commands and input %
SOLVED> the code I used was.
for i in range(len(pids)):
final.append(subprocess.Popen(["sudo pmap -d %s | grep private |awk
'{print $1}' | awk -FK '{print $1}'" % pids[i]], shell=True,
stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()[0])
Allowed me to append the subprocess output to my list.
Thanks for the help everyone :-)
On Mon, Nov 16, 2015 at 3:07 PM, Vusa Moyo wrote:
> The following code seems to be pointing me to the right direction, BUT, my
> list has 0's instead of the output generated.
>
> >>> for i in range(len(pids)):
> ... final.append(subprocess.call(["sudo pmap -d %s | grep private |awk
> '{print $1}' | awk -FK '{print $1}'" % pids[i]], shell=True))
> ...
> 60772
> 106112
> 3168
> 13108
> 14876
> 8028
> 3328
> 8016
> 139424
> 6037524
> 5570492
> 4128
> 144364
> 154980
> 154980
> >>> pmap_str
> [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
>
> I;m assuming the zero's are exit codes, which then populate the list,
> which is not what I'm after. .
>
> On Mon, Nov 16, 2015 at 2:17 PM, Vusa Moyo wrote:
>
>> Hi Guys,
>>
>> OS = SuSE Enterprise Linux
>> Python V2.7
>>
>> My code is as follows
>>
>> # this list contains system process ID's
>> pidst=[1232, 4543, 12009]
>>
>> pmap_str=[]
>> command="pmap -d %s | grep private |awk '{print $1}' | awk -FK '{print
>> $1}'"
>>
>> for i in range(len(pids)):
>> pmap_str.append(os.popen("(command) % pidlist_int[i])")) # <--
>> this is where I need help, please
>>
>> As I'm sure you can see, I'm trying to output the os.popen output to a
>> new list.
>>
>> On the shell console I can run the pmap command as follows
>>
>> pmap -d | grep private |awk '{print $1}' | awk -FK '{print $1}'
>>
>> Output will be a single number such as 485921.
>>
>> My error is on the line of code shown above. Please assist.
>>
>> Kind Regards
>>
>
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Debugging in Python
Hello forum, I am trying Python 3.4 on Ubuntu and I am a bit confused with the debugging scope of python in general. I wrote a small function and then I tried to run with the following call: /// import hilbert hilbert.hilbert(3) /// Please note that , I am not using the IDLE IDE. Instead I am using the typical IDE that shows up with the command "python3" in the linux command line. I encountered some error in the source , then I fixed it and tried to run the module with the above snippet again , but the the error prevails even though the error part is commented and the updated version saved. If I get out of the python3 console entirely, get into it again, import the module and execute, then the last update in the source is effected. Why is it so clumsy ? I believe that there is better way to program in python which I am missing ? Thanks ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Debugging in Python
On 16/11/15 09:55, Sajjadul Islam wrote:
Hello forum,
I am trying Python 3.4 on Ubuntu and I am a bit confused with the debugging
scope of python in general.
I wrote a small function and then I tried to run with the following call:
///
import hilbert
hilbert.hilbert(3)
///
Please note that , I am not using the IDLE IDE. Instead I am using the
typical IDE that shows up with the command "python3" in the linux command
line.
Thats not an IDE its just a raw interpreter.
IDLE is a full IDE that includes a debugger.
I encountered some error in the source , then I fixed it and tried to run
the module with the above snippet again , but the the error prevails even
though the error part is commented and the updated version saved.
You need to reload the module but sadly there is no simple command to do
that in the interpreter. There is in IDLE ("Restart Shell" menu item)
So if you use IDLE your issues will be resolved. You can install
IDLE3 from the Ubuntu package manager.
Why is it so clumsy ? I believe that there is better way to program in
python which I am missing ?
Use IDLE :-)
Or any of the other IDEs.
Alternatively use multiple windows.
I personally prefer to have a vim window for editing the code and
a separate terminal for running it. But that assumes I have a
main file and can just type
$ python myfile.py.
In the terminal.
If I'm only creating a module I use the IDLE shell instead of the
terminal window.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] os.popen - using commands and input %
On 16/11/15 14:05, Vusa Moyo wrote:
SOLVED> the code I used was.
for i in range(len(pids)):
final.append(subprocess.Popen(["sudo pmap -d %s | grep private |awk
'{print $1}' | awk -FK '{print $1}'" % pids[i]], shell=True,
Glad you solved it and using subprocess is fine for the pmap stuff.
But the grep and awk stuff are usually a lot faster and almost as
easy using Python.
eg.
for line in pmap_output
if private in line:
print line.split()[0]# {print $1}
Or, to create a list instead of printing
col1 = [line.split()[0] for line in pmap_output if private in line]
I can't remember what -FK does (F is field sep but K?)
But I'm fairly sure it will be easy to replicate in Python.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Debugging in Python
On 16 November 2015 at 15:43, Alan Gauld wrote:
> Thats not an IDE its just a raw interpreter.
> IDLE is a full IDE that includes a debugger.
It’s an awful piece of garbage that pretends to be an IDE.
>> I encountered some error in the source , then I fixed it and tried to run
>> the module with the above snippet again , but the the error prevails even
>> though the error part is commented and the updated version saved.
>
>
> You need to reload the module but sadly there is no simple command to do
> that in the interpreter. There is in IDLE ("Restart Shell" menu item)
>
> So if you use IDLE your issues will be resolved. You can install
> IDLE3 from the Ubuntu package manager.
No, they won’t. They will be replaced by a much worse and less
friendly IDE. Please don’t bother installing IDLE and use the normal
`python3` shell, ipython or bpython.
The correct fix is to exit() from the python3 shell and start it again.
Alternatively, add some main code at the end of your file and use
`python3 hlibert.py`:
if __name__ == '__main__':
hilbert(3)
--
Chris Warrick
Re: [Tutor] Debugging in Python
On 16/11/15 15:42, Chris Warrick wrote:
On 16 November 2015 at 15:43, Alan Gauld wrote:
Thats not an IDE its just a raw interpreter.
IDLE is a full IDE that includes a debugger.
It’s an awful piece of garbage that pretends to be an IDE.
Would you care to expand. Its been doing a fair impression
for the past 20 odd years. And in its IDLEX incarnation most
of the niggles have been removed. It's not Eclipse but it
doesn't pretend to be. But for beginners like the OP its
adequate IMHO. Certainly easier than the raw interpreter
prompt.
You need to reload the module but sadly there is no simple command to do
that in the interpreter. There is in IDLE ("Restart Shell" menu item)
So if you use IDLE your issues will be resolved. You can install
IDLE3 from the Ubuntu package manager.
No, they won’t. They will be replaced by a much worse and less
friendly IDE. Please don’t bother installing IDLE and use the normal
`python3` shell, ipython or bpython.
He's tried the normal shell and that didn't work for him.
iPython and bPython are certainly other more advanced options
but he needs to install them too and both have a steeper
learning curve for the average user than IDLE.
The correct fix is to exit() from the python3 shell and start it again.
He's tried that and didn't find it satisfactory. That's why
he wants a "better" workflow.
Alternatively, add some main code at the end of your file and use
`python3 hlibert.py`:
if __name__ == '__main__':
hilbert(3)
That depends on the nature of his module and the type of testing he
wants to do. He could, I agree, create a new test file that imports
hilbert and then add his test code there but that loses the
interactivity of a shell prompt which may be important.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] question about "__main__"
Hello, I’m learning Python. I want to know what does the following line represent and why I see some python codes have two of them ? if __name__ == ‘__main__’: Thanks in advance Cuong ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] question about "__main__"
On 16/11/15 15:30, CUONG LY wrote: Hello, I’m learning Python. Hello, welcome. I want to know what does the following line represent if __name__ == ‘__main__’: The short answer is that it tests whether the file is being imported as a module or executed as a program. If its being imported as a module its __name__ attribute will be set to the name of the module. If its being executed as the main (top level) program file its __name__ will be set to "__main__". So when you see: if __name__ == ‘__main__’: # some code here the 'some code here' block will only be executed if the file is being executed as a program. It will not be executed if the file is being imported by another file. That will only make sense if you have come across modules yet. If not don't worry about it, you will get there eventually. Then it will all become clear. > and why I see some python codes have two of them ? I don't know, I've never seen two. But it is perfectly acceptable to do it, just a little pointless - unless perhaps they are inside a function or something. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
