[Tutor] Moving a conda environment to an off-line computer
Dear colleagues,
Soon I'll start to use one of the powerful computers on my university as a tool
in my Ph.D. The computer does not have an internet connection and I need to
find a way to install a conda environment on it.
At first I tried to install and set the conda environment that I need in a
computer with internet connection and taking care to copy everything in a path
that is similar in the off-line computer (i.e I installed everything on
/home/henrique/bin/anaconda3 at home and tried to copy everything to
/home/henrique/bin/anaconda3 in the off-line computer - with the same .bashrc)
but when I run conda I get an error(it works on my home computer):
(deepchem) [henrique@europio qm7] $ python qm7_ANI.py
/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29:
DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and
should not be imported. It will be removed in a future NumPy release.
from numpy.core.umath_tests import inner1d
Traceback (most recent call last):
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
1026, in _send_output
self.send(msg)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
964, in send
self.connect()
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", line
936, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py",
line 704, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py",
line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "qm7_ANI.py", line 15, in
featurizer='BPSymmetryFunction', split='stratified', move_mean=False)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/deepchem/molnet/load_function/qm7_datasets.py",
line 50, in load_qm7_from_mat
'http://deepchem.io.s3-website-us-west-1.amazonaws.com/datasets/qm7.mat'
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/deepchem/utils/__init__.py",
line 85, in download_url
urlretrieve(url, os.path.join(dest_dir, name))
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 248, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 223, in urlopen
return opener.open(url, data, timeout)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 526, in open
response = self._open(req, data)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 544, in _open
'_open', req)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 504, in _call_chain
result = func(*args)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 1346, in http_open
return self.do_open(http.client.HTTPConnection, req)
File
"/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
line 1320, in do_open
raise URLError(err)
urllib.error.URLError:
Any help is much appreciated
--
Henrique C. S. Junior
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving a conda environment to an off-line computer
Hi, The computer is a cluster with 4 nodes each running Linux (our Fedora-based distro), 12 cores/64 GB RAM per node using CUDA. At first, I'll be using only one node. -- Henrique C. S. Junior From: Tutor on behalf of Alan Gauld via Tutor Sent: Friday, November 30, 2018 22:33 To: tutor@python.org Subject: Re: [Tutor] Moving a conda environment to an off-line computer On 30/11/2018 13:47, Henrique Castro wrote: > ... use one of the powerful computers on my university It would help if we knew what kind of powerful computer this is. What OS does it run? It sounds like some variety of Unix, but which? Does it have a C compiler on board? > The computer does not have an internet connection > and I need to find a way to install a conda environment on it. This may be a question that the Anaconda community can better answer since building your own Anaconda suite is non trivial. So unless you find a pre-built environment for your new OS which you can download and copy to removable media you will likely need specialist help. That's more likely to be found on the Anaconda fora than on a general Python list like this one. > File > "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py", > line 936, in connect > (self.host,self.port), self.timeout, self.source_address) > File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py", > line 704, in create_connection > for res in getaddrinfo(host, port, 0, SOCK_STREAM): > File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py", > line 745, in getaddrinfo > for res in _socket.getaddrinfo(host, port, family, type, proto, flags): > socket.gaierror: [Errno -2] Name or service not known Apparently the test file tries to access the internet so will obviously fail on a non internet connected device. (BTW is it on any kind of network or is it entirely standalone?) -- Alan G Author of the Learn to Program web site https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.alan-g.me.uk%2F&data=02%7C01%7C%7C2a739252872d48babd6208d65725104e%7C84df9e7fe9f640afb435%7C1%7C0%7C636792214023931869&sdata=8KuHyIuihRIwDB1bKIF7ZS7PJRwdtNLMVk7AYNOW%2B1s%3D&reserved=0 https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.amazon.com%2Fauthor%2Falan_gauld&data=02%7C01%7C%7C2a739252872d48babd6208d65725104e%7C84df9e7fe9f640afb435%7C1%7C0%7C636792214023931869&sdata=mgXxj1AChik%2B%2F2lCfL%2BNdJdfhUFDc2XLmEwgIMs8n50%3D&reserved=0 Follow my photo-blog on Flickr at: https://eur04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.flickr.com%2Fphotos%2Falangauldphotos&data=02%7C01%7C%7C2a739252872d48babd6208d65725104e%7C84df9e7fe9f640afb435%7C1%7C0%7C636792214023931869&sdata=HtkISkcgLNnKMFTIZcqT3VEZaBbPVfdRV11M16YjpQI%3D&reserved=0 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Ftutor&data=02%7C01%7C%7C2a739252872d48babd6208d65725104e%7C84df9e7fe9f640afb435%7C1%7C0%7C636792214023931869&sdata=vrmETDxK13BxuOWQ2cSMG13xmb%2FX3ly85NvVwqEUjXg%3D&reserved=0 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving a conda environment to an off-line computer
Thank you guys, I'll try to contact the Conda community. Alan Gauld, good to know. I suggest that you try to spread the news to Fermi Lab, CERN and other research centers. Sometimes when you need a specific setting it is just easier to pre-build your distro than to repeat the same configuration hundreds of times. -- Henrique C. S. Junior From: Tutor on behalf of Alan Gauld via Tutor Sent: Saturday, December 1, 2018 07:12 To: tutor@python.org Subject: Re: [Tutor] Moving a conda environment to an off-line computer On 01/12/2018 00:43, Henrique Castro wrote: >a cluster with 4 nodes each running Linux (our Fedora-based distro), OK, The good news is you should be able to get a pre-built distro rather than try to build your own. But I still think the Conda users are more likely to be able to assist. -- Alan G Author of the Learn to Program web site https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.alan-g.me.uk%2F&data=02%7C01%7C%7C29a2850e262c4fd9148c08d6576d86d0%7C84df9e7fe9f640afb435%7C1%7C0%7C636792525248838704&sdata=0xTl3WSOMgfGih2FYN8AYSzXORCRy9fKLN6iLk3txlk%3D&reserved=0 https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.amazon.com%2Fauthor%2Falan_gauld&data=02%7C01%7C%7C29a2850e262c4fd9148c08d6576d86d0%7C84df9e7fe9f640afb435%7C1%7C0%7C636792525248838704&sdata=FVJGUemjFp3fRyu%2FK7PpkefITCLyjLmZ1DGuhmfLqtU%3D&reserved=0 Follow my photo-blog on Flickr at: https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.flickr.com%2Fphotos%2Falangauldphotos&data=02%7C01%7C%7C29a2850e262c4fd9148c08d6576d86d0%7C84df9e7fe9f640afb435%7C1%7C0%7C636792525248838704&sdata=UTIgec8%2BlrRQ75j8j%2FFla5CYsU7%2FWywZb0D4kY6Gefs%3D&reserved=0 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Ftutor&data=02%7C01%7C%7C29a2850e262c4fd9148c08d6576d86d0%7C84df9e7fe9f640afb435%7C1%7C0%7C636792525248838704&sdata=lR6inUjJWRGm9TZ2MwyoWXLkXSinM1lCbkCcJV6MGuw%3D&reserved=0 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Moving a conda environment to an off-line computer
Thank you for all the insightful replies!
It looks like the problem is exactly that deepchem is looking for an internet
connection and crashing.
I'll see what I can do and provide some feedback here.
Again, thank you for using some of your time to take a look at this.
--
Henrique C. S. Junior
From: Matt Ruffalo
Sent: Sunday, December 2, 2018 13:27
To: Henrique Castro; tutor@python.org
Subject: Re: Moving a conda environment to an off-line computer
Hi Henrique-
It is quite easy to transfer an Anaconda installation from one machine
to the other by copying all of the files -- I have done this repeatedly
with cluster compute environments. It is sometimes nicer to run `conda
upgrade --all` in a local VM and then `rsync` the updated Anaconda
installation between machines, since (as you mentioned) internet access
can sometimes be an issue.
It looks like you did everything correctly, and everything is "working"
as well as you would expect. As Alan mentioned, though, it looks like
the 'deepchem' package is trying to access the internet to load one of
its data sets, and this is what is failing. You could perhaps download
that data set and put it somewhere on the cluster where deepchem would
know where to look for it, to avoid having to download it, but I am
completely unfamiliar with deepchem so I can't offer any advice about
how to do that.
MMR...
On 30/11/18 08:47, Henrique Castro wrote:
> Dear colleagues,
> Soon I'll start to use one of the powerful computers on my university as a
> tool in my Ph.D. The computer does not have an internet connection and I need
> to find a way to install a conda environment on it.
> At first I tried to install and set the conda environment that I need in a
> computer with internet connection and taking care to copy everything in a
> path that is similar in the off-line computer (i.e I installed everything on
> /home/henrique/bin/anaconda3 at home and tried to copy everything to
> /home/henrique/bin/anaconda3 in the off-line computer - with the same
> .bashrc) but when I run conda I get an error(it works on my home computer):
>
> (deepchem) [henrique@europio qm7] $ python qm7_ANI.py
> /home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29:
> DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and
> should not be imported. It will be removed in a future NumPy release.
> from numpy.core.umath_tests import inner1d
> Traceback (most recent call last):
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/urllib/request.py",
> line 1318, in do_open
> encode_chunked=req.has_header('Transfer-encoding'))
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 1239, in request
> self._send_request(method, url, body, headers, encode_chunked)
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 1285, in _send_request
> self.endheaders(body, encode_chunked=encode_chunked)
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 1234, in endheaders
> self._send_output(message_body, encode_chunked=encode_chunked)
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 1026, in _send_output
> self.send(msg)
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 964, in send
> self.connect()
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/http/client.py",
> line 936, in connect
> (self.host,self.port), self.timeout, self.source_address)
> File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py",
> line 704, in create_connection
> for res in getaddrinfo(host, port, 0, SOCK_STREAM):
> File "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/socket.py",
> line 745, in getaddrinfo
> for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
> socket.gaierror: [Errno -2] Name or service not known
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File "qm7_ANI.py", line 15, in
> featurizer='BPSymmetryFunction', split='stratified', move_mean=False)
> File
> "/home/henrique/bin/anaconda3/envs/deepchem/lib/python3.6/site-packages/deepchem/molnet/load_function/qm7_datasets.py",
> line 50, in load_qm7_from_mat
>
> 'https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdeepchem.io.s3-website-us-west-1.amazonaws.
