4. Anaconda, Pytorch, Tensorflow, and Rapids
And finally, we reach the crux. We will install the software which we will interface with most of the times.
We need to install Python with virtual environments. I have downloaded python3 as it is the most stable version as of now, and it is time to say goodbye to Python 2.7. It was great while it lasted. And we will also install Pytorch and Tensorflow. I prefer them both for specific tasks as applicable.
You can go to the anaconda distribution page and download the package.
Once downloaded you can simply run the shell script:
sudo sh Anaconda3-2019.10-Linux-x86_64.sh
You will also need to run these commands on your shell to add some commands to your
~/.bashrc file, and update the conda distribution with the latest libraries versions.
cat >> ~/.bashrc << 'EOF'
conda upgrade -y – all
The next step is creating a new environment for your deep learning pursuits or using an existing one. I created a new Conda environment using:
conda create – name py37
Here py37 is the name we provide to this new conda environment. You can activate this conda environment using:
conda activate py37
You should now be able to see something like:
We can now add all our required packages to this environment using pip or conda. The latest version 1.3, as seen from the pytorch site, is not yet available for CUDA 10.2, as I already mentioned, so we are in luck with CUDA 10.1. Also, we will need to specify the version of TensorFlow as 2.1.0, as this version was built using 10.1 CUDA.
I also install RAPIDS, which is a library to get your various data science workloads to GPUs. Why use GPUs only for deep learning and not for Data processing? You can get the command to install rapids from the rapids release selector:
sudo apt install python3-pipconda install -c rapidsai -c nvidia -c conda-forge -c defaults rapids=0.11 python=3.7 cudatoolkit=10.1pip install torchvision
Since PyTorch installation interfered with TensorFlow, I installed TensorFlow in another environment.
conda create – name tf
conda activate tf
pip install – upgrade tensorflow
Now we can check if the TF and Pytorch installations are correctly done by using the below commands in their own environments:
# Should print True
python3 -c "import tensorflow as tf; print(tf.test.is_gpu_available())"# should print cuda
python3 -c "import torch; print(torch.device('cuda' if torch.cuda.is_available() else 'cpu'))"
If the install is showing some errors for TensorFlow or the GPU test is failing, you might want to add these two additional lines at the end of your
bashrc file and restart the terminal:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/lib64export CUDA_HOME=/usr/local/cuda
You might also want to install
jupyter lab or
jupyter notebook. Thanks to the developers, the process is as easy as just running
jupyter notebook in your terminal, whichever you do prefer. I personally like notebook better without all the unnecessary clutter.