Docker

When I started programming in Python about four years ago I used the Anaconda distribution for package and environment management. This worked well most of the time, but I had some issues, notably with TA-lib. I gradually discovered that there weren’t as many packages (or specific versions of packages) available through conda as there were on pypi using pip. While some people used both conda and pip with gay abandon, I did see some warnings not to do that. Very confusing. Perhaps the issue was related to using Spyder as my IDE. It had to be installed in each environment, and perhaps had problems with pip-installed packages. I switched to PyCharm as an IDE but still had problems once I started on Machine Learning, especially confilcts between PyTorch/TensorFlow and CPU/GPU installs.

Lately I’ve been using Docker for environment management, which seems to work well although I’m not attempting to use GPU versions of the ML libraries. Biggest problem is that one can’t install new packages in an image without rebuilding the image. Anyway process at the moment is to use a Docker container as a remote interpreter, with the actual python project (or tutorial) files in some directory outside the container. I have to use PyCharm Pro for this, as the Community Edition doesn’t support Docker based remote interpreters.