Wednesday, July 3, 2019

Fix tmux issue with conda

source
Problem: When running a conda environment and opening tmux on macOS, a utility called path_helper is run again. Essentially, the shell is initialized twice which messes up the ${PATH} so that the wrong Python version shows up within tmux.

Solution  If using bash, edit /etc/profile and add one line. (For zsh, edit /etc/zprofile)

Sunday, May 5, 2019

Emacs as Python3 IDE in Mac

Install the following using your preferred method:

pip3 install rope jedi importmagic autopep8 flake8

In Emacs:

M-x package-install RET elpy RET
M-x package-install RET exec-path-from-shell RET
M-x package-install RET pyenv RET
M-x package-install RET anaconda-mode RET

Edit .emacs as following:

(elpy-enable)
(when (memq window-system '(mac ns x))
    (exec-path-from-shell-initialize))
(add-hook 'python-mode-hook 'anaconda-mode)
(setenv "WORKON_HOME" "/anaconda3/envs")
(pyvenv-mode 1)

Restart your emacs and enjoy using the best Python IDE

Use M-x pyvenv-workon to switch between anaconda environments

Monday, April 8, 2019

Mosh

Mosh is similar to SSH, remote terminal application but it allows roaming, and supports intermittent connectivity. Its usage is similar to SSH. It has to be installed both on the server and client.

On RedHat/CentOS remote server:

sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

sudo yum install mosh

sudo firewall-cmd --zone=public --permanent --add-port=60000-61000/udp

sudo firewall-cmd --reload

Azure:

If you're using Azure, make sure to allow the inbound port:



On local Mac:

brew install mosh

Sunday, April 7, 2019

How to recover a shell after a disconnection?


It is not possible! but the cure is tmux. I start tmux, start the operation and go on my way. If I return and find the connection has been broken, all I have to do is reconnect and type tmux attach

For installing tmux on Mac:
   brew install tmux
tmux in the top image is running with vtop on the top pane and PM2 on the bottom.
Here is the cheatsheet for tmux:

Saturday, March 23, 2019

Machine Learning vs. Optimization

Although it may be an odd comparison, sometimes this can be confusing for approaching problems. On one hand, mathematical optimization is used in machine learning during model training, when we are trying to minimize the cost of errors between our model and our data points. On the other hand, machine learning can be used to solve optimization problems. For many actual problems, the data cannot be known accurately for a variety of reasons. The first reason is due to simple measurement error. The second and more fundamental reason is that some data represent information about the future (e.g., product demand or price for a future time period) and simply cannot be known with certainty. The third reason is when obtaining the data is computationally intensive, even close to impossible; for instance, forecasting travel times between every two locations in a city. In these cases, we can use the power of machine learning to do the forecasting job for us. Or Robust optimization techniques can be used when the parameters are known only within certain bounds; the goal is to find a solution that is feasible for all data and optimal in some senses. Stochastic programming models take advantage of the fact that probability distributions governing the data are known or can be estimated; the goal is to find some policy that is feasible for all (or almost all) the possible data instances and optimize the expected performance of the model.

Use Azure as your Jupyter notebook Server

Make sure to follow the steps from this post to setup your Azure Virtual Machine:
http://www.etedal.net/2019/03/setting-up-azure-nc6-for-kaggle.html

Setup Jupyter remote ssh connection:
On Azure: jupyter notebook --no-browser --port=2001
On your machine: ssh -N -f -L localhost:2002:localhost:2001 remoteuser@remotehost
(Optional if you use mosh instead): mosh -ssh "ssh -N -f -L localhost:2002:localhost:2001" remoteuser@remotehost
On your browser: localhost:2002

Wednesday, March 13, 2019

Windows Subsystem for Debian

If you're frustrated using Windows and there is no way around not using it use the following:
https://docs.microsoft.com/en-us/windows/wsl/install-win10

Open PowerShell as Administrator:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Install Debian:
https://www.microsoft.com/en-ca/p/debian/9msvkqc78pk6?rtc=1&activetab=pivot:overviewtab

sudo apt update

Sunday, March 10, 2019

Setting up Azure NC6 for Kaggle Competitions

I have chosen NC6 and Ubuntu 18.04

Install the following using Lambda Stack:
Deep Learning frameworks: TensorFlow, Keras, PyTorch, Caffe, Caffe 2
GPU software: CUDA, cuDNN, NVIDIA drivers
Development tools: git, tmux, screen, vim, emacs, htop, valgrind, build-essential
Using Kaggle's beta API, you can interact with Competitions and Datasets to download data, make submissions, and more via the command line. 


Thursday, March 7, 2019

Google Test (gtest) on Debian

Here is the link to the project source https://github.com/google/googletest

Here is the code for installing Google Test (gtest)

Tuesday, January 22, 2019

Initial Steps for Python

Change python version system-wide

This page has a great explanation for switching between Python and Python3:
https://linuxconfig.org/how-to-change-from-default-to-alternative-python-version-on-debian-linux

Installing Jupyter

Since in Python 3, ConfigParser has been renamed to configparser I switched to Python using:
Then for installing and running a Notebook: