I hate inconsistency. It drives me crazy. I’m reviewing basic operations in PyTorch and I think I’ve discovered a major cause of the confusion I often feel when reading pytorch related code.
Given a matrix (or 2D tensor) m x n, one can select columns. If you select 2 or more columns you get an m x no. of cols tensor as a result. Two dimensions. But if you select only one column, you don’t. You get a one dimensional m vector as a result. The second dimension has been dropped!! No wonder there are so many squeeze and unsqueeze and reshape calls in pytorch code. To my simple mind if you select a column you should get a column, but no. Well, maybe now I can reduce my confusion when reading this sort of code somewhat. I just wish they hadn’t done that. I’m probably just proving what an idiot I am.