Hello. Thanks for the great work. I know from the paper that the last layer is supposed to be computed as an affine transformation W*x + b, but it's not being done that way in the code (apparently unintentionally). In the decode() function you are checking if the weights are at the last layer kind=self._nl_type if ind!=self._last else 'none', however we know that number of weights matrices = number of layers - 1, and since ind corresponds to the index of the current weight matrix, and since indexing starts at 0, then self._last should be set as self._last = len(layer_sizes) - 2 and not as self._last = len(layer_sizes) - 1.

A good test case is to run the test units AutoEncoder with nl_type set to sigmoid, it will perform badly because the last layer is computing values in the range of [0,1] which does not correspond to the range of ratings. After applying the fix it will perform way better.

That actually explains why sigmoid and tanh activation functions did not perform well as described on the paper.