Tags: rinor/gorgonia
Tags
V0.9.0 (gorgonia#195) Ongoing notes: * **CUDA**: Better CUDA support (IN PROGRESS) * ~ColMajor used by default if engine is CUDA.~ (ColMajor is supported, but defaults to using RowMajor for all the major cuBLAS versions. Careful reasoning of the parameters obviates the need for ColMajor by default, which causes more headaches. It is still supported) * Transposition will be automatically done when performing transports back to CPU. * cudnn operations supported (IN PROGRESS) (note: these are the ones I use more often hence gets bigger attention): * [x] Conv2d * [x] Dropout * [x] Maxpool2d * [x] BatchNorm * [x] Rectify * Other CUDA related optimizations * [x] full cuBLAS support * **New Ops**: * BatchNorm * InvSqrt * CUDA enabled ops in `ops/nn` (preview for how things will start to look in v0.10.0) * **New Features**: * Limited shape inference. Working towards a calculus for shapes (first raised in gorgonia#96 and gorgonia#97). * **Optimizations**: * Optimizations of basic ops to use engine functions if available, otherwise, fall back to using `Apply`, which adds a penalty from repeatedly calling functions. * Faster VMs (1 of 2 VMs): ~greedy goroutines grabs gigs from a priority queue. This causes faster execution of code in general.~ (this is moved to a future version of 0.9.xx): ``` benchmark old ns/op new ns/op delta BenchmarkTapeMachineExecution-8 3129074510 2695304022 -13.86% benchmark old allocs new allocs delta BenchmarkTapeMachineExecution-8 25745 25122 -2.42% benchmark old bytes new bytes delta BenchmarkTapeMachineExecution-8 4804578705 4803784111 -0.02% ```` * **Code generation**: some exported API is now auto generated * **New Solver** : @ynqa added the Momentum solver. * **Breaking API**: `Solver` now take a slice of `ValueGrad` instead of `Nodes`. `ValueGrad` is an interface, of which a `*Node` fulfils. An additional utility function `NodesToValueGrads` has been added to aid with refactoring. This was done for two reasons: * ~The support for BatchNorm operation, which is a verily impure and highly stateful function. The BatchNorm Op has internal states that need to have their gradients updated as well. But the internal state of BatchNorm isn't really part of the expression graph, and really it shouldn't be.~ Turns out there was a better API for `BatchNorm`. * In the next version, v0.10.0. We aim to do [better package organization](gorgonia#91) for managability. With this API breaking change, the solver now is less dependent on the other parts of Gorgonia and can be easily separated. * **Breaking Semantics**: A `gorgonia.VM` now implements `io.Closer`. It should be treated as a resource as well as a computation device - the VM must be `Close()`d in order for the resources acquired by the VM to actually be released. Turns out, automatic resource management is too difficult. Who'd thunk that?
Merge pull request gorgonia#202 from trigun117/master Сorrected typos, fixed ineffassign, gofmt, go_vet
Merge pull request gorgonia#200 from gorgonia/variousbugfixes Fixed a tiny bug in UnbindNonInputs.
Merge pull request gorgonia#192 from siquus/newSolversAndTests New Barzilai-Borwein solver and solver tests
Merge pull request gorgonia#190 from gorgonia/189fix Updated convnet example
Merge pull request gorgonia#179 from bradleyjkemp/patch-1 Fix table formatting
Merge pull request gorgonia#159 from chewxy/convolution Convolution
Merge pull request gorgonia#162 from chewxy/v0.7.3-2 v0.7.3 Fixes gorgonia#160 but the stacked autoencoder example will still be re-written for 0.8.0
PreviousNext