Implementing LeNet for MNist in Wolfram Mathematica

In the previous post in our series on the Bregman algorithm we discussed how to solve convex optimization problems. In this post we want to give reference to some variations and extensions of the Bregman algorithm.

In a previous post we discussed how to solve constrained optimization problems by using the Bregman algorithm. Here we want to extend the approach unconstrained problems.
Let’s start simple. Assume we want to minimize a convex and smooth function $f\colon\mathbb{R}^{n}\to\mathbb{R}$.

In a previous post we discussed how to find a common point in a family of convex sets by using the Bregman algorithm. Actually the algorithm is capable of more. We can use it to solve constrained optimization problems.

In the 1960s Lev Meerovich Bregman developed an optimization algorithm [1] which became rather popular beginning of 2000s. It’s not my intention to present the proofs for all the algorithmic finesse, but rather the general ideas why it is so appealing.

Let us consider a microphone array comprising $n$ microphones at known locations (see figure above). These microphones register the sound that is emitted by a number of sources with unknown locations.

Lossy image compression methods based on partial differential equations have received much attention in recent years. They may yield high-quality results but rely on the computationally expensive task of finding an optimal selection of data. For the …

We have investigated high performing optimization algorithms and matrix differential calculus technique in the context of Photometric Stereo and presented the results at the BMVC 2016
Source Code A github repository with the code is maintained by Yvain Quéau.

I’ve developed optimization algorithms for variational optical flow models based on the split Bregman algorithm in my Master thesis. A follow-up investigation on the necessity of certain intermediate filtering steps was published at the EMMCVPR 2011.

Oral presentation