Michael Joswig, Klaus-Robert Müller
Marek Kaluba (TU)
01.01.2019 – 31.12.2020
Finding a meaningful data representation that captures the relevant information for some task or problem lies at the very core of any data-driven discipline such as statistics or machine learning. In contrast to handcrafted data features, representation learning aims to learn such representations from the data itself. Convexity is a very natural and fundamental representation property in machine learning in general. The separation of classes via multiple hyperplanes or half-spaces in supervised classification for instance, such as in Support Vector Machines (SVMs) in the Reproducing Kernel Hilbert Space (RKHS) via the hinge loss or in deep neural networks (DNNs) in the output space via the cross-entropy loss, forms polyhedra that are convex and generally unbounded. Moreover, many well-known unsupervised learning methods make explicit convexity assumptions on the support of the data representation, such as Voronoi cells in k-means clustering or ellipsoids in Gaussian Mixture Models (GMMs). Yet it is worth noting that convexity is also implicit in Gaussian prior assumptions in popular deep generative models such as Variational Autoencoders (VAEs), or Generative Adversarial Networks (GANs). Finally, one-class classification methods for support estimation such as the One-Class SVM or (Deep) Support Vector Data Description (SVDD) also rely on convex assumptions on the data representation via maximum-margin hyperplane and minimum enclosing hypersphere descriptions, respectively.
Although a convex representation of classes and clusters is such a natural and commonly desired property, measuring convexity in a robust and efficient way in high-dimensional spaces is a challenging and fairly non-trivial problem. In this project we propose a scalable and robust method called Random Polytope Descriptor (RPD) for evaluating convexity in representation learning. Our method is based on concepts from convex geometry and constructs a polytope (a piecewise-linear, bounded convex body) around the training data in representation space. Since polytopes by themselves may also suffer from a combinatorial explosion, we construct our descriptor from random convex polytopes instead which also enhances its robustness. Finally, using the proximities to such polytopes allows us to judge the geometric disentanglement of representations, i.e. how well classes and clusters are separated into convex bodies. Our main contributions are the following:
The project combines state-of-the-art Machine learning tools with established tools in computational polyhedral geometry using the novel Julia programming language.
[Kaluba2020] M. Kaluba, B. Lorenz, and S. Timme. Polymake.jl a new interface to `polymake`. Accepted to ICMS, 2020.
[Kaluba2020a] M. Kaluba, L. Ruffs, M. Joswig. Geometric Disentanglement by Random Convex Polytopes. In preparation.
Please insert any kind of pictures (photos, diagramms, simulations, graphics) related to the project in the above right field (Image with Text), by choosing the green plus image on top of the text editor. (You will be directed to the media library where you can add new files.)
(We need pictures for a lot of purposes in different contexts, like posters, scientific reports, flyers, website,…
Please upload pictures that might be just nice to look at, illustrate, explain or summarize your work.)
As Title in the above form please add a copyright.
And please give a short description of the picture and the context in the above textbox.
Don’t forget to press the “Save changes” button at the bottom of the box.
If you want to add more pictures, please use the “clone”-button at the right top of the above grey box.