Abstract

The discretization of parametric or stochastic PDEs and ODEs describing complex physical systems leads to high-dimensional problems, the solution of which requires a massive computational effort. Data analysis of large high-dimensional datasets or prediction tasks in spatio-temporal statistics may also require a huge computational effort and storage costs.

In these and also in many other cases the data can be conceptually arranged in the format of a tensor of high degree, and stored in some truncated or lossy compressed format. We look at some common post-processing tasks which are too time and storage consuming in the uncompressed data format and not obvious in the compressed format, as such huge data sets can not be stored in their entirety, and the value of an element is not readily accessible through simple look-up.

The tasks we consider are finding the location of maximum or minimum, or finding all elements in some interval (level sets), or the number of elements with a value in such a level set, the probability of an element being in a particular level set, and the mean and variance of the total collection.

The algorithms to be described are fixed point iterations of particular point-wise functions of the data, which will then exhibit the desired result. We allow the actual computational representation to be a lossy compression, and we allow the algebra operations to be performed in an approximate fashion, to maintain a high compression level. One such example format which is addressed explicitly and described in some detail is the representation of the data as a tensor with compression in the form of a low-rank representation.