The current paradigm for the analysis of low bit rate image transform coding is
based on non-linear signal approximation. Images are considered to be ``highly''
non-Gaussian processes that are not suited for the familiar linear approach,
i.e. a fixed coefficient bit allocation based on the signal statistics. Instead,
the non-linear approximation tries to find the ``best'' basis for a given
\emph{individual} signal and a given rate. In many practical coders (wavelet,
JPEG), this ``best'' basis consists in specifying the indices of a small number
of quantized coefficients, which describe the signal with the desired accuracy.
This works well with wavelet transforms of piecewise regular functions, since
there will be only a few non-zero coefficients, mostly around the signal
singularities. The key aspect is a rate trade-off between the \emph{lossless}
code for the coefficient positions and the \emph{lossy} code for the values of
those coefficients. By assuming that the signal belongs to certain functional
spaces, one can find the ``optimal'' rate trade-off and from this an approximate
operational rate distortion curve. In general this is \emph{not} the
information-theoretic rate distortion function. Therefore it gives no
information on how far from the theoretical optimum such algorithms operate.
Our goal is to find an alternative, i.e. information-theoretic, framework for
the $R(D)$ analysis of such non-linear approximation schemes. As a tool we
introduce the \emph{spike process}, which captures the idea of a single isolated
non-zero coefficient. This can be extended to multiple spikes by independent
superposition or, more efficiently, by joint description. We provide a
definition of spike processes and then investigate the $R(D)$ behavior for
Hamming distortion, which corresponds to \emph{lossy} position coding. We are
trying to extend these results to Gaussian-distributed spikes and squared error
measure, but so far we were only able to derive upper bounds.