Representing examples in a way that is compatible with the underlying
assifier can greatly enhance the performance of a learning system. In this
per we investigate scalable techniques for inducing discriminative features
taking advantage of simple second order structure in the data. We focus on
lticlass classification and show that features extracted from the generalized
genvectors of the class conditional second moments lead to classifiers with
cellent empirical performance. Moreover, these features have attractive
eoretical properties, such as inducing representations that are invariant to
near transformations of the input. We evaluate classifiers built from these
atures on three different tasks, obtaining state of the art results.