Feature Selection and Classification by a Modified Model with Latent Structure

The general classification or discrimination problem is about guessing or predicting the uknown nature of the object of the interest. The uknown nature of the object is called a class which is denoted by ω and takes values in a finite set Ω = {ω 1, ω 2,..., ω C}. An object is described by a D—dimensional vector of features x = (x 1, x 2,..., x D )T ∈ X ⊂ R D . We wish to build a rule r(x): R D → Ω, which represents one’s guess of a given x. The mapping is called classifier. Considering the statistical approach to the problem of classification the objects are supposed to occur randomly according to some true class conditional pdfs p ★(x|ω) and the respective a priori probabilities P ★(ω). Vector x can be then optimally classified using the Bayes decision rule r(x), r : X → Ω: $$if P*({\omega _i}|x) \geqslant P*({\omega _j}|x) for all j = 1,2,...,C$$ (1) then the object will be classified as belonging to class ω i , i.e. r(x) = ω i . Here P ★(ω j |x) are the posterior class probabilities $$P*(\omega |x) = \frac{{P*(\omega )p*(x|\omega )}}{{\sum\nolimits_{\omega \in \Omega } {P*(\omega )p*(x|\omega )} }}$$ (2) normalised so that \(\sum\nolimits_{\omega \in \Omega } {P*(\omega |x) = 1}\).