merged 2.4 into trunk
This commit is contained in:
Vadim Pisarevsky
@@ -3,7 +3,7 @@ Extremely randomized trees
|
||||
|
||||
Extremely randomized trees have been introduced by Pierre Geurts, Damien Ernst and Louis Wehenkel in the article "Extremely randomized trees", 2006 [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.65.7485&rep=rep1&type=pdf]. The algorithm of growing Extremely randomized trees is similar to :ref:`Random Trees` (Random Forest), but there are two differences:
|
||||
|
||||
#. Extremely randomized trees don't apply the bagging procedure to constract the training samples for each tree. The same input training set is used to train all trees.
|
||||
#. Extremely randomized trees don't apply the bagging procedure to construct a set of the training samples for each tree. The same input training set is used to train all trees.
|
||||
|
||||
#. Extremely randomized trees pick a node split very extremely (both a variable index and variable splitting value are chosen randomly), whereas Random Forest finds the best split (optimal one by variable index and variable splitting value) among random subset of variables.
|
||||
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
|
||||
.. _ML_Expectation Maximization:
|
||||
|
||||
|
||||
Expectation Maximization
|
||||
========================
|
||||
|
||||
@@ -85,87 +89,67 @@ already a good enough approximation).
|
||||
*
|
||||
Bilmes98 J. A. Bilmes. *A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models*. Technical Report TR-97-021, International Computer Science Institute and Computer Science Division, University of California at Berkeley, April 1998.
|
||||
|
||||
EM
|
||||
--
|
||||
.. ocv:class:: EM
|
||||
|
||||
CvEMParams
|
||||
----------
|
||||
.. ocv:class:: CvEMParams
|
||||
|
||||
Parameters of the EM algorithm. All parameters are public. You can initialize them by a constructor and then override some of them directly if you want.
|
||||
The class implements the EM algorithm as described in the beginning of this section. It is inherited from :ocv:class:`Algorithm`.
|
||||
|
||||
|
||||
EM::EM
|
||||
------
|
||||
The constructor of the class
|
||||
|
||||
CvEMParams::CvEMParams
|
||||
----------------------
|
||||
The constructors
|
||||
.. ocv:function:: EM::EM(int nclusters=EM::DEFAULT_NCLUSTERS, int covMatType=EM::COV_MAT_DIAGONAL, const TermCriteria& termCrit=TermCriteria(TermCriteria::COUNT+TermCriteria::EPS, EM::DEFAULT_MAX_ITERS, FLT_EPSILON) )
|
||||
|
||||
.. ocv:function:: CvEMParams::CvEMParams()
|
||||
|
||||
.. ocv:function:: CvEMParams::CvEMParams( int nclusters, int cov_mat_type=CvEM::COV_MAT_DIAGONAL, int start_step=CvEM::START_AUTO_STEP, CvTermCriteria term_crit=cvTermCriteria(CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, FLT_EPSILON), const CvMat* probs=0, const CvMat* weights=0, const CvMat* means=0, const CvMat** covs=0 )
|
||||
|
||||
:param nclusters: The number of mixture components in the gaussian mixture model. Some of EM implementation could determine the optimal number of mixtures within a specified value range, but that is not the case in ML yet.
|
||||
:param nclusters: The number of mixture components in the Gaussian mixture model. Default value of the parameter is ``EM::DEFAULT_NCLUSTERS=5``. Some of EM implementation could determine the optimal number of mixtures within a specified value range, but that is not the case in ML yet.
|
||||
|
||||
:param cov_mat_type: Constraint on covariance matrices which defines type of matrices. Possible values are:
|
||||
:param covMatType: Constraint on covariance matrices which defines type of matrices. Possible values are:
|
||||
|
||||
* **CvEM::COV_MAT_SPHERICAL** A scaled identity matrix :math:`\mu_k * I`. There is the only parameter :math:`\mu_k` to be estimated for earch matrix. The option may be used in special cases, when the constraint is relevant, or as a first step in the optimization (for example in case when the data is preprocessed with PCA). The results of such preliminary estimation may be passed again to the optimization procedure, this time with ``cov_mat_type=CvEM::COV_MAT_DIAGONAL``.
|
||||
* **EM::COV_MAT_SPHERICAL** A scaled identity matrix :math:`\mu_k * I`. There is the only parameter :math:`\mu_k` to be estimated for each matrix. The option may be used in special cases, when the constraint is relevant, or as a first step in the optimization (for example in case when the data is preprocessed with PCA). The results of such preliminary estimation may be passed again to the optimization procedure, this time with ``covMatType=EM::COV_MAT_DIAGONAL``.
|
||||
|
||||
* **CvEM::COV_MAT_DIAGONAL** A diagonal matrix with positive diagonal elements. The number of free parameters is ``d`` for each matrix. This is most commonly used option yielding good estimation results.
|
||||
* **EM::COV_MAT_DIAGONAL** A diagonal matrix with positive diagonal elements. The number of free parameters is ``d`` for each matrix. This is most commonly used option yielding good estimation results.
|
||||
|
||||
* **CvEM::COV_MAT_GENERIC** A symmetric positively defined matrix. The number of free parameters in each matrix is about :math:`d^2/2`. It is not recommended to use this option, unless there is pretty accurate initial estimation of the parameters and/or a huge number of training samples.
|
||||
* **EM::COV_MAT_GENERIC** A symmetric positively defined matrix. The number of free parameters in each matrix is about :math:`d^2/2`. It is not recommended to use this option, unless there is pretty accurate initial estimation of the parameters and/or a huge number of training samples.
|
||||
|
||||
:param termCrit: The termination criteria of the EM algorithm. The EM algorithm can be terminated by the number of iterations ``termCrit.maxCount`` (number of M-steps) or when relative change of likelihood logarithm is less than ``termCrit.epsilon``. Default maximum number of iterations is ``EM::DEFAULT_MAX_ITERS=100``.
|
||||
|
||||
:param start_step: The start step of the EM algorithm:
|
||||
EM::train
|
||||
---------
|
||||
Estimates the Gaussian mixture parameters from a samples set.
|
||||
|
||||
* **CvEM::START_E_STEP** Start with Expectation step. You need to provide means :math:`a_k` of mixture components to use this option. Optionally you can pass weights :math:`\pi_k` and covariance matrices :math:`S_k` of mixture components.
|
||||
* **CvEM::START_M_STEP** Start with Maximization step. You need to provide initial probabilities :math:`p_{i,k}` to use this option.
|
||||
* **CvEM::START_AUTO_STEP** Start with Expectation step. You need not provide any parameters because they will be estimated by the k-means algorithm.
|
||||
.. ocv:function:: bool EM::train(InputArray samples, OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())
|
||||
|
||||
:param term_crit: The termination criteria of the EM algorithm. The EM algorithm can be terminated by the number of iterations ``term_crit.max_iter`` (number of M-steps) or when relative change of likelihood logarithm is less than ``term_crit.epsilon``.
|
||||
.. ocv:function:: bool EM::trainE(InputArray samples, InputArray means0, InputArray covs0=noArray(), InputArray weights0=noArray(), OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())
|
||||
|
||||
.. ocv:function:: bool EM::trainM(InputArray samples, InputArray probs0, OutputArray logLikelihoods=noArray(), OutputArray labels=noArray(), OutputArray probs=noArray())
|
||||
|
||||
:param probs: Initial probabilities :math:`p_{i,k}` of sample :math:`i` to belong to mixture component :math:`k`. It is a floating-point matrix of :math:`nsamples \times nclusters` size. It is used and must be not NULL only when ``start_step=CvEM::START_M_STEP``.
|
||||
:param samples: Samples from which the Gaussian mixture model will be estimated. It should be a one-channel matrix, each row of which is a sample. If the matrix does not have ``CV_64F`` type it will be converted to the inner matrix of such type for the further computing.
|
||||
|
||||
:param means0: Initial means :math:`a_k` of mixture components. It is a one-channel matrix of :math:`nclusters \times dims` size. If the matrix does not have ``CV_64F`` type it will be converted to the inner matrix of such type for the further computing.
|
||||
|
||||
:param weights: Initial weights :math:`\pi_k` of mixture components. It is a floating-point vector with :math:`nclusters` elements. It is used (if not NULL) only when ``start_step=CvEM::START_E_STEP``.
|
||||
:param covs0: The vector of initial covariance matrices :math:`S_k` of mixture components. Each of covariance matrices is a one-channel matrix of :math:`dims \times dims` size. If the matrices do not have ``CV_64F`` type they will be converted to the inner matrices of such type for the further computing.
|
||||
|
||||
:param weights0: Initial weights :math:`\pi_k` of mixture components. It should be a one-channel floating-point matrix with :math:`1 \times nclusters` or :math:`nclusters \times 1` size.
|
||||
|
||||
:param probs0: Initial probabilities :math:`p_{i,k}` of sample :math:`i` to belong to mixture component :math:`k`. It is a one-channel floating-point matrix of :math:`nsamples \times nclusters` size.
|
||||
|
||||
:param means: Initial means :math:`a_k` of mixture components. It is a floating-point matrix of :math:`nclusters \times dims` size. It is used used and must be not NULL only when ``start_step=CvEM::START_E_STEP``.
|
||||
:param logLikelihoods: The optional output matrix that contains a likelihood logarithm value for each sample. It has :math:`nsamples \times 1` size and ``CV_64FC1`` type.
|
||||
|
||||
:param covs: Initial covariance matrices :math:`S_k` of mixture components. Each of covariance matrices is a valid square floating-point matrix of :math:`dims \times dims` size. It is used (if not NULL) only when ``start_step=CvEM::START_E_STEP``.
|
||||
:param labels: The optional output "class label" for each sample: :math:`\texttt{labels}_i=\texttt{arg max}_k(p_{i,k}), i=1..N` (indices of the most probable mixture component for each sample). It has :math:`nsamples \times 1` size and ``CV_32SC1`` type.
|
||||
|
||||
:param probs: The optional output matrix that contains posterior probabilities of each Gaussian mixture component given the each sample. It has :math:`nsamples \times nclusters` size and ``CV_64FC1`` type.
|
||||
|
||||
The default constructor represents a rough rule-of-the-thumb:
|
||||
Three versions of training method differ in the initialization of Gaussian mixture model parameters and start step:
|
||||
|
||||
::
|
||||
* **train** - Starts with Expectation step. Initial values of the model parameters will be estimated by the k-means algorithm.
|
||||
|
||||
CvEMParams() : nclusters(10), cov_mat_type(1/*CvEM::COV_MAT_DIAGONAL*/),
|
||||
start_step(0/*CvEM::START_AUTO_STEP*/), probs(0), weights(0), means(0), covs(0)
|
||||
{
|
||||
term_crit=cvTermCriteria( CV_TERMCRIT_ITER+CV_TERMCRIT_EPS, 100, FLT_EPSILON );
|
||||
}
|
||||
* **trainE** - Starts with Expectation step. You need to provide initial means :math:`a_k` of mixture components. Optionally you can pass initial weights :math:`\pi_k` and covariance matrices :math:`S_k` of mixture components.
|
||||
|
||||
* **trainM** - Starts with Maximization step. You need to provide initial probabilities :math:`p_{i,k}` to use this option.
|
||||
|
||||
With another constructor it is possible to override a variety of parameters from a single number of mixtures (the only essential problem-dependent parameter) to initial values for the mixture parameters.
|
||||
|
||||
|
||||
CvEM
|
||||
----
|
||||
.. ocv:class:: CvEM
|
||||
|
||||
The class implements the EM algorithm as described in the beginning of this section.
|
||||
|
||||
|
||||
CvEM::train
|
||||
-----------
|
||||
Estimates the Gaussian mixture parameters from a sample set.
|
||||
|
||||
.. ocv:function:: void CvEM::train( const Mat& samples, const Mat& sample_idx=Mat(), CvEMParams params=CvEMParams(), Mat* labels=0 )
|
||||
|
||||
.. ocv:function:: bool CvEM::train( const CvMat* samples, const CvMat* sampleIdx=0, CvEMParams params=CvEMParams(), CvMat* labels=0 )
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.train(samples[, sampleIdx[, params]]) -> retval, labels
|
||||
|
||||
:param samples: Samples from which the Gaussian mixture model will be estimated.
|
||||
|
||||
:param sample_idx: Mask of samples to use. All samples are used by default.
|
||||
|
||||
:param params: Parameters of the EM algorithm.
|
||||
|
||||
:param labels: The optional output "class label" for each sample: :math:`\texttt{labels}_i=\texttt{arg max}_k(p_{i,k}), i=1..N` (indices of the most probable mixture component for each sample).
|
||||
The methods return ``true`` if the Gaussian mixture model was trained successfully, otherwise it returns ``false``.
|
||||
|
||||
Unlike many of the ML models, EM is an unsupervised learning algorithm and it does not take responses (class labels or function values) as input. Instead, it computes the
|
||||
*Maximum Likelihood Estimate* of the Gaussian mixture parameters from an input sample set, stores all the parameters inside the structure:
|
||||
@@ -178,121 +162,37 @@ Unlike many of the ML models, EM is an unsupervised learning algorithm and it do
|
||||
The trained model can be used further for prediction, just like any other classifier. The trained model is similar to the
|
||||
:ocv:class:`CvNormalBayesClassifier`.
|
||||
|
||||
For an example of clustering random samples of the multi-Gaussian distribution using EM, see ``em.cpp`` sample in the OpenCV distribution.
|
||||
EM::predict
|
||||
-----------
|
||||
Returns a likelihood logarithm value and an index of the most probable mixture component for the given sample.
|
||||
|
||||
.. ocv:function:: Vec2d predict(InputArray sample, OutputArray probs=noArray()) const
|
||||
|
||||
:param sample: A sample for classification. It should be a one-channel matrix of :math:`1 \times dims` or :math:`dims \times 1` size.
|
||||
|
||||
CvEM::predict
|
||||
-------------
|
||||
Returns a mixture component index of a sample.
|
||||
:param probs: Optional output matrix that contains posterior probabilities of each component given the sample. It has :math:`1 \times nclusters` size and ``CV_64FC1`` type.
|
||||
|
||||
.. ocv:function:: float CvEM::predict( const Mat& sample, Mat* probs=0 ) const
|
||||
The method returns a two-element ``double`` vector. Zero element is a likelihood logarithm value for the sample. First element is an index of the most probable mixture component for the given sample.
|
||||
|
||||
.. ocv:function:: float CvEM::predict( const CvMat* sample, CvMat* probs ) const
|
||||
CvEM::isTrained
|
||||
---------------
|
||||
Returns ``true`` if the Gaussian mixture model was trained.
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.predict(sample) -> retval, probs
|
||||
.. ocv:function:: bool EM::isTrained() const
|
||||
|
||||
:param sample: A sample for classification.
|
||||
|
||||
:param probs: If it is not null then the method will write posterior probabilities of each component given the sample data to this parameter.
|
||||
|
||||
|
||||
CvEM::getNClusters
|
||||
------------------
|
||||
Returns the number of mixture components :math:`M` in the gaussian mixture model.
|
||||
|
||||
.. ocv:function:: int CvEM::getNClusters() const
|
||||
|
||||
.. ocv:function:: int CvEM::get_nclusters() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getNClusters() -> retval
|
||||
|
||||
|
||||
CvEM::getMeans
|
||||
------------------
|
||||
Returns mixture means :math:`a_k`.
|
||||
|
||||
.. ocv:function:: Mat CvEM::getMeans() const
|
||||
|
||||
.. ocv:function:: const CvMat* CvEM::get_means() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getMeans() -> means
|
||||
|
||||
|
||||
CvEM::getCovs
|
||||
-------------
|
||||
Returns mixture covariance matrices :math:`S_k`.
|
||||
|
||||
.. ocv:function:: void CvEM::getCovs(std::vector<cv::Mat>& covs) const
|
||||
|
||||
.. ocv:function:: const CvMat** CvEM::get_covs() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getCovs([covs]) -> covs
|
||||
|
||||
|
||||
CvEM::getWeights
|
||||
----------------
|
||||
Returns mixture weights :math:`\pi_k`.
|
||||
|
||||
.. ocv:function:: Mat CvEM::getWeights() const
|
||||
|
||||
.. ocv:function:: const CvMat* CvEM::get_weights() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getWeights() -> weights
|
||||
|
||||
|
||||
CvEM::getProbs
|
||||
--------------
|
||||
Returns vectors of probabilities for each training sample.
|
||||
|
||||
.. ocv:function:: Mat CvEM::getProbs() const
|
||||
|
||||
.. ocv:function:: const CvMat* CvEM::get_probs() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getProbs() -> probs
|
||||
|
||||
For each training sample :math:`i` (that have been passed to the constructor or to :ocv:func:`CvEM::train`) returns probabilities :math:`p_{i,k}` to belong to a mixture component :math:`k`.
|
||||
|
||||
|
||||
CvEM::getLikelihood
|
||||
EM::read, EM::write
|
||||
-------------------
|
||||
Returns logarithm of likelihood.
|
||||
|
||||
.. ocv:function:: double CvEM::getLikelihood() const
|
||||
|
||||
.. ocv:function:: double CvEM::get_log_likelihood() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getLikelihood() -> likelihood
|
||||
|
||||
|
||||
CvEM::getLikelihoodDelta
|
||||
------------------------
|
||||
Returns difference between logarithm of likelihood on the last iteration and logarithm of likelihood on the previous iteration.
|
||||
|
||||
.. ocv:function:: double CvEM::getLikelihoodDelta() const
|
||||
|
||||
.. ocv:function:: double CvEM::get_log_likelihood_delta() const
|
||||
|
||||
.. ocv:pyfunction:: cv2.EM.getLikelihoodDelta() -> likelihood delta
|
||||
|
||||
CvEM::write_params
|
||||
------------------
|
||||
Writes used parameters of the EM algorithm to a file storage.
|
||||
|
||||
.. ocv:function:: void CvEM::write_params( CvFileStorage* fs ) const
|
||||
|
||||
:param fs: A file storage where parameters will be written.
|
||||
|
||||
|
||||
CvEM::read_params
|
||||
-----------------
|
||||
Reads parameters of the EM algorithm.
|
||||
|
||||
.. ocv:function:: void CvEM::read_params( CvFileStorage* fs, CvFileNode* node )
|
||||
|
||||
:param fs: A file storage with parameters of the EM algorithm.
|
||||
|
||||
:param node: The parent map. If it is NULL, the function searches a node with parameters in all the top-level nodes (streams), starting with the first one.
|
||||
|
||||
The function reads EM parameters from the specified file storage node. For example of clustering random samples of multi-Gaussian distribution using EM see em.cpp sample in OpenCV distribution.
|
||||
See :ocv:func:`Algorithm::read` and :ocv:func:`Algorithm::write`.
|
||||
|
||||
EM::get, EM::set
|
||||
----------------
|
||||
See :ocv:func:`Algorithm::get` and :ocv:func:`Algorithm::set`. The following parameters are available:
|
||||
|
||||
* ``"nclusters"``
|
||||
* ``"covMatType"``
|
||||
* ``"maxIters"``
|
||||
* ``"epsilon"``
|
||||
* ``"weights"`` *(read-only)*
|
||||
* ``"means"`` *(read-only)*
|
||||
* ``"covs"`` *(read-only)*
|
||||
..
|
||||
|
||||
@@ -34,7 +34,7 @@ Class for loading the data from a ``.csv`` file.
|
||||
void mix_train_and_test_idx();
|
||||
|
||||
const CvMat* get_var_idx();
|
||||
void chahge_var_idx( int vi, bool state );
|
||||
void change_var_idx( int vi, bool state );
|
||||
|
||||
const CvMat* get_var_types();
|
||||
void set_var_types( const char* str );
|
||||
@@ -158,11 +158,11 @@ The method returns the indices of variables (columns) used in the ``values`` mat
|
||||
|
||||
It returns ``0`` if the used subset is not set. It throws an exception if the data has not been loaded from the file yet. Returned matrix is a single-row matrix of the type ``CV_32SC1``. Its column count is equal to the size of the used variable subset.
|
||||
|
||||
CvMLData::chahge_var_idx
|
||||
CvMLData::change_var_idx
|
||||
------------------------
|
||||
Enables or disables particular variable in the loaded data
|
||||
|
||||
.. ocv:function:: void CvMLData::chahge_var_idx( int vi, bool state )
|
||||
.. ocv:function:: void CvMLData::change_var_idx( int vi, bool state )
|
||||
|
||||
By default, after reading the data set all variables in the ``values`` matrix (see :ocv:func:`CvMLData::get_values`) are used. But you may want to use only a subset of variables and include/exclude (depending on ``state`` value) a variable with the ``vi`` index from the used subset. If the data has not been loaded from the file yet, an exception is thrown.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user