generic-library/opencv
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Rename parameters lambda, gamma0 and c.

Browse Source
This commit is contained in:
Marina Noskova 2016-02-24 13:22:07 +03:00
parent 02cd8cf039
commit f3c4a6aab8
5 changed files with 74 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
modules/ml/include/opencv2/ml.hpp
View File

@ -1511,9 +1511,9 @@ The gradient descent show amazing performance for large-scale problems, reducing
The classifier has 5 parameters. These are
- model type,
- margin type,
- \f$\lambda\f$ (strength of restrictions on outliers),
- \f$\gamma_0\f$ (initial step size),
- \f$c\f$ (power coefficient for decreasing of step size),
- marginRegularization (\f$\lambda\f$),
- initialStepSize (\f$\gamma_0\f$),
- stepDecreasingPower (\f$c\f$),
- and termination criteria.
The model type may have one of the following values: \ref SGD and \ref ASGD.
@ -1538,27 +1538,27 @@ The margin type may have one of the following values: \ref SOFT_MARGIN or \ref H
- In the general case (if you know nothing about linear separability of your sets), use SOFT_MARGIN.
The other parameters may be described as follows:
- \f$\lambda\f$ parameter is responsible for weights decreasing at each step and for the strength of restrictions on outliers
- marginRegularization parameter is responsible for weights decreasing at each step and for the strength of restrictions on outliers
(the less the parameter, the less probability that an outlier will be ignored).
Recommended value for SGD model is 0.0001, for ASGD model is 0.00001.
- \f$\gamma_0\f$ parameter is the initial value for the step size \f$\gamma(t)\f$.
- initialStepSize parameter is the initial value for the step size \f$\gamma(t)\f$.
You will have to find the best \f$\gamma_0\f$ for your problem.
- \f$c\f$ is the power parameter for \f$\gamma(t)\f$ decreasing by the formula, mentioned above.
- stepDecreasingPower is the power parameter for \f$\gamma(t)\f$ decreasing by the formula, mentioned above.
Recommended value for SGD model is 1, for ASGD model is 0.75.
- Termination criteria can be TermCriteria::COUNT, TermCriteria::EPS or TermCriteria::COUNT + TermCriteria::EPS.
You will have to find the best termination criteria for your problem.
Note that the parameters \f$\lambda\f$, \f$\gamma_0\f$, and \f$c\f$ should be positive.
Note that the parameters marginRegularization, initialStepSize, and stepDecreasingPower should be positive.
To use SVMSGD algorithm do as follows:
- first, create the SVMSGD object.
- then set parameters (model type, margin type, \f$\lambda\f$, \f$\gamma_0\f$, \f$c\f$) using the functions
setSvmsgdType(), setMarginType(), setLambda(), setGamma0(), and setC(), or the function setOptimalParameters().
- then set parameters (model type, margin type, marginRegularization, initialStepSize, stepDecreasingPower) using the functions
setSvmsgdType(), setMarginType(), setMarginRegularization(), setInitialStepSize(), and setStepDecreasingPower(), or the function setOptimalParameters().
- then the SVM model can be trained using the train features and the correspondent labels by the method train().
@ -1618,16 +1618,16 @@ public:
/** @brief Function sets optimal parameters values for chosen SVM SGD model.
* If chosen type is ASGD, function sets the following values for parameters of model:
* \f$\lambda = 0.00001\f$;
* \f$\gamma_0 = 0.05\f$;
* \f$c = 0.75\f$;
* marginRegularization = 0.00001;
* initialStepSize = 0.05;
* stepDecreasingPower = 0.75;
* termCrit.maxCount = 100000;
* termCrit.epsilon = 0.00001;
*
* If SGD:
* \f$\lambda = 0.0001\f$;
* \f$\gamma_0 = 0.05\f$;
* \f$c = 1\f$;
* marginRegularization = 0.0001;
* initialStepSize = 0.05;
* stepDecreasingPower = 1;
* termCrit.maxCount = 100000;
* termCrit.epsilon = 0.00001;
* @param svmsgdType is the type of SVMSGD classifier. Legal values are SvmsgdType::SGD and SvmsgdType::ASGD.
@ -1650,23 +1650,23 @@ public:
CV_WRAP virtual void setMarginType(int marginType) = 0;
/** @brief Parameter \f$\lambda\f$ of a %SVMSGD optimization problem. Default value is 0. */
/** @see setLambda */
CV_WRAP virtual float getLambda() const = 0;
/** @copybrief getLambda @see getLambda */
CV_WRAP virtual void setLambda(float lambda) = 0;
/** @brief Parameter marginRegularization of a %SVMSGD optimization problem. Default value is 0. */
/** @see setMarginRegularization */
CV_WRAP virtual float getMarginRegularization() const = 0;
/** @copybrief getMarginRegularization @see getMarginRegularization */
CV_WRAP virtual void setMarginRegularization(float marginRegularization) = 0;
/** @brief Parameter \f$\gamma_0\f$ of a %SVMSGD optimization problem. Default value is 0. */
/** @see setGamma0 */
CV_WRAP virtual float getGamma0() const = 0;
/** @copybrief getGamma0 @see getGamma0 */
CV_WRAP virtual void setGamma0(float gamma0) = 0;
/** @see setInitialStepSize */
CV_WRAP virtual float getInitialStepSize() const = 0;
/** @copybrief getInitialStepSize @see getInitialStepSize */
CV_WRAP virtual void setInitialStepSize(float InitialStepSize) = 0;
/** @brief Parameter \f$c\f$ of a %SVMSGD optimization problem. Default value is 0. */
/** @see setC */
CV_WRAP virtual float getC() const = 0;
/** @copybrief getC @see getC */
CV_WRAP virtual void setC(float c) = 0;
/** @brief Parameter stepDecreasingPower of a %SVMSGD optimization problem. Default value is 0. */
/** @see setStepDecreasingPower */
CV_WRAP virtual float getStepDecreasingPower() const = 0;
/** @copybrief getStepDecreasingPower @see getStepDecreasingPower */
CV_WRAP virtual void setStepDecreasingPower(float stepDecreasingPower) = 0;
/** @brief Termination criteria of the training algorithm.
You can specify the maximum number of iterations (maxCount) and/or how much the error could

81
modules/ml/src/svmsgd.cpp
View File

@ -91,13 +91,13 @@ public:
CV_IMPL_PROPERTY(int, SvmsgdType, params.svmsgdType)
CV_IMPL_PROPERTY(int, MarginType, params.marginType)
CV_IMPL_PROPERTY(float, Lambda, params.lambda)
CV_IMPL_PROPERTY(float, Gamma0, params.gamma0)
CV_IMPL_PROPERTY(float, C, params.c)
CV_IMPL_PROPERTY(float, MarginRegularization, params.marginRegularization)
CV_IMPL_PROPERTY(float, InitialStepSize, params.initialStepSize)
CV_IMPL_PROPERTY(float, StepDecreasingPower, params.stepDecreasingPower)
CV_IMPL_PROPERTY_S(cv::TermCriteria, TermCriteria, params.termCrit)
private:
void updateWeights(InputArray sample, bool isPositive, float gamma, Mat &weights);
void updateWeights(InputArray sample, bool isPositive, float stepSize, Mat &weights);
std::pair<bool,bool> areClassesEmpty(Mat responses);
@ -122,9 +122,9 @@ private:
// Parameters for learning
struct SVMSGDParams
{
float lambda; //regularization
float gamma0; //learning rate
float c;
float marginRegularization;
float initialStepSize;
float stepDecreasingPower;
TermCriteria termCrit;
int svmsgdType;
int marginType;
@ -166,8 +166,7 @@ void SVMSGDImpl::normalizeSamples(Mat &samples, Mat &average, float &multiplier)
average = Mat(1, featuresCount, samples.type());
for (int featureIndex = 0; featureIndex < featuresCount; featureIndex++)
{
Scalar scalAverage = mean(samples.col(featureIndex));
average.at<float>(featureIndex) = static_cast<float>(scalAverage[0]);
average.at<float>(featureIndex) = static_cast<float>(mean(samples.col(featureIndex))[0]);
}
for (int sampleIndex = 0; sampleIndex < samplesCount; sampleIndex++)
@ -193,7 +192,7 @@ void SVMSGDImpl::makeExtendedTrainSamples(const Mat &trainSamples, Mat &extended
cv::hconcat(normalizedTrainSamples, onesCol, extendedTrainSamples);
}
void SVMSGDImpl::updateWeights(InputArray _sample, bool firstClass, float gamma, Mat& weights)
void SVMSGDImpl::updateWeights(InputArray _sample, bool firstClass, float stepSize, Mat& weights)
{
Mat sample = _sample.getMat();
@ -202,18 +201,18 @@ void SVMSGDImpl::updateWeights(InputArray _sample, bool firstClass, float gamma,
if ( sample.dot(weights) * response > 1)
{
// Not a support vector, only apply weight decay
weights *= (1.f - gamma * params.lambda);
weights *= (1.f - stepSize * params.marginRegularization);
}
else
{
// It's a support vector, add it to the weights
weights -= (gamma * params.lambda) * weights - (gamma * response) * sample;
weights -= (stepSize * params.marginRegularization) * weights - (stepSize * response) * sample;
}
}
float SVMSGDImpl::calcShift(InputArray _samples, InputArray _responses) const
{
float distanceToClasses[2] = { std::numeric_limits<float>::max(), std::numeric_limits<float>::max() };
float margin[2] = { std::numeric_limits<float>::max(), std::numeric_limits<float>::max() };
Mat trainSamples = _samples.getMat();
int trainSamplesCount = trainSamples.rows;
@ -225,18 +224,18 @@ float SVMSGDImpl::calcShift(InputArray _samples, InputArray _responses) const
Mat currentSample = trainSamples.row(samplesIndex);
float dotProduct = static_cast<float>(currentSample.dot(weights_));
bool firstClass = isPositive(trainResponses.at<float>(samplesIndex));
int index = firstClass ? 0 : 1;
float signToMul = firstClass ? 1.f : -1.f;
float curDistance = dotProduct * signToMul;
bool positive = isPositive(trainResponses.at<float>(samplesIndex));
int index = positive ? 0 : 1;
float signToMul = positive ? 1.f : -1.f;
float curMargin = dotProduct * signToMul;
if (curDistance < distanceToClasses[index])
if (curMargin < margin[index])
{
distanceToClasses[index] = curDistance;
margin[index] = curMargin;
}
}
return -(distanceToClasses[0] - distanceToClasses[1]) / 2.f;
return -(margin[0] - margin[1]) / 2.f;
}
bool SVMSGDImpl::train(const Ptr<TrainData>& data, int)
@ -292,9 +291,9 @@ bool SVMSGDImpl::train(const Ptr<TrainData>& data, int)
Mat currentSample = extendedTrainSamples.row(randomNumber);
float gamma = params.gamma0 * std::pow((1 + params.lambda * params.gamma0 * (float)iter), (-params.c)); //update gamma
float stepSize = params.initialStepSize * std::pow((1 + params.marginRegularization * params.initialStepSize * (float)iter), (-params.stepDecreasingPower)); //update stepSize
updateWeights( currentSample, isPositive(trainResponses.at<float>(randomNumber)), gamma, extendedWeights );
updateWeights( currentSample, isPositive(trainResponses.at<float>(randomNumber)), stepSize, extendedWeights );
//average weights (only for ASGD model)
if (params.svmsgdType == ASGD)
@ -370,7 +369,7 @@ bool SVMSGDImpl::isClassifier() const
&&
(params.marginType == SOFT_MARGIN || params.marginType == HARD_MARGIN)
&&
(params.lambda > 0) && (params.gamma0 > 0) && (params.c >= 0);
(params.marginRegularization > 0) && (params.initialStepSize > 0) && (params.stepDecreasingPower >= 0);
}
bool SVMSGDImpl::isTrained() const
@ -423,9 +422,9 @@ void SVMSGDImpl::writeParams( FileStorage& fs ) const
fs << "marginType" << marginTypeStr;
fs << "lambda" << params.lambda;
fs << "gamma0" << params.gamma0;
fs << "c" << params.c;
fs << "marginRegularization" << params.marginRegularization;
fs << "initialStepSize" << params.initialStepSize;
fs << "stepDecreasingPower" << params.stepDecreasingPower;
fs << "term_criteria" << "{:";
if( params.termCrit.type & TermCriteria::EPS )
@ -467,14 +466,14 @@ void SVMSGDImpl::readParams( const FileNode& fn )
params.marginType = marginType;
CV_Assert ( fn["lambda"].isReal() );
params.lambda = (float)fn["lambda"];
CV_Assert ( fn["marginRegularization"].isReal() );
params.marginRegularization = (float)fn["marginRegularization"];
CV_Assert ( fn["gamma0"].isReal() );
params.gamma0 = (float)fn["gamma0"];
CV_Assert ( fn["initialStepSize"].isReal() );
params.initialStepSize = (float)fn["initialStepSize"];
CV_Assert ( fn["c"].isReal() );
params.c = (float)fn["c"];
CV_Assert ( fn["stepDecreasingPower"].isReal() );
params.stepDecreasingPower = (float)fn["stepDecreasingPower"];
FileNode tcnode = fn["term_criteria"];
if( !tcnode.empty() )
@ -504,9 +503,9 @@ SVMSGDImpl::SVMSGDImpl()
params.marginType = -1;
// Parameters for learning
params.lambda = 0; // regularization
params.gamma0 = 0; // learning rate (ideally should be large at beginning and decay each iteration)
params.c = 0;
params.marginRegularization = 0; // regularization
params.initialStepSize = 0; // learning rate (ideally should be large at beginning and decay each iteration)
params.stepDecreasingPower = 0;
TermCriteria _termCrit(TermCriteria::COUNT + TermCriteria::EPS, 0, 0);
params.termCrit = _termCrit;
@ -520,9 +519,9 @@ void SVMSGDImpl::setOptimalParameters(int svmsgdType, int marginType)
params.svmsgdType = SGD;
params.marginType = (marginType == SOFT_MARGIN) ? SOFT_MARGIN :
(marginType == HARD_MARGIN) ? HARD_MARGIN : -1;
params.lambda = 0.0001f;
params.gamma0 = 0.05f;
params.c = 1.f;
params.marginRegularization = 0.0001f;
params.initialStepSize = 0.05f;
params.stepDecreasingPower = 1.f;
params.termCrit = TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 100000, 0.00001);
break;
@ -530,9 +529,9 @@ void SVMSGDImpl::setOptimalParameters(int svmsgdType, int marginType)
params.svmsgdType = ASGD;
params.marginType = (marginType == SOFT_MARGIN) ? SOFT_MARGIN :
(marginType == HARD_MARGIN) ? HARD_MARGIN : -1;
params.lambda = 0.00001f;
params.gamma0 = 0.05f;
params.c = 0.75f;
params.marginRegularization = 0.00001f;
params.initialStepSize = 0.05f;
params.stepDecreasingPower = 0.75f;
params.termCrit = TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 100000, 0.00001);
break;

6
modules/ml/test/test_mltests2.cpp
View File

@ -469,9 +469,9 @@ int CV_MLBaseTest::train( int testCaseIdx )
int marginType = str_to_margin_type( marginTypeStr );
m->setMarginType(marginType);
m->setLambda(modelParamsNode["lambda"]);
m->setGamma0(modelParamsNode["gamma0"]);
m->setC(modelParamsNode["c"]);
m->setMarginRegularization(modelParamsNode["marginRegularization"]);
m->setInitialStepSize(modelParamsNode["initialStepSize"]);
m->setStepDecreasingPower(modelParamsNode["stepDecreasingPower"]);
m->setTermCriteria(TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 10000, 0.00001));
model = m;
}

1
modules/ml/test/test_svmsgd.cpp
View File

@ -200,7 +200,6 @@ void CV_SVMSGDTrainTest::run( int /*start_from*/ )
}
float err = (float)errCount / testSamplesCount;
std::cout << "err " << err << std::endl;
if ( err > precision )
{

4
samples/cpp/train_svmsgd.cpp
View File

@ -28,7 +28,7 @@ struct Data
bool doTrain(const Mat samples, const Mat responses, Mat &weights, float &shift);
//function finds two points for drawing line (wx = 0)
bool findPointsForLine(const Mat &weights, float shift, Point (&points)[2], int width, int height);
bool findPointsForLine(const Mat &weights, float shift, Point points[], int width, int height);
// function finds cross point of line (wx = 0) and segment ( (y = HEIGHT, 0 <= x <= WIDTH) or (x = WIDTH, 0 <= y <= HEIGHT) )
bool findCrossPointWithBorders(const Mat &weights, float shift, const std::pair<Point,Point> &segment, Point &crossPoint);
@ -119,7 +119,7 @@ bool findCrossPointWithBorders(const Mat &weights, float shift, const std::pair<
return false;
}
bool findPointsForLine(const Mat &weights, float shift, Point (&points)[2], int width, int height)
bool findPointsForLine(const Mat &weights, float shift, Point points[2], int width, int height)
{
if (weights.empty())
{