This page gives the Python API reference of xgboost, please also refer to Python Package Introduction for more information about python package.
Core XGBoost Library.
xgboost.
DMatrix
(data, label=None, missing=None, weight=None, silent=False, feature_names=None, feature_types=None, nthread=None)¶Bases: object
Data Matrix used in XGBoost.
DMatrix is a internal data structure that used by XGBoost which is optimized for both memory efficiency and training speed. You can construct DMatrix from numpy.arrays
Parameters: 


get_float_info
(field)¶Get float property from the DMatrix.
Parameters:  field (str) – The field name of the information 

Returns:  info – a numpy array of float information of the data 
Return type:  array 
get_label
()¶Get the label of the DMatrix.
Returns:  label 

Return type:  array 
get_uint_info
(field)¶Get unsigned integer property from the DMatrix.
Parameters:  field (str) – The field name of the information 

Returns:  info – a numpy array of unsigned integer information of the data 
Return type:  array 
get_weight
()¶Get the weight of the DMatrix.
Returns:  weight 

Return type:  array 
num_col
()¶Get the number of columns (features) in the DMatrix.
Returns:  number of columns 

Return type:  int 
save_binary
(fname, silent=True)¶Save DMatrix to an XGBoost buffer. Saved binary can be later loaded
by providing the path to xgboost.DMatrix()
as input.
Parameters: 


set_base_margin
(margin)¶Set base margin of booster to start from.
This can be used to specify a prediction value of existing model to be base_margin However, remember margin is needed, instead of transformed prediction e.g. for logistic regression: need to put in value before logistic transformation see also example/demo.py
Parameters:  margin (array like) – Prediction margin of each datapoint 

set_float_info
(field, data)¶Set float type property into the DMatrix.
Parameters: 


set_float_info_npy2d
(field, data)¶Parameters: 


set_group
(group)¶Set group size of DMatrix (used for ranking).
Parameters:  group (array like) – Group size of each group 

set_label
(label)¶Set label of dmatrix
Parameters:  label (array like) – The label information to be set into DMatrix 

set_label_npy2d
(label)¶Set label of dmatrix
Parameters:  label (array like) – The label information to be set into DMatrix from numpy 2D array 

set_uint_info
(field, data)¶Set uint type property into the DMatrix.
Parameters: 


set_weight
(weight)¶Set weight of each instance.
Parameters:  weight (array like) – Weight for each data point Note For ranking task, weights are pergroup. In ranking task, one weight is assigned to each group (not each data point). This is because we only care about the relative ordering of data points within each group, so it doesn’t make sense to assign weights to individual data points. 

set_weight_npy2d
(weight)¶Parameters:  weight (array like) – Weight for each data point in numpy 2D array Note For ranking task, weights are pergroup. In ranking task, one weight is assigned to each group (not each data point). This is because we only care about the relative ordering of data points within each group, so it doesn’t make sense to assign weights to individual data points. 

slice
(rindex, allow_groups=False)¶Slice the DMatrix and return a new DMatrix that only contains rindex.
Parameters: 


Returns:  res – A new DMatrix containing only selected indices. 
Return type: 
xgboost.
Booster
(params=None, cache=(), model_file=None)¶Bases: object
A Booster of XGBoost.
Booster is the model of xgboost, that contains low level routines for training, prediction and evaluation.
Parameters: 

attr
(key)¶Get attribute string from the Booster.
Parameters:  key (str) – The key to get attribute from. 

Returns:  value – The attribute value of the key, returns None if attribute do not exist. 
Return type:  str 
attributes
()¶Get attributes stored in the Booster as a dictionary.
Returns:  result – Returns an empty dict if there’s no attributes. 

Return type:  dictionary of attribute_name: attribute_value pairs of strings. 
boost
(dtrain, grad, hess)¶Boost the booster for one iteration, with customized gradient
statistics. Like xgboost.core.Booster.update()
, this
function should not be called directly by users.
Parameters: 

copy
()¶Copy the booster object.
Returns:  booster – a copied booster model 

Return type:  Booster 
dump_model
(fout, fmap='', with_stats=False, dump_format='text')¶Dump model into a text or JSON file.
Parameters: 


eval
(data, name='eval', iteration=0)¶Evaluate the model on mat.
Parameters:  

Returns:  result – Evaluation result string. 
Return type: 
eval_set
(evals, iteration=0, feval=None)¶Evaluate a set of data.
Parameters:  

Returns:  result – Evaluation result string. 
Return type: 
get_dump
(fmap='', with_stats=False, dump_format='text')¶Returns the model dump as a list of strings.
Parameters: 


get_fscore
(fmap='')¶Get feature importance of each feature.
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Note
Zeroimportance features will not be included
Keep in mind that this function does not include zeroimportance feature, i.e. those features that have not been used in any split conditions.
Parameters:  fmap (str (optional)) – The name of feature map file 

get_score
(fmap='', importance_type='weight')¶Get feature importance of each feature. Importance type can be defined as:
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Parameters: 

get_split_value_histogram
(feature, fmap='', bins=None, as_pandas=True)¶Get split value histogram of a feature
Parameters: 


Returns: 

load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature_names) will not be loaded. To preserve all attributes, pickle the Booster object.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

load_rabit_checkpoint
()¶Initialize the model by load from rabit checkpoint.
Returns:  version – The version number of the model. 

Return type:  integer 
predict
(data, output_margin=False, ntree_limit=0, pred_leaf=False, pred_contribs=False, approx_contribs=False, pred_interactions=False, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call bst.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature_names) will not be saved. To preserve all attributes, pickle the Booster object.
Parameters:  fname (string) – Output file name 

save_rabit_checkpoint
()¶Save the current booster to rabit checkpoint.
save_raw
()¶Save the model to a in memory buffer representation
Returns:  

Return type:  a in memory buffer representation of the model 
set_attr
(**kwargs)¶Set the attribute of the Booster.
Parameters:  **kwargs – The attributes to set. Setting a value to None deletes an attribute. 

set_param
(params, value=None)¶Set parameters into the Booster.
Parameters: 


trees_to_dataframe
(fmap='')¶Parse a boosted tree model text dump into a pandas DataFrame structure.
This feature is only defined when the decision tree model is chosen as base learner (booster in {gbtree, dart}). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Parameters:  fmap (str (optional)) – The name of feature map file. 

update
(dtrain, iteration, fobj=None)¶Update for one iteration, with objective function calculated internally. This function should not be called directly by users.
Parameters: 

Training Library containing training routines.
xgboost.
train
(params, dtrain, num_boost_round=10, evals=(), obj=None, feval=None, maximize=False, early_stopping_rounds=None, evals_result=None, verbose_eval=True, xgb_model=None, callbacks=None, learning_rates=None)¶Train a booster with given parameters.
Parameters: 


Returns:  Booster 
Return type:  a trained booster model 
xgboost.
cv
(params, dtrain, num_boost_round=10, nfold=3, stratified=False, folds=None, metrics=(), obj=None, feval=None, maximize=False, early_stopping_rounds=None, fpreproc=None, as_pandas=True, verbose_eval=None, show_stdv=True, seed=0, callbacks=None, shuffle=True)¶Crossvalidation with given parameters.
Parameters: 


Returns:  evaluation history 
Return type:  list(string) 
ScikitLearn Wrapper interface for XGBoost.
xgboost.
XGBRegressor
(max_depth=3, learning_rate=0.1, n_estimators=100, verbosity=1, silent=None, objective='reg:squarederror', booster='gbtree', n_jobs=1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, subsample=1, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=1, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, base_score=0.5, random_state=0, seed=None, missing=None, importance_type='gain', **kwargs)¶Bases: xgboost.sklearn.XGBModel
, object
Implementation of the scikitlearn API for XGBoost regression.
Parameters: 


Note
A custom objective function can be provided for the objective
parameter. In this case, it should have the signature
objective(y_true, y_pred) > grad, hess
:
apply
(X, ntree_limit=0)¶Return the predicted leaf every tree for each sample.
Parameters: 


Returns:  X_leaves – For each datapoint x in X and for each tree, return the index of the
leaf x ends up in. Leaves are numbered within

Return type:  array_like, shape=[n_samples, n_trees] 
coef_
¶Coefficients property
Note
Coefficients are defined only for linear learners
Coefficients are only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  coef_ 

Return type:  array of shape [n_features] or [n_classes, n_features] 
evals_result
()¶Return the evaluation results.
If eval_set is passed to the fit function, you can call
evals_result()
to get evaluation results for all passed eval_sets.
When eval_metric is also passed to the fit function, the
evals_result will contain the eval_metrics passed to the fit function.
Returns:  evals_result 

Return type:  dictionary 
Example
param_dist = {'objective':'binary:logistic', 'n_estimators':2}
clf = xgb.XGBModel(**param_dist)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric='logloss',
verbose=True)
evals_result = clf.evals_result()
The variable evals_result will contain:
{'validation_0': {'logloss': ['0.604835', '0.531479']},
'validation_1': {'logloss': ['0.41965', '0.17686']}}
feature_importances_
¶Feature importances property
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Returns:  feature_importances_ 

Return type:  array of shape [n_features] 
fit
(X, y, sample_weight=None, eval_set=None, eval_metric=None, early_stopping_rounds=None, verbose=True, xgb_model=None, sample_weight_eval_set=None, callbacks=None)¶Fit the gradient boosting model
Parameters: 


get_booster
()¶Get the underlying xgboost Booster of this model.
This will raise an exception when fit was not called
Returns:  booster 

Return type:  a xgboost booster of underlying model 
get_num_boosting_rounds
()¶Gets the number of xgboost boosting rounds.
get_params
(deep=False)¶Get parameters.
get_xgb_params
()¶Get xgboost type parameters.
intercept_
¶Intercept (bias) property
Note
Intercept is defined only for linear learners
Intercept (bias) is only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  intercept_ 

Return type:  array of shape (1,) or [n_classes] 
load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

predict
(data, output_margin=False, ntree_limit=None, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string) – Output file name 

set_params
(**params)¶Set the parameters of this estimator. Modification of the sklearn method to allow unknown kwargs. This allows using the full range of xgboost parameters that are not defined as member variables in sklearn grid search. :returns: :rtype: self
xgboost.
XGBClassifier
(max_depth=3, learning_rate=0.1, n_estimators=100, verbosity=1, silent=None, objective='binary:logistic', booster='gbtree', n_jobs=1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, subsample=1, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=1, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, base_score=0.5, random_state=0, seed=None, missing=None, **kwargs)¶Bases: xgboost.sklearn.XGBModel
, object
Implementation of the scikitlearn API for XGBoost classification.
Parameters: 


Note
A custom objective function can be provided for the objective
parameter. In this case, it should have the signature
objective(y_true, y_pred) > grad, hess
:
apply
(X, ntree_limit=0)¶Return the predicted leaf every tree for each sample.
Parameters: 


Returns:  X_leaves – For each datapoint x in X and for each tree, return the index of the
leaf x ends up in. Leaves are numbered within

Return type:  array_like, shape=[n_samples, n_trees] 
coef_
¶Coefficients property
Note
Coefficients are defined only for linear learners
Coefficients are only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  coef_ 

Return type:  array of shape [n_features] or [n_classes, n_features] 
evals_result
()¶Return the evaluation results.
If eval_set is passed to the fit function, you can call
evals_result()
to get evaluation results for all passed eval_sets.
When eval_metric is also passed to the fit function, the
evals_result will contain the eval_metrics passed to the fit function.
Returns:  evals_result 

Return type:  dictionary 
Example
param_dist = {'objective':'binary:logistic', 'n_estimators':2}
clf = xgb.XGBClassifier(**param_dist)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric='logloss',
verbose=True)
evals_result = clf.evals_result()
The variable evals_result will contain
{'validation_0': {'logloss': ['0.604835', '0.531479']},
'validation_1': {'logloss': ['0.41965', '0.17686']}}
feature_importances_
¶Feature importances property
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Returns:  feature_importances_ 

Return type:  array of shape [n_features] 
fit
(X, y, sample_weight=None, eval_set=None, eval_metric=None, early_stopping_rounds=None, verbose=True, xgb_model=None, sample_weight_eval_set=None, callbacks=None)¶Fit gradient boosting classifier
Parameters: 


get_booster
()¶Get the underlying xgboost Booster of this model.
This will raise an exception when fit was not called
Returns:  booster 

Return type:  a xgboost booster of underlying model 
get_num_boosting_rounds
()¶Gets the number of xgboost boosting rounds.
get_params
(deep=False)¶Get parameters.
get_xgb_params
()¶Get xgboost type parameters.
intercept_
¶Intercept (bias) property
Note
Intercept is defined only for linear learners
Intercept (bias) is only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  intercept_ 

Return type:  array of shape (1,) or [n_classes] 
load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

predict
(data, output_margin=False, ntree_limit=None, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
predict_proba
(data, ntree_limit=None, validate_features=True)¶Predict the probability of each data example being of a given class.
Note
This function is not thread safe
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict
Parameters: 


Returns:  prediction – a numpy array with the probability of each data example being of a given class. 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string) – Output file name 

set_params
(**params)¶Set the parameters of this estimator. Modification of the sklearn method to allow unknown kwargs. This allows using the full range of xgboost parameters that are not defined as member variables in sklearn grid search. :returns: :rtype: self
xgboost.
XGBRanker
(max_depth=3, learning_rate=0.1, n_estimators=100, verbosity=1, silent=None, objective='rank:pairwise', booster='gbtree', n_jobs=1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, subsample=1, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=1, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, base_score=0.5, random_state=0, seed=None, missing=None, **kwargs)¶Bases: xgboost.sklearn.XGBModel
Implementation of the ScikitLearn API for XGBoost Ranking.
Parameters: 


Note
A custom objective function is currently not supported by XGBRanker.
Note
Group information is required for ranking tasks.
Before fitting the model, your data need to be sorted by group. When fitting the model, you need to provide an additional array that contains the size of each group.
For example, if your original data look like:
qid  label  features 
1  0  x_1 
1  1  x_2 
1  0  x_3 
2  0  x_4 
2  1  x_5 
2  1  x_6 
2  1  x_7 
then your group array should be [3, 4]
.
apply
(X, ntree_limit=0)¶Return the predicted leaf every tree for each sample.
Parameters: 


Returns:  X_leaves – For each datapoint x in X and for each tree, return the index of the
leaf x ends up in. Leaves are numbered within

Return type:  array_like, shape=[n_samples, n_trees] 
coef_
¶Coefficients property
Note
Coefficients are defined only for linear learners
Coefficients are only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  coef_ 

Return type:  array of shape [n_features] or [n_classes, n_features] 
evals_result
()¶Return the evaluation results.
If eval_set is passed to the fit function, you can call
evals_result()
to get evaluation results for all passed eval_sets.
When eval_metric is also passed to the fit function, the
evals_result will contain the eval_metrics passed to the fit function.
Returns:  evals_result 

Return type:  dictionary 
Example
param_dist = {'objective':'binary:logistic', 'n_estimators':2}
clf = xgb.XGBModel(**param_dist)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric='logloss',
verbose=True)
evals_result = clf.evals_result()
The variable evals_result will contain:
{'validation_0': {'logloss': ['0.604835', '0.531479']},
'validation_1': {'logloss': ['0.41965', '0.17686']}}
feature_importances_
¶Feature importances property
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Returns:  feature_importances_ 

Return type:  array of shape [n_features] 
fit
(X, y, group, sample_weight=None, eval_set=None, sample_weight_eval_set=None, eval_group=None, eval_metric=None, early_stopping_rounds=None, verbose=False, xgb_model=None, callbacks=None)¶Fit the gradient boosting model
Parameters: 


get_booster
()¶Get the underlying xgboost Booster of this model.
This will raise an exception when fit was not called
Returns:  booster 

Return type:  a xgboost booster of underlying model 
get_num_boosting_rounds
()¶Gets the number of xgboost boosting rounds.
get_params
(deep=False)¶Get parameters.
get_xgb_params
()¶Get xgboost type parameters.
intercept_
¶Intercept (bias) property
Note
Intercept is defined only for linear learners
Intercept (bias) is only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  intercept_ 

Return type:  array of shape (1,) or [n_classes] 
load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

predict
(data, output_margin=False, ntree_limit=0, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string) – Output file name 

set_params
(**params)¶Set the parameters of this estimator. Modification of the sklearn method to allow unknown kwargs. This allows using the full range of xgboost parameters that are not defined as member variables in sklearn grid search. :returns: :rtype: self
xgboost.
XGBRFRegressor
(max_depth=3, learning_rate=1, n_estimators=100, verbosity=1, silent=None, objective='reg:squarederror', n_jobs=1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, subsample=0.8, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=0.8, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, base_score=0.5, random_state=0, seed=None, missing=None, **kwargs)¶Bases: xgboost.sklearn.XGBRegressor
Experimental implementation of the scikitlearn API for XGBoost random forest regression.
Parameters: 


Note
A custom objective function can be provided for the objective
parameter. In this case, it should have the signature
objective(y_true, y_pred) > grad, hess
:
apply
(X, ntree_limit=0)¶Return the predicted leaf every tree for each sample.
Parameters: 


Returns:  X_leaves – For each datapoint x in X and for each tree, return the index of the
leaf x ends up in. Leaves are numbered within

Return type:  array_like, shape=[n_samples, n_trees] 
coef_
¶Coefficients property
Note
Coefficients are defined only for linear learners
Coefficients are only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  coef_ 

Return type:  array of shape [n_features] or [n_classes, n_features] 
evals_result
()¶Return the evaluation results.
If eval_set is passed to the fit function, you can call
evals_result()
to get evaluation results for all passed eval_sets.
When eval_metric is also passed to the fit function, the
evals_result will contain the eval_metrics passed to the fit function.
Returns:  evals_result 

Return type:  dictionary 
Example
param_dist = {'objective':'binary:logistic', 'n_estimators':2}
clf = xgb.XGBModel(**param_dist)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric='logloss',
verbose=True)
evals_result = clf.evals_result()
The variable evals_result will contain:
{'validation_0': {'logloss': ['0.604835', '0.531479']},
'validation_1': {'logloss': ['0.41965', '0.17686']}}
feature_importances_
¶Feature importances property
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Returns:  feature_importances_ 

Return type:  array of shape [n_features] 
fit
(X, y, sample_weight=None, eval_set=None, eval_metric=None, early_stopping_rounds=None, verbose=True, xgb_model=None, sample_weight_eval_set=None, callbacks=None)¶Fit the gradient boosting model
Parameters: 


get_booster
()¶Get the underlying xgboost Booster of this model.
This will raise an exception when fit was not called
Returns:  booster 

Return type:  a xgboost booster of underlying model 
get_num_boosting_rounds
()¶Gets the number of xgboost boosting rounds.
get_params
(deep=False)¶Get parameters.
get_xgb_params
()¶Get xgboost type parameters.
intercept_
¶Intercept (bias) property
Note
Intercept is defined only for linear learners
Intercept (bias) is only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  intercept_ 

Return type:  array of shape (1,) or [n_classes] 
load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

predict
(data, output_margin=False, ntree_limit=None, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string) – Output file name 

set_params
(**params)¶Set the parameters of this estimator. Modification of the sklearn method to allow unknown kwargs. This allows using the full range of xgboost parameters that are not defined as member variables in sklearn grid search. :returns: :rtype: self
xgboost.
XGBRFClassifier
(max_depth=3, learning_rate=1, n_estimators=100, verbosity=1, silent=None, objective='binary:logistic', n_jobs=1, nthread=None, gamma=0, min_child_weight=1, max_delta_step=0, subsample=0.8, colsample_bytree=1, colsample_bylevel=1, colsample_bynode=0.8, reg_alpha=0, reg_lambda=1, scale_pos_weight=1, base_score=0.5, random_state=0, seed=None, missing=None, **kwargs)¶Bases: xgboost.sklearn.XGBClassifier
Experimental implementation of the scikitlearn API for XGBoost random forest classification.
Parameters: 


Note
A custom objective function can be provided for the objective
parameter. In this case, it should have the signature
objective(y_true, y_pred) > grad, hess
:
apply
(X, ntree_limit=0)¶Return the predicted leaf every tree for each sample.
Parameters: 


Returns:  X_leaves – For each datapoint x in X and for each tree, return the index of the
leaf x ends up in. Leaves are numbered within

Return type:  array_like, shape=[n_samples, n_trees] 
coef_
¶Coefficients property
Note
Coefficients are defined only for linear learners
Coefficients are only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  coef_ 

Return type:  array of shape [n_features] or [n_classes, n_features] 
evals_result
()¶Return the evaluation results.
If eval_set is passed to the fit function, you can call
evals_result()
to get evaluation results for all passed eval_sets.
When eval_metric is also passed to the fit function, the
evals_result will contain the eval_metrics passed to the fit function.
Returns:  evals_result 

Return type:  dictionary 
Example
param_dist = {'objective':'binary:logistic', 'n_estimators':2}
clf = xgb.XGBClassifier(**param_dist)
clf.fit(X_train, y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric='logloss',
verbose=True)
evals_result = clf.evals_result()
The variable evals_result will contain
{'validation_0': {'logloss': ['0.604835', '0.531479']},
'validation_1': {'logloss': ['0.41965', '0.17686']}}
feature_importances_
¶Feature importances property
Note
Feature importance is defined only for tree boosters
Feature importance is only defined when the decision tree model is chosen as base learner (booster=gbtree). It is not defined for other base learner types, such as linear learners (booster=gblinear).
Returns:  feature_importances_ 

Return type:  array of shape [n_features] 
fit
(X, y, sample_weight=None, eval_set=None, eval_metric=None, early_stopping_rounds=None, verbose=True, xgb_model=None, sample_weight_eval_set=None, callbacks=None)¶Fit gradient boosting classifier
Parameters: 


get_booster
()¶Get the underlying xgboost Booster of this model.
This will raise an exception when fit was not called
Returns:  booster 

Return type:  a xgboost booster of underlying model 
get_num_boosting_rounds
()¶Gets the number of xgboost boosting rounds.
get_params
(deep=False)¶Get parameters.
get_xgb_params
()¶Get xgboost type parameters.
intercept_
¶Intercept (bias) property
Note
Intercept is defined only for linear learners
Intercept (bias) is only defined when the linear model is chosen as base learner (booster=gblinear). It is not defined for other base learner types, such as tree learners (booster=gbtree).
Returns:  intercept_ 

Return type:  array of shape (1,) or [n_classes] 
load_model
(fname)¶Load the model from a file.
The model is loaded from an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string or a memory buffer) – Input file name or memory buffer(see also save_raw) 

predict
(data, output_margin=False, ntree_limit=None, validate_features=True)¶Predict with data.
Note
This function is not thread safe.
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict()
.
Note
Using predict()
with DART booster
If the booster object is DART type, predict()
will perform dropouts, i.e. only
some of the trees will be evaluated. This will produce incorrect results if data
is
not the training data. To obtain correct results on test sets, set ntree_limit
to
a nonzero value, e.g.
preds = bst.predict(dtest, ntree_limit=num_round)
Parameters: 


Returns:  prediction 
Return type:  numpy array 
predict_proba
(data, ntree_limit=None, validate_features=True)¶Predict the probability of each data example being of a given class.
Note
This function is not thread safe
For each booster object, predict can only be called from one thread.
If you want to run prediction using multiple thread, call xgb.copy()
to make copies
of model object and then call predict
Parameters: 


Returns:  prediction – a numpy array with the probability of each data example being of a given class. 
Return type:  numpy array 
save_model
(fname)¶Save the model to a file.
The model is saved in an XGBoost internal binary format which is universal among the various XGBoost interfaces. Auxiliary attributes of the Python Booster object (such as feature names) will not be loaded. Label encodings (text labels to numeric labels) will be also lost. If you are using only the Python interface, we recommend pickling the model object for best results.
Parameters:  fname (string) – Output file name 

set_params
(**params)¶Set the parameters of this estimator. Modification of the sklearn method to allow unknown kwargs. This allows using the full range of xgboost parameters that are not defined as member variables in sklearn grid search. :returns: :rtype: self
Plotting Library.
xgboost.
plot_importance
(booster, ax=None, height=0.2, xlim=None, ylim=None, title='Feature importance', xlabel='F score', ylabel='Features', importance_type='weight', max_num_features=None, grid=True, show_values=True, **kwargs)¶Plot importance based on fitted trees.
Parameters: 


Returns:  ax 
Return type:  matplotlib Axes 
xgboost.
plot_tree
(booster, fmap='', num_trees=0, rankdir='UT', ax=None, **kwargs)¶Plot specified tree.
Parameters: 


Returns:  ax 
Return type:  matplotlib Axes 
xgboost.
to_graphviz
(booster, fmap='', num_trees=0, rankdir='UT', yes_color='#0000FF', no_color='#FF0000', condition_node_params=None, leaf_node_params=None, **kwargs)¶Convert specified tree to graphviz instance. IPython can automatically plot the returned graphiz instance. Otherwise, you should call .render() method of the returned graphiz instance.
Parameters: 


Returns:  ax 
Return type:  matplotlib Axes 
xgboost.callback.
print_evaluation
(period=1, show_stdv=True)¶Create a callback that print evaluation result.
We print the evaluation results every period iterations and on the first and the last iterations.
Parameters:  

Returns:  callback – A callback that print evaluation every period iterations. 
Return type:  function 
xgboost.callback.
record_evaluation
(eval_result)¶Create a call back that records the evaluation history into eval_result.
Parameters:  eval_result (dict) – A dictionary to store the evaluation results. 

Returns:  callback – The requested callback function. 
Return type:  function 
xgboost.callback.
reset_learning_rate
(learning_rates)¶Reset learning rate after iteration 1
NOTE: the initial learning rate will still take ineffect on first iteration.
Parameters:  learning_rates (list or function) – List of learning rate for each boosting round or a customized function that calculates eta in terms of current number of round and the total number of boosting round (e.g. yields learning rate decay)


Returns:  callback – The requested callback function. 
Return type:  function 
xgboost.callback.
early_stop
(stopping_rounds, maximize=False, verbose=True)¶Create a callback that activates early stoppping.
Validation error needs to decrease at least
every stopping_rounds round(s) to continue training.
Requires at least one item in evals.
If there’s more than one, will use the last.
Returns the model from the last iteration (not the best one).
If early stopping occurs, the model will have three additional fields:
bst.best_score
, bst.best_iteration
and bst.best_ntree_limit
.
(Use bst.best_ntree_limit
to get the correct value if num_parallel_tree
and/or num_class
appears in the parameters)
Parameters:  

Returns:  callback – The requested callback function. 
Return type:  function 
Dask extensions for distributed training. See xgboost/demo/dask for examples.
xgboost.dask.
run
(client, func, *args)¶Launch arbitrary function on dask workers. Workers are connected by rabit, allowing distributed training. The environment variable OMP_NUM_THREADS is defined on each worker according to dask  this means that calls to xgb.train() will use the threads allocated by dask by default, unless the user overrides the nthread parameter.
Note: Windows platforms are not officially supported. Contributions are welcome here.
Parameters: 


training code. :param args: Arguments to be forwarded to func :return: Dict containing the function return value for each worker
xgboost.dask.
create_worker_dmatrix
(*args, **kwargs)¶Creates a DMatrix object local to a given worker. Simply forwards arguments onto the standard DMatrix constructor, if one of the arguments is a dask dataframe, unpack the data frame to get the local components.
All dask dataframe arguments must use the same partitioning.
Parameters:  args – DMatrix constructor args. 

Returns:  DMatrix object containing data local to current dask worker 
xgboost.dask.
get_local_data
(data)¶Unpacks a distributed data object to get the rows local to this worker
Parameters:  data – A distributed dask data object 

Returns:  Local data partition e.g. numpy or pandas 