xgboost
Namespaces | Classes | Typedefs | Enumerations | Functions | Variables
xgboost::common Namespace Reference

Namespaces

 aft
 
 base64
 namespace of base64 decoding and encoding table
 
 detail
 

Classes

struct  AFTLoss
 The AFT loss function. More...
 
struct  AFTParam
 Parameter structure for AFT loss and metric. More...
 
class  Base64InStream
 the stream that reads from base64, note we take from file pointers More...
 
class  Base64OutStream
 the stream that write to base64, note we take from file pointers More...
 
class  BlockedSpace2d
 
class  Column
 a column storage, to be used with ApplySplit. Note that each bin id is stored as index[i] + index_base. Different types of column index for each column allow to reduce the memory usage. More...
 
class  ColumnMatrix
 a collection of columns, with support for construction from GHistIndexMatrix. More...
 
class  ColumnSampler
 Handles selection of columns due to colsample_bytree, colsample_bylevel and colsample_bynode parameters. Should be initialised before tree construction and to reset when tree construction is completed. More...
 
class  CompressedBufferWriter
 Writes bit compressed symbols to a memory buffer. Use CompressedIterator to read symbols back from buffer. Currently limited to a maximum symbol size of 28 bits. More...
 
class  CompressedIterator
 Read symbols from a bit compressed memory buffer. Usable on device and host. More...
 
class  ConfigParser
 Implementation of config reader. More...
 
class  DenseColumn
 
struct  ExtremeDistribution
 
class  FixedSizeStream
 A simple class used to consume ‘dmlc::Stream’ all at once. More...
 
class  GHistBuilder
 builder for histograms of gradient statistics More...
 
class  HistCollection
 histogram of gradient statistics for multiple nodes More...
 
class  HistogramCuts
 
class  HostSketchContainer
 
struct  Index
 Optionally compressed gradient index. The compression works only with dense data. More...
 
class  IndexTransformIter
 Transform iterator that takes an index and calls transform operator. More...
 
struct  IsCatOp
 
struct  LogisticDistribution
 
class  MemStackAllocator
 A C-style array with in-stack allocation. As long as the array is smaller than MaxStackSize, it will be allocated inside the stack. Otherwise, it will be heap-allocated. More...
 
struct  Monitor
 Timing utility used to measure total method execution time over the lifetime of the containing object. More...
 
struct  NormalDistribution
 
struct  OptionalWeights
 
class  ParallelGHistBuilder
 Stores temporary histograms to compute them in parallel Supports processing multiple tree-nodes for nested parallelism Able to reduce histograms across threads in efficient way. More...
 
class  ParallelGroupBuilder
 multi-thread version of group builder More...
 
class  PartitionBuilder
 
class  PeekableInStream
 Input stream that support additional PeekRead operation, besides read. More...
 
class  QuantileSketchTemplate
 template for all quantile sketch algorithm that uses merge/prune scheme More...
 
class  Range
 
class  Range1d
 
class  RowSetCollection
 collection of rowset More...
 
struct  Sched
 
class  SketchContainerImpl
 
struct  SortedQuantile
 Quantile structure accepts sorted data, extracted from histmaker. More...
 
class  SortedSketchContainer
 
class  Span
 span class implementation, based on ISO++20 span<T>. The interface should be the same. More...
 
class  SparseColumn
 
class  StreamBufferReader
 buffer reader of the stream that allows you to get More...
 
struct  Timer
 
class  Transform
 Do Transformation on HostDeviceVectors. More...
 
struct  WQSummary
 experimental wsummary More...
 
class  WQuantileSketch
 Quantile sketch use WQSummary. More...
 
struct  WXQSummary
 try to do efficient pruning More...
 
class  WXQuantileSketch
 Quantile sketch use WXQSummary. More...
 

Typedefs

using CatBitField = LBitField32
 
using KCatBitField = CLBitField32
 
using CompressedByteT = unsigned char
 
using GHistIndexRow = Span< uint32_t const >
 A single row in global histogram index. Directly represent the global index in the histogram entry. More...
 
template<typename GradientSumT >
using GHistRow = Span< xgboost::detail::GradientPairInternal< GradientSumT > >
 
using MemoryFixSizeBuffer = rabit::utils::MemoryFixSizeBuffer
 
using MemoryBufferStream = rabit::utils::MemoryBufferStream
 
using RandomEngine = std::mt19937
 Define mt19937 as default type Random Engine. More...
 
using GlobalRandomEngine = RandomEngine
 global random engine More...
 

Enumerations

enum  byte : unsigned char
 
enum  ColumnType : uint8_t { kDenseColumn, kSparseColumn }
 column type More...
 
enum  BinTypeSize : uint32_t { kUint8BinsTypeSize = 1, kUint16BinsTypeSize = 2, kUint32BinsTypeSize = 4 }
 
enum  ProbabilityDistributionType : int { ProbabilityDistributionType::kNormal = 0, ProbabilityDistributionType::kLogistic = 1, ProbabilityDistributionType::kExtreme = 2 }
 Enum encoding possible choices of probability distribution. More...
 
enum  CensoringType : uint8_t { CensoringType::kUncensored, CensoringType::kRightCensored, CensoringType::kLeftCensored, CensoringType::kIntervalCensored }
 

Functions

template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE bool operator== (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool operator!= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool operator< (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool operator<= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool operator> (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool operator>= (Span< T, X > l, Span< U, Y > r)
 
template<class T , std::size_t E>
XGBOOST_DEVICE auto as_bytes (Span< T, E > s) __span_noexcept -> Span< const byte, detail::ExtentAsBytesValue< T, E >::value >
 
template<class T , std::size_t E>
XGBOOST_DEVICE auto as_writable_bytes (Span< T, E > s) __span_noexcept -> Span< byte, detail::ExtentAsBytesValue< T, E >::value >
 
template<typename T >
XGBOOST_DEVICE bst_cat_t AsCat (T const &v)
 
XGBOOST_DEVICE bool IsCat (Span< FeatureType const > ft, bst_feature_t fidx)
 
constexpr bst_cat_t OutOfRangeCat ()
 
XGBOOST_DEVICE bool InvalidCat (float cat)
 
template<bool validate = true>
XGBOOST_DEVICE bool Decision (common::Span< uint32_t const > cats, float cat, bool dft_left)
 
void InvalidCategory ()
 
void CheckMaxCat (float max_cat, size_t n_categories)
 
XGBOOST_DEVICE bool UseOneHot (uint32_t n_cats, uint32_t max_cat_to_onehot)
 Whether should we use onehot encoding for categorical data. More...
 
std::vector< std::string > Split (const std::string &s, char delim)
 Split a string by delimiter. More...
 
template<typename T >
XGBOOST_DEVICEMax (T a, T b)
 
template<typename T >
std::string ToString (const T &data)
 
template<typename T1 , typename T2 >
XGBOOST_DEVICE T1 DivRoundUp (const T1 a, const T2 b)
 
template<class T , std::size_t N>
constexpr auto UnpackArr (std::array< T, N > &&arr)
 
template<typename Fn >
auto MakeIndexTransformIter (Fn &&fn)
 
int AllVisibleGPUs ()
 
void AssertGPUSupport ()
 
void AssertOneAPISupport ()
 
template<typename Idx , typename Container , typename V = typename Container::value_type, typename Comp = std::less<V>>
std::vector< Idx > ArgSort (Container const &array, Comp comp=std::less< V >{})
 
HistogramCuts SketchOnDMatrix (DMatrix *m, int32_t max_bins, int32_t n_threads, bool use_sorted=false, Span< float > const hessian={})
 Run CPU sketching on DMatrix. More...
 
template<typename GradientIndex >
int32_t XGBOOST_HOST_DEV_INLINE BinarySearchBin (size_t begin, size_t end, GradientIndex const &data, uint32_t const fidx_begin, uint32_t const fidx_end)
 
template<typename GradientSumT >
void InitilizeHistByZeroes (GHistRow< GradientSumT > hist, size_t begin, size_t end)
 fill a histogram by zeros More...
 
template<typename GradientSumT >
void IncrementHist (GHistRow< GradientSumT > dst, const GHistRow< GradientSumT > add, size_t begin, size_t end)
 Increment hist as dst += add in range [begin, end) More...
 
template<typename GradientSumT >
void CopyHist (GHistRow< GradientSumT > dst, const GHistRow< GradientSumT > src, size_t begin, size_t end)
 Copy hist from src to dst in range [begin, end) More...
 
template<typename GradientSumT >
void SubtractionHist (GHistRow< GradientSumT > dst, const GHistRow< GradientSumT > src1, const GHistRow< GradientSumT > src2, size_t begin, size_t end)
 Compute Subtraction: dst = src1 - src2 in range [begin, end) More...
 
std::string LoadSequentialFile (std::string uri, bool stream=false)
 Helper function for loading consecutive file to avoid dmlc Stream when possible. More...
 
std::string FileExtension (std::string fname, bool lower=true)
 Get file extension from file name. More...
 
std::string ReadAll (dmlc::Stream *fi, PeekableInStream *fp)
 Read the whole buffer from dmlc stream. More...
 
XGBOOST_DEVICE float Sigmoid (float x)
 calculate the sigmoid of the input. More...
 
template<typename T , typename U >
constexpr XGBOOST_DEVICE bool CloseTo (T a, U b)
 Equality test for both integer and floating point. More...
 
template<typename Iterator >
XGBOOST_DEVICE void Softmax (Iterator start, Iterator end)
 Do inplace softmax transformaton on start to end. More...
 
template<typename Iterator >
XGBOOST_DEVICE Iterator FindMaxIndex (Iterator begin, Iterator end)
 Find the maximum iterator within the iterators. More...
 
float LogSum (float x, float y)
 perform numerically safe logsum More...
 
template<typename Iterator >
float LogSum (Iterator begin, Iterator end)
 perform numerically safe logsum More...
 
template<typename T >
XGBOOST_DEVICE std::enable_if< std::numeric_limits< T >::is_integer, bool >::type CheckNAN (T)
 
XGBOOST_DEVICE bool CheckNAN (float x)
 
XGBOOST_DEVICE bool CheckNAN (double x)
 
template<typename T >
XGBOOST_DEVICELogGamma (T v)
 
GlobalRandomEngineGlobalRandom ()
 global singleton of a random engine. This random engine is thread-local and only visible to current thread. More...
 
template<typename T >
std::vector< T > WeightedSamplingWithoutReplacement (std::vector< T > const &array, std::vector< float > const &weights, size_t n)
 
template<typename Func >
void ParallelFor2d (const BlockedSpace2d &space, int nthreads, Func func)
 
template<typename Index , typename Func >
void ParallelFor (Index size, int32_t n_threads, Sched sched, Func fn)
 
template<typename Index , typename Func >
void ParallelFor (Index size, int32_t n_threads, Func fn)
 
int32_t OmpGetThreadLimit ()
 
int32_t GetCfsCPUCount () noexcept
 
int32_t OmpGetNumThreads (int32_t n_threads)
 

Variables

constexpr std::size_t dynamic_extent = std::numeric_limits<std::size_t>::max()
 
constexpr double kPI = 3.14159265358979323846
 Constant PI. More...
 
constexpr double kEulerMascheroni = 0.57721566490153286060651209008240243104215933593992
 The Euler-Mascheroni_constant. More...
 
constexpr size_t kBlockThreads = 256
 

Typedef Documentation

◆ CatBitField

◆ CompressedByteT

using xgboost::common::CompressedByteT = typedef unsigned char

◆ GHistIndexRow

using xgboost::common::GHistIndexRow = typedef Span<uint32_t const>

A single row in global histogram index. Directly represent the global index in the histogram entry.

◆ GHistRow

template<typename GradientSumT >
using xgboost::common::GHistRow = typedef Span<xgboost::detail::GradientPairInternal<GradientSumT> >

◆ GlobalRandomEngine

global random engine

◆ KCatBitField

◆ MemoryBufferStream

using xgboost::common::MemoryBufferStream = typedef rabit::utils::MemoryBufferStream

◆ MemoryFixSizeBuffer

using xgboost::common::MemoryFixSizeBuffer = typedef rabit::utils::MemoryFixSizeBuffer

◆ RandomEngine

using xgboost::common::RandomEngine = typedef std::mt19937

Define mt19937 as default type Random Engine.

Enumeration Type Documentation

◆ BinTypeSize

Enumerator
kUint8BinsTypeSize 
kUint16BinsTypeSize 
kUint32BinsTypeSize 

◆ byte

enum xgboost::common::byte : unsigned char
strong

◆ CensoringType

enum xgboost::common::CensoringType : uint8_t
strong
Enumerator
kUncensored 
kRightCensored 
kLeftCensored 
kIntervalCensored 

◆ ColumnType

column type

Enumerator
kDenseColumn 
kSparseColumn 

◆ ProbabilityDistributionType

Enum encoding possible choices of probability distribution.

Enumerator
kNormal 
kLogistic 
kExtreme 

Function Documentation

◆ AllVisibleGPUs()

int xgboost::common::AllVisibleGPUs ( )

◆ ArgSort()

template<typename Idx , typename Container , typename V = typename Container::value_type, typename Comp = std::less<V>>
std::vector<Idx> xgboost::common::ArgSort ( Container const &  array,
Comp  comp = std::less<V>{} 
)

◆ as_bytes()

template<class T , std::size_t E>
XGBOOST_DEVICE auto xgboost::common::as_bytes ( Span< T, E >  s) -> Span<const byte, detail::ExtentAsBytesValue<T, E>::value>

◆ as_writable_bytes()

template<class T , std::size_t E>
XGBOOST_DEVICE auto xgboost::common::as_writable_bytes ( Span< T, E >  s) -> Span<byte, detail::ExtentAsBytesValue<T, E>::value>

◆ AsCat()

template<typename T >
XGBOOST_DEVICE bst_cat_t xgboost::common::AsCat ( T const &  v)

◆ AssertGPUSupport()

void xgboost::common::AssertGPUSupport ( )
inline

◆ AssertOneAPISupport()

void xgboost::common::AssertOneAPISupport ( )
inline

◆ BinarySearchBin()

template<typename GradientIndex >
int32_t XGBOOST_HOST_DEV_INLINE xgboost::common::BinarySearchBin ( size_t  begin,
size_t  end,
GradientIndex const &  data,
uint32_t const  fidx_begin,
uint32_t const  fidx_end 
)

◆ CheckMaxCat()

void xgboost::common::CheckMaxCat ( float  max_cat,
size_t  n_categories 
)
inline

◆ CheckNAN() [1/3]

XGBOOST_DEVICE bool xgboost::common::CheckNAN ( double  x)
inline

◆ CheckNAN() [2/3]

XGBOOST_DEVICE bool xgboost::common::CheckNAN ( float  x)
inline

◆ CheckNAN() [3/3]

template<typename T >
XGBOOST_DEVICE std::enable_if< std::numeric_limits<T>::is_integer, bool>::type xgboost::common::CheckNAN ( )

◆ CloseTo()

template<typename T , typename U >
constexpr XGBOOST_DEVICE bool xgboost::common::CloseTo ( a,
b 
)
constexpr

Equality test for both integer and floating point.

◆ CopyHist()

template<typename GradientSumT >
void xgboost::common::CopyHist ( GHistRow< GradientSumT >  dst,
const GHistRow< GradientSumT >  src,
size_t  begin,
size_t  end 
)

Copy hist from src to dst in range [begin, end)

◆ Decision()

template<bool validate = true>
XGBOOST_DEVICE bool xgboost::common::Decision ( common::Span< uint32_t const >  cats,
float  cat,
bool  dft_left 
)
inline

◆ DivRoundUp()

template<typename T1 , typename T2 >
XGBOOST_DEVICE T1 xgboost::common::DivRoundUp ( const T1  a,
const T2  b 
)

◆ FileExtension()

std::string xgboost::common::FileExtension ( std::string  fname,
bool  lower = true 
)

Get file extension from file name.

Parameters
lowerReturn in lower case.
Returns
File extension without the .

◆ FindMaxIndex()

template<typename Iterator >
XGBOOST_DEVICE Iterator xgboost::common::FindMaxIndex ( Iterator  begin,
Iterator  end 
)
inline

Find the maximum iterator within the iterators.

Parameters
beginThe beginning iterator.
endThe end iterator.
Returns
the iterator point to the maximum value.
Template Parameters
IteratorThe type of the iterator.

◆ GetCfsCPUCount()

int32_t xgboost::common::GetCfsCPUCount ( )
noexcept

◆ GlobalRandom()

GlobalRandomEngine& xgboost::common::GlobalRandom ( )

global singleton of a random engine. This random engine is thread-local and only visible to current thread.

◆ IncrementHist()

template<typename GradientSumT >
void xgboost::common::IncrementHist ( GHistRow< GradientSumT >  dst,
const GHistRow< GradientSumT >  add,
size_t  begin,
size_t  end 
)

Increment hist as dst += add in range [begin, end)

◆ InitilizeHistByZeroes()

template<typename GradientSumT >
void xgboost::common::InitilizeHistByZeroes ( GHistRow< GradientSumT >  hist,
size_t  begin,
size_t  end 
)

fill a histogram by zeros

◆ InvalidCat()

XGBOOST_DEVICE bool xgboost::common::InvalidCat ( float  cat)
inline

◆ InvalidCategory()

void xgboost::common::InvalidCategory ( )
inline

◆ IsCat()

XGBOOST_DEVICE bool xgboost::common::IsCat ( Span< FeatureType const >  ft,
bst_feature_t  fidx 
)
inline

◆ LoadSequentialFile()

std::string xgboost::common::LoadSequentialFile ( std::string  uri,
bool  stream = false 
)

Helper function for loading consecutive file to avoid dmlc Stream when possible.

Parameters
uriURI or file name to file.
streamUse dmlc Stream unconditionally if set to true. Used for running test without remote filesystem.
Returns
File content.

◆ LogGamma()

template<typename T >
XGBOOST_DEVICE T xgboost::common::LogGamma ( v)
inline

◆ LogSum() [1/2]

float xgboost::common::LogSum ( float  x,
float  y 
)
inline

perform numerically safe logsum

Parameters
xleft input operand
yright input operand
Returns
log(exp(x) + exp(y))

◆ LogSum() [2/2]

template<typename Iterator >
float xgboost::common::LogSum ( Iterator  begin,
Iterator  end 
)
inline

perform numerically safe logsum

Parameters
beginThe beginning iterator.
endThe end iterator.
Returns
the iterator point to the maximum value.
Template Parameters
IteratorThe type of the iterator.

◆ MakeIndexTransformIter()

template<typename Fn >
auto xgboost::common::MakeIndexTransformIter ( Fn &&  fn)

◆ Max()

template<typename T >
XGBOOST_DEVICE T xgboost::common::Max ( a,
b 
)

◆ OmpGetNumThreads()

int32_t xgboost::common::OmpGetNumThreads ( int32_t  n_threads)
inline

◆ OmpGetThreadLimit()

int32_t xgboost::common::OmpGetThreadLimit ( )
inline

◆ operator!=()

template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool xgboost::common::operator!= ( Span< T, X >  l,
Span< U, Y >  r 
)
constexpr

◆ operator<()

template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool xgboost::common::operator< ( Span< T, X >  l,
Span< U, Y >  r 
)
constexpr

◆ operator<=()

template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool xgboost::common::operator<= ( Span< T, X >  l,
Span< U, Y >  r 
)
constexpr

◆ operator==()

template<class T , std::size_t X, class U , std::size_t Y>
XGBOOST_DEVICE bool xgboost::common::operator== ( Span< T, X >  l,
Span< U, Y >  r 
)

◆ operator>()

template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool xgboost::common::operator> ( Span< T, X >  l,
Span< U, Y >  r 
)
constexpr

◆ operator>=()

template<class T , std::size_t X, class U , std::size_t Y>
constexpr XGBOOST_DEVICE bool xgboost::common::operator>= ( Span< T, X >  l,
Span< U, Y >  r 
)
constexpr

◆ OutOfRangeCat()

constexpr bst_cat_t xgboost::common::OutOfRangeCat ( )
inlineconstexpr

◆ ParallelFor() [1/2]

template<typename Index , typename Func >
void xgboost::common::ParallelFor ( Index  size,
int32_t  n_threads,
Func  fn 
)

◆ ParallelFor() [2/2]

template<typename Index , typename Func >
void xgboost::common::ParallelFor ( Index  size,
int32_t  n_threads,
Sched  sched,
Func  fn 
)

◆ ParallelFor2d()

template<typename Func >
void xgboost::common::ParallelFor2d ( const BlockedSpace2d space,
int  nthreads,
Func  func 
)

◆ ReadAll()

std::string xgboost::common::ReadAll ( dmlc::Stream *  fi,
PeekableInStream fp 
)
inline

Read the whole buffer from dmlc stream.

◆ Sigmoid()

XGBOOST_DEVICE float xgboost::common::Sigmoid ( float  x)
inline

calculate the sigmoid of the input.

Parameters
xinput parameter
Returns
the transformed value.

◆ SketchOnDMatrix()

HistogramCuts xgboost::common::SketchOnDMatrix ( DMatrix m,
int32_t  max_bins,
int32_t  n_threads,
bool  use_sorted = false,
Span< float > const  hessian = {} 
)
inline

Run CPU sketching on DMatrix.

Parameters
use_sortedWhether should we use SortedCSC for sketching, it's more efficient but consumes more memory.

◆ Softmax()

template<typename Iterator >
XGBOOST_DEVICE void xgboost::common::Softmax ( Iterator  start,
Iterator  end 
)
inline

Do inplace softmax transformaton on start to end.

Template Parameters
IteratorInput iterator type
Parameters
startStart iterator of input
endend iterator of input

◆ Split()

std::vector<std::string> xgboost::common::Split ( const std::string &  s,
char  delim 
)
inline

Split a string by delimiter.

Parameters
sString to be split.
delimThe delimiter.

◆ SubtractionHist()

template<typename GradientSumT >
void xgboost::common::SubtractionHist ( GHistRow< GradientSumT >  dst,
const GHistRow< GradientSumT >  src1,
const GHistRow< GradientSumT >  src2,
size_t  begin,
size_t  end 
)

Compute Subtraction: dst = src1 - src2 in range [begin, end)

◆ ToString()

template<typename T >
std::string xgboost::common::ToString ( const T &  data)
inline

◆ UnpackArr()

template<class T , std::size_t N>
constexpr auto xgboost::common::UnpackArr ( std::array< T, N > &&  arr)
constexpr

◆ UseOneHot()

XGBOOST_DEVICE bool xgboost::common::UseOneHot ( uint32_t  n_cats,
uint32_t  max_cat_to_onehot 
)
inline

Whether should we use onehot encoding for categorical data.

◆ WeightedSamplingWithoutReplacement()

template<typename T >
std::vector<T> xgboost::common::WeightedSamplingWithoutReplacement ( std::vector< T > const &  array,
std::vector< float > const &  weights,
size_t  n 
)

Variable Documentation

◆ dynamic_extent

constexpr std::size_t xgboost::common::dynamic_extent = std::numeric_limits<std::size_t>::max()
constexpr

◆ kBlockThreads

constexpr size_t xgboost::common::kBlockThreads = 256
constexpr

◆ kEulerMascheroni

constexpr double xgboost::common::kEulerMascheroni = 0.57721566490153286060651209008240243104215933593992
constexpr

The Euler-Mascheroni_constant.

◆ kPI

constexpr double xgboost::common::kPI = 3.14159265358979323846
constexpr

Constant PI.