How to create column description (CD) file for Catboost

Question

I'm declaring a CD file for reading it from:

pool = Pool(features_file, CDfile)

Where CDfile contains text and <\t> delimiters:

0   Target
1   Categ   cat_reg
97  Categ   cat_dow
98  Categ   cat_nweek
99  Categ   cat_month
100 Categ   cat_hour
101 Categ   cat_is_month_start
102 Categ   cat_is_year_end
103 Categ   cat_is_year_start
104 Categ   cat_anomaly2016

I am having such a result: Factor False in column 102 and row 1 is declared as numeric and cannot be parsed as float. Try correcting column description file.

Here is slice for whole features= DataFrame()

len(cat_features), len(features.columns) 9 105

   cat_columns            cat_positions values
    cat_reg                         1   1075
    cat_dow                        97      5
    cat_nweek                      98     17
    cat_month                      99      4
    cat_hour                      100      1
    cat_is_month_start            101  False
    cat_is_year_end               102  False
    cat_is_year_start             103  False
    cat_anomaly2016               104      0

What's wrong with column 102? Why is it declared as numeric (not categorical) feature ?

score 0 · Answer 1 · answered Dec 27 '17 at 22:26

The bag was a redundant unprintable char in a CD file. I didn't catch which one. Here is the code for generating CD file.

def catboostCD(fname, cat_features, cat_features_names, sep='\t' ):
    with open(fname,"w") as fout:
        fout.write('0{0}Target'.format(sep))
        fout.write(''.join(['\n{0}{1}Categ{1}{2}'.format(el[0], sep, el[1]) for el in zip(cat_features, cat_features_names)]))

where

cat_features = np.ravel( np.where( np.char.startswith(list(features.columns), prefix='cat_') ) )
cat_features_names = features.columns.values[ cat_features]

score 0 · Answer 2 · answered Apr 08 '19 at 15:45

CatBoost Python library has a create_cd() method (https://catboost.ai/docs/concepts/python-reference_utils_create_cd.html)

Here is an example of how to use it to create a file with different types of columns:

from catboost.utils import create_cd
feature_names = {
    4: 'Categ1',
    5: 'Categ2',
    12: 'Num1'
}

create_cd(
    label=0,
    cat_features=(4, 5, 6),
    weight=1,
    baseline=2,
    doc_id=3,
    group_id=7,
    subgroup_id=8,
    timestamp=9,
    auxiliary_columns=(10, 11),
    feature_names=feature_names,
    output_path='train.cd'
)

How to create column description (CD) file for Catboost

2 Answers2