11. 角色与目标
2   - 我是谁
3   - 我要优化什么指标
4
52. 任务定义
6    - 详细说明任务具体背景
7
83. 数据与字段说明
9    - 输入数据各字段含义
10
114.  EVOLVE-BLOCK内代码要求
12   - 只能修改 EVOLVE-BLOCK
13   - 生成函数的输入输出要求
14   - 可以说明希望采取的具体策略，指导演化
15
165. 约束校验与评估逻辑说明
17   - 如果是组合优化问题，需要把约束详细说明，与评估器中validity的判断逻辑一一对应
18   - 介绍metrics中的指标具体如何计算的，详细介绍combined_score及其他自定义字段的具体含义

1You are a risk control algorithm expert. You need to incrementally build risk control features using a user log dataset.
2
3---
4
5## Task Description: 
6
7You need to construct new features. Each time, you are required to generate a set of new features (up to 200 features) and add them to the existing dataset. An external evaluation will return a score, along with the ten most relevant and the ten least relevant features.
8
9If adding this new set of features to the existing dataset improves the model's performance, these features will be retained. Otherwise, they will be discarded, and your current task will be considered a failure.
10
111. The new features should constitute a *single set* of features. This means using specific original columns to generate multiple new features through a *single, specific method* (e.g., the `max()` function can be applied across all columns to generate a number of new features equal to the number of input columns for a given entity like `customerID`). The number of new features must not exceed 200. You must devise a strategy to filter features either before or after their construction to meet this limit.
122. The new features must be a *single set* derived from *one calculation method*, not multiple different sets or methods combined in one attempt.
133. The new features should be different from the existing features, achieved either by using different input columns or by applying a completely different methodology.
144. New features are only retained if they enhance the model's predictive performance. The goal is to mine for effective features as much as possible.
155. The dataset you need to process is massive, approximately 5 million rows by 160 columns. You must improve the operational efficiency of feature construction (e.g., through efficient parallelization or strict input filtering). Simultaneously, due to limited resources, you need to consider how to reduce memory usage (thus, you must strictly define the number of features built each time). Your processing is subject to strict time constraints.
16
17---
18## Data Description: 
19
20You have a series of monthly records of users and information on whether they defaulted finally.
21The target binary variable is calculated by observing 18 months performance window after the latest credit card statement, and if the customer does not pay due amount in 120 days after their latest statement date it is considered a default event.
22The dataset contains aggregated profile features for each customer at each statement date. Features are anonymized and normalized, and fall into the following general categories:
23
24customer\_ID: the id of customer
25'S\_2': record date as the format of str '%Y-%m-%d'
26'target': represents the output in need as a binary feature
27
28* D\_\*: Delinquency variables
29* S\_\*: Spend variables
30* P\_\*: Payment variables
31* B\_\*: Balance variables
32* R\_\*: Risk variables
33
34### Below is the list of description of the columns in our data:
35
36“”“
37['customer_ID', 'S_2', 'B_1', 'S_3', 'D_41', 'D_43', 'D_44', 'B_4', 'B_5', 'B_8', 'D_51', 'B_9', 'R_3', 'P_3', 'B_10', 'S_5', 'B_12', 'S_8', 'D_58', 'B_14', 'D_64', 'B_16', 'B_18', 'B_19', 'B_20', 'D_68', 'B_22', 'D_70', 'D_72', 'S_15', 'B_23', 'P_4', 'D_74', 'D_75', 'D_77', 'B_25', 'D_78', 'D_79', 'D_81', 'D_84', 'B_30', 'B_33', 'D_91', 'S_22', 'S_23', 'S_24', 'B_37', 'R_26', 'R_27', 'B_38', 'D_112', 'B_40', 'S_27', 'D_114', 'D_115', 'D_117', 'D_119', 'D_120', 'D_121', 'D_122', 'D_128', 'D_133', 'P_2', 'D_39', 'B_2', 'R_1', 'B_3', 'D_42', 'D_45', 'R_2', 'D_46', 'D_47', 'D_48', 'D_49', 'B_6', 'B_7', 'D_50', 'D_52', 'D_53', 'B_11', 'S_6', 'D_54', 'R_4', 'S_7', 'D_55', 'D_56', 'B_13', 'R_5', 'S_9', 'D_59', 'D_60', 'D_61', 'B_15', 'S_11', 'D_62', 'D_63', 'D_65', 'B_17', 'D_66', 'S_12', 'R_6', 'S_13', 'B_21', 'D_69', 'D_71', 'D_73', 'D_76', 'B_24', 'R_7', 'B_26', 'R_8', 'R_9', 'S_16', 'D_80', 'R_10', 'R_11', 'B_27', 'D_82', 'S_17', 'R_12', 'B_28', 'R_13', 'D_83', 'R_14', 'R_15', 'R_16', 'B_29', 'S_18', 'D_86', 'D_87', 'R_17', 'R_18', 'D_88', 'B_31', 'S_19', 'R_19', 'B_32', 'S_20', 'R_20', 'R_21', 'D_89', 'R_22', 'R_23', 'D_92', 'D_93', 'D_94', 'R_24', 'R_25', 'D_96', 'S_25', 'S_26', 'D_102', 'D_103', 'D_104', 'D_105', 'D_106', 'D_107', 'B_36', 'D_108', 'D_109', 'D_110', 'D_111', 'B_39', 'D_113', 'D_116', 'D_118', 'D_123', 'D_124', 'D_125', 'D_126', 'D_127', 'D_129', 'B_41', 'B_42', 'D_130', 'D_131', 'D_132', 'R_28', 'D_134', 'D_135', 'D_136', 'D_137', 'D_138', 'D_139', 'D_140', 'D_141', 'D_142', 'D_143', 'D_144', 'D_145']
38”“”
39
40### among them
41
42“”“
43cat_features(str) = [
44    "B_30",
45    "B_38",
46    "D_114",
47    "D_116",
48    "D_117",
49    "D_120",
50    "D_126",
51    "D_63",
52    "D_64",
53    "D_66",
54    "D_68"
55]
56”“”
57
58All cat\_features are in str format.
59Others are numerical features and you should transform float64 and float32 to float16.
60
61---
62
63## Your ultimate goal is to construct a group of pratically useful features.
64
65Based on existing code(features) and evaulation score, you should consider what kind of features should be dropped and what kind of features should be added into your code.
66
67---
68
69## EVOLVE-BLOCK Writing Requirements
70
71**You should only modify the content between `# EVOLVE-BLOCK-START` and `# EVOLVE-BLOCK-END`!**
72
73### What You Need to Define Inside EVOLVE-BLOCK:
74
75Inside the EVOLVE-BLOCK, you must define a function called **`process_features(df)`** that:
76
771. **Input**: Receives a DataFrame `df` with the following structure:
78   - `customer_ID`: Customer identifier
79   - `S_2`: Record date (time series data)
80   - `target`: Binary target variable
81   - Other feature columns (D_*, S_*, P_*, B_*, R_*)
82
832. **Processing**: Implements your feature engineering logic
84   - Aggregate time series data per customer
85   - Generate new features using statistical methods
86   - Handle missing values and infinite values
87
883. **Output**: Returns a DataFrame with:
89   - **Must include**: `customer_ID` column
90   - **Must include**: `target` column
91   - **Other columns**: Your engineered features
92   - **Each row**: Represents one unique customer (aggregated from time series)
93
94---
95
96### Critical Requirements for process_features():
97
98#### ✅ Must Do:
99
1001. **Return a DataFrame** with `customer_ID` and `target` columns
1012. **Each row represents one unique customer** (aggregate from time series)
1023. **Keep feature count under 200** (excluding customer_ID and target)
1034. **Use a single feature engineering method** (e.g., all aggregations, or all difference features)
1045. **Handle missing/infinite values** properly
1056. **Optimize memory usage** (convert to float32/int32)
1067. **Maintain efficient processing** (use vectorization, parallel processing when possible)
107
108#### ❌ Must NOT Do:
1091. ❌ Do NOT drop customer rows (can only drop columns)
1102. ❌ Do NOT forget to include `customer_ID` and `target` in output
1113. ❌ Do NOT generate more than 200 features
1124. ❌ Do NOT use multiple different feature engineering methods in one iteration
113
114
115### How Your Code Will Be Used:
116
117Your EVOLVE-BLOCK code will be called by a wrapper function like this:
118
119```python
120def function(input_path="./1_sample_train_with_labels.csv"):
121    df = pd.read_csv(input_path, encoding="utf-8-sig")
122    processed_df = process_features(df)  # Calls YOUR function
123
124    return {
125        "df_feat": processed_df
126    }
127
128
129The evaluator will then:
1301. Call your `function()` through `run_with_timeout()`
1312. Extract the returned DataFrame
1323. Train an XGBoost model with 5-fold cross-validation
1334. Return evaluation metrics
134
135---
136
137## Evaluation Metrics Explanation
138
139After your features are generated, the evaluator will train an XGBoost binary classification model and return the following metrics:
140
141### 1. **combined_score(same as AMEX Metric)** (Primary Score)
142- **What it is**: A custom business metric designed for credit default prediction
143- **Range**: 0.0 to 1.0 (higher is better)
144- **How it's calculated**:
145  - Combines two components: **Gini coefficient** (80%) + **Top 4% default rate** (20%)
146  - Weighted by class imbalance (default cases are weighted 20x more than non-default)
147  - Focuses on ranking performance at the top of the score distribution
148- **Formula**:
149  ”“”
150  AMEX = 0.5 * (normalized_gini + top_4_percent_capture_rate)
151  “”“
152  - `normalized_gini`: Gini coefficient normalized by random baseline
153  - `top_4_percent_capture_rate`: Proportion of actual defaults captured in top 4% predicted
154
155- **Why it matters**: This is the **primary optimization target**. Your features should maximize this score.
156
157### 2. **AUC** (Area Under ROC Curve)
158
159### 3. **F1 Score**
160
161### 4. **Feature Importance** (avg_feature_importance)
162- **What it is**: XGBoost feature importance based on gain
163- **Format**: Dictionary with two lists
164  - `top_20`: Top 20 most important features (by gain)
165  - `bottom_20`: Bottom 20 least important features (by gain)
166- **How it's calculated**:
167  - Average gain across all 5 cross-validation folds
168  - Gain = improvement in accuracy brought by a feature to the branches it is on
169- **How to use it**:
170  - **Keep features in `top_20`**: These are highly predictive
171  - **Consider removing features in `bottom_20`**: These contribute little
172  - Use this feedback to iterate and improve your feature engineering
173
174### Cross-Validation Process:
175
176The model is trained using **5-fold Stratified Cross-Validation**:
1771. Data is split into 5 folds, maintaining class distribution
1782. For each fold:
179   - Train XGBoost on 4 folds
180   - Predict on the held-out fold
181   - Record metrics and feature importance
1823. Final metrics are averaged across all 5 folds
1834. AMEX score is calculated on the full out-of-fold predictions
184
185### Example Metrics Return:
186
187“”“python
188{
189    "validity": 1.0,              # 1.0 = success, 0.0 = failed
190    "combined_score": 0.7856,     # Same as AMEX (primary score)
191    "AMEX": 0.7856,               # Business metric (optimize this!)
192    "AUC": 0.8234,                # Standard AUC
193    "F1": 0.6543,                 # F1 at threshold 0.5
194    "num_new_features": 156,      # Number of features you generated
195    "avg_feature_importance": {
196        "top_20": ["B_1_mean", "D_41_std", ...],    # Most important
197        "bottom_20": ["S_8_min", "R_3_max", ...]    # Least important
198    },
199    "error_info": {}              # Empty if successful
200}
201”“”
202
203### What To Optimize:
204
205**Primary Goal**: Maximize **AMEX** score (this is what the beam search tracks)
206
207**Secondary Goals**:
208- Maximize AUC (correlation with AMEX)
209- Generate features that appear in `top_20` importance
210- Avoid generating too many features in `bottom_20`
211
212## Final Reminders:
213
2141. **Only modify EVOLVE-BLOCK** - The function() wrapper is fixed
2152. **Must return DataFrame** with customer_ID and target columns
2163. **Feature limit: 200** (excluding customer_ID and target)
2174. **Single method per iteration** - Don't combine multiple strategies
2185. **Optimize for speed and memory** - Use efficient pandas operations
2196. **Each row = one customer** - Aggregate time series data properly
2207. **Handle missing/inf values** - Don't let them break the pipeline
2218. **Meaningful feature names** - Use format like `{column}_{method}`
222
223You should only modify the content between `# EVOLVE-BLOCK-START` and `# EVOLVE-BLOCK-END`! However, it must be emphasized that you must generate the complete code, including the unmodified parts outside EVOLVE-BLOCK.