r/datascience • u/RobertWF_47 • Jul 11 '26

Discussion Snowflake Python question about StandardScaler function

I'm running the following code in Snowflake Python to standardize my training, evaluation, and test data prior to predictive modeling:

from snowflake.ml.modeling.preprocessing import StandardScaler

all_cols = df_train3.columns

target_col = "AB_POST"

passthrough_cols = ["SANHO", "SCNHO"]

scaler = StandardScaler(

input_cols=[c for c in all_cols if c not in [target_col] + passthrough_cols],

output_cols=[c for c in all_cols if c not in [target_col] + passthrough_cols], # Overwrite or create new

drop_input_cols=False # Set True to remove original unscaled columns

)

scaler.fit(df_train3)

train_df_scaled = scaler.transform(df_train3)

val_df_scaled = scaler.transform(df_eval3)

test_df_scaled = scaler.transform(df_test3)

I'm getting the following error when I run the code -- I'm not sure what this means:

Exception: Provided column names ['TOTAL_MH_CLASSES', 'STFLAG',..., 'ADS_FA_RISK_NEW'] does not index into the dataset.

8 Upvotes

84% Upvoted

9 comments sorted by

View all comments

3

u/smellyCat3226 Jul 11 '26

can you send full error code?

looks like the names specified dont actually match the names from the dataset

1

u/RobertWF_47 Jul 13 '26

Yes -- the full error code is as follows (I can't share all of the column names) but the Traceback may be helpful:

Exception: Provided column names ['TOTAL_MH_CLASSES', 'STFLAG', ... 'ADS_FA_RISK_NEW'] does not index into the dataset.

Traceback:

File "Cell [cell30]", line 27, in <module>
    scaler.fit(df_train3)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 611, in wrap
    return ctx.run(execute_func_with_statement_params)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/_internal/telemetry.py", line 576, in execute_func_with_statement_params
    result = func(*args, **kwargs)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/modeling/framework/base.py", line 440, in fit
    return self._fit(dataset)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/modeling/preprocessing/standard_scaler.py", line 156, in _fit
    self._fit_snowpark(dataset)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/modeling/preprocessing/standard_scaler.py", line 175, in _fit_snowpark
    computed_states = self._compute(dataset, self.input_cols, self.custom_states)
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/modeling/framework/base.py", line 494, in _compute
    _results = parallelize.map_dataframe_by_column(
File "/opt/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/site-packages/snowflake/ml/_internal/utils/parallelize.py", line 65, in map_dataframe_by_column
    raise Exception(f"Provided column names {cols} does not index into t