Tuner component using Hyperband algorithm fails to write out best hyperparameters due to missing max_trials

[edi-bice]

System information

• Have I specified the code to reproduce the issue (Yes, No): Yes
• Environment in which the code is executed (e.g., Local(Linux/MacOS/Windows),
  Interactive Notebook, Google Cloud, etc): Kubeflow
• TensorFlow version: 2.15.1
• TFX Version: 1.15.1
• Python version: 3.9.2
• Python dependencies (from pip freeze output):
  keras 2.15.0
  keras-tuner 1.4.7

Describe the current behavior

import kerastuner

def _get_keras_hyperparameters() -> kerastuner.HyperParameters:
"""Returns hyperparameters for building Keras model."""
hp = kerastuner.HyperParameters()
# Defines search space.
hp.Choice('activation', ['relu', 'leaky_relu'])
hp.Choice('learning_rate', [1e-3, 1e-4, 1e-5], default=1e-4)
hp.Fixed('tune_epochs', 300)
return hp

tuner = kerastuner.Hyperband(
    hypermodel=_build_keras_model_transform,
    max_epochs=hparams.get('tune_epochs'), # Maximum number of epochs to train a model
    factor=3, # Reduction factor for successive halving
    hyperparameters=hparams,
    tune_new_entries=True,
    allow_new_entries=True,
    objective=kerastuner.Objective('val_mean_absolute_error', 'min'),
    directory=fn_args.working_dir, 
    project_name=configs.PIPELINE_NAME + '_tuning'
)

Fails to write out the tuning results and best hyperparameters due to comparison with missing max_trials (which is not provided in the Hyperband tuner)

File "/usr/local/lib/python3.10/site-packages/tfx/components/tuner/executor.py", line 127, in Do
write_best_hyperparameters(tuner, output_dict)
File "/usr/local/lib/python3.10/site-packages/tfx/components/tuner/executor.py", line 75, in write_best_hyperparameters
for trial_obj in tuner.oracle.get_best_trials(tuner.oracle.max_trials):
File "/usr/local/lib/python3.10/site-packages/keras_tuner/src/engine/oracle.py", line 677, in get_best_trials
if len(sorted_trials) < num_trials:
TypeError: '<' not supported between instances of 'int' and 'NoneType'
�[2K
Trial 725 Complete [00h 00m 19s]
val_mean_absolute_error: 3.9970703125
Best val_mean_absolute_error So Far: 1.5847426652908325
Total elapsed time: 05h 48m 32s
Results summary

keras-team/keras-tuner#1037

[pritamdodeja]

@edi-bice it appears that sorted_trials is not an integer. Do you have a reproducible example for this? I can help you solve this if needed.

[pritamdodeja]

A potential root cause I can think of is the hyperparameters are instantiated within the body of the model building function. Here max_epochs is being treated like a hyperparameter. To achieve the objective, it may make sense to instead compute the number of steps that should be run within the body of the model building function.