Note
Click here to download the full example code
Lightweight, versatile, and platform agnostic architecture¶
Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.
Quadratic Function Example¶
Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).
First of all, import optuna.
import optuna
In optuna, conventionally functions to be optimized are named objective.
def objective(trial):
x = trial.suggest_float("x", -10, 10)
return (x - 2) ** 2
This function returns the value of \((x - 2)^2\). Our goal is to find the value of x
that minimizes the output of the objective function. This is the “optimization.”
During the optimization, Optuna repeatedly calls and evaluates the objective function with
different values of x.
A Trial object corresponds to a single execution of the objective
function and is internally instantiated upon each invocation of the function.
The suggest APIs (for example, suggest_float()) are called
inside the objective function to obtain parameters for a trial.
suggest_float() selects parameters uniformly within the range
provided. In our example, from \(-10\) to \(10\).
To start the optimization, we create a study object and pass the objective function to method
optimize() as follows.
study = optuna.create_study()
study.optimize(objective, n_trials=100)
You can get the best parameter as follows.
best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Out:
Found x: 2.004100167287421, (x - 2)^2: 1.681137178483744e-05
We can see that the x value found by Optuna is close to the optimal value of 2.
Note
When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.
Study Object¶
Let us clarify the terminology in Optuna as follows:
Trial: A single call of the objective function
Study: An optimization session, which is a set of trials
Parameter: A variable whose value is to be optimized, such as
xin the above example
In Optuna, we use the study object to manage optimization.
Method create_study() returns a study object.
A study object has useful properties for analyzing the optimization outcome.
To get the dictionary of parameter name and parameter values:
study.best_params
Out:
{'x': 2.004100167287421}
To get the best observed value of the objective function:
study.best_value
Out:
1.681137178483744e-05
To get the best trial:
study.best_trial
Out:
FrozenTrial(number=94, values=[1.681137178483744e-05], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 794649), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 798406), params={'x': 2.004100167287421}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None)
To get all trials:
study.trials
Out:
[FrozenTrial(number=0, values=[53.25961232949523], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 432102), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 432373), params={'x': 9.297918355907747}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[41.97934012218632], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 432611), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 432811), params={'x': 8.479146558165382}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[47.74798169765851], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 433029), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 433232), params={'x': -4.909991439767383}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[75.06687384085649], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 433408), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 433601), params={'x': -6.664114140571815}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[0.2601575625127726], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 433772), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 433964), params={'x': 1.4899435692859342}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[5.468780533503443], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 434131), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 434328), params={'x': 4.338542395062241}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[2.0983380067595236], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 434508), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 434703), params={'x': 0.5514358810326954}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[85.96464985084232], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 434868), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 435071), params={'x': -7.271712347287438}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[27.204356858670476], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 435236), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 435441), params={'x': -3.2157796021947167}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[120.01751173401574], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 435606), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 435820), params={'x': -8.955250418589971}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[0.21128615678763077], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 435984), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 439919), params={'x': 2.459658739487928}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[0.21002672673395117], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 440115), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 443644), params={'x': 2.458286729825282}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[8.451417680227673], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 443835), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 447546), params={'x': 4.9071322089350655}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[6.05210399269371], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 447741), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 452794), params={'x': 4.460102435406646}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[18.54065490883601], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 453106), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 457768), params={'x': -2.305886077085181}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[0.18961112699994112], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 457975), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 462050), params={'x': 2.4354435979549374}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[22.580593147779183], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 462256), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 466578), params={'x': 6.751904160205589}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[12.018904431830453], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 466846), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 470425), params={'x': -1.4668291610390112}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[0.2561146382339627], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 470650), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 476645), params={'x': 2.5060776997991145}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[4.75870284467665], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 476862), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 480369), params={'x': -0.18144512758782216}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[20.786465025338742], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 480572), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 485310), params={'x': 6.559217589163599}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[0.3707129027425021], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 485658), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 490494), params={'x': 2.6088619734738754}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[0.8190144428983893], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 490714), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 497246), params={'x': 2.904994167328381}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[7.378763848912355], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 497666), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 502562), params={'x': -0.7163880151613751}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[16.819706137352682], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 502891), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 507811), params={'x': 6.1011835044719325}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[1.79949050429875], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 508054), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 513047), params={'x': 0.6585491047754488}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[2.100908536886279], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 513286), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 518359), params={'x': 3.4494511157283916}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[0.13326036178057382], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 518626), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 525832), params={'x': 1.634951562418665}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[0.5230264572425081], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 526054), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 529645), params={'x': 1.2767943188535449}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[48.67228759790232], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 529865), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 533535), params={'x': 8.976552701578504}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[26.356627081882944], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 533759), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 537324), params={'x': -3.1338705751005205}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[3.2054841721118764e-05], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 537536), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 541184), params={'x': 2.005661699543522}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[7.070488479710156], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 541398), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 544958), params={'x': -0.6590390143264457}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[10.077156400429272], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 545166), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 548789), params={'x': 5.174453716851023}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[0.12685657422670094], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 549143), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 552732), params={'x': 1.6438306944349346}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[0.7070139637668947], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 552970), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 556589), params={'x': 1.159158776125424}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[30.027679982226307], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 556795), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 560387), params={'x': 7.4797518175758935}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[0.1687373134820305], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 560623), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 564272), params={'x': 1.5892235236992867}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[3.742835518675958], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 564508), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 569271), params={'x': 3.9346409275821594}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[38.22623932402027], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 569522), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 573179), params={'x': -4.182737203215115}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[12.470139702597914], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 573426), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 577075), params={'x': -1.5313084972284585}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.11692382616673008], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 577313), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 582656), params={'x': 1.6580587387185775}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[1.0957426092748024], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 582881), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 586797), params={'x': 0.9532227508802065}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[0.11047916335096672], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 587014), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 590603), params={'x': 1.6676159399866373}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[3.4911855096697466], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 590813), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 594961), params={'x': 0.1315285633251584}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[0.006625779437793946], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 595170), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 599756), params={'x': 1.9186011091120159}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[2.094734932926883], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 599976), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 604062), params={'x': 3.447319913815492}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[11.680519682005707], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 604276), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 607888), params={'x': 5.417677527503979}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[0.003852362010400515], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 608093), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 611774), params={'x': 1.937932601066256}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[4.778881213406164], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 611984), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 616741), params={'x': 4.186065235395816}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[9.072411211111191], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 616958), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 620621), params={'x': -1.0120443574275584}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.004485284751942695], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 620828), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 625203), params={'x': 1.9330277314708342}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[3.6075670924953527], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 625413), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 629239), params={'x': 0.1006403467233088}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[2.3521217127572633], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 629458), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 633077), params={'x': 3.5336628419431904}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[0.015010179920281377], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 633287), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 636904), params={'x': 1.8774839605591114}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[0.045767845087115754], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 637116), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 640720), params={'x': 2.2139342073795487}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[0.9546275771014068], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 640930), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 644869), params={'x': 2.9770504475723896}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[0.03858212717552982], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 645085), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 649553), params={'x': 2.1964233366367902}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[2.05546855809764], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 649856), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 653495), params={'x': 0.5663094622277651}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[7.423865021441197], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 653707), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 657322), params={'x': 4.724677049017222}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[0.026053463236573024], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 657533), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 661368), params={'x': 2.1614108522887263}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.06539834503250866], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 662248), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 666091), params={'x': 2.25573100131292}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[1.4446212770071114], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 666323), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 669975), params={'x': 3.20192398969615}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[3.6803184685269175], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 670193), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 673802), params={'x': 0.08158438587283245}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[3.266170753414546], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 674138), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 677752), params={'x': 3.807255032753968}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[0.09284157455247317], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 677964), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 681582), params={'x': 2.3046991541709185}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[1.4878862711514713], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 681796), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 685845), params={'x': 0.7802105627808251}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[13.755085151128666], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 686058), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 689750), params={'x': -1.7087848618015937}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[6.000222950289198], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 689970), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 693599), params={'x': -0.44953525189763277}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[5.931685098481389], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 693814), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 697438), params={'x': 4.435505101304735}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[0.8062489223888858], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 697652), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 701289), params={'x': 2.8979136497397096}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.0005985044592382368], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 701500), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 705149), params={'x': 2.0244643507830933}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.013534913724667665], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 705522), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 709210), params={'x': 1.8836603518800763}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[1.043638043111002], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 709423), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 713079), params={'x': 0.9784139570692042}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[13.759000204641213], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 713289), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 716932), params={'x': 5.709312632367514}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[0.6584184716484933], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 717145), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 720808), params={'x': 2.8114298932430906}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[0.0067165076688846855], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 721141), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 726643), params={'x': 1.9180456976792268}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[0.32780346495999174], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 726879), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 730583), params={'x': 1.4274587657120303}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[2.426276224029091], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 730802), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 734497), params={'x': 0.4423491328192024}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[5.1833237817369575], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 734719), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 738376), params={'x': -0.27669141117916074}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[3.763075455631333], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 738617), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 742289), params={'x': 3.9398648034415524}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.002183686627900702], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 742534), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 746734), params={'x': 1.9532700671100343}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[0.0006763128078155507], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 746958), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 750652), params={'x': 2.026006014839178}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[0.8754917722766967], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 750876), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 754556), params={'x': 1.0643228268912954}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[0.4812127263605853], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 754777), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 758763), params={'x': 2.69369498078088}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[1.499577141393903], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 758983), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 762682), params={'x': 3.224572227920388}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[0.0405132584211614], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 762897), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 766590), params={'x': 1.7987209439083107}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[9.099466200515446], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 766808), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 770511), params={'x': -1.0165321480991123}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[0.4569886237099541], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 770726), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 774382), params={'x': 1.3239906630009064}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[128.8783495696771], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 774703), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 778622), params={'x': -9.352460066861152}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[2.3038840251847934], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 778842), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 782545), params={'x': 0.4821449261590245}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.007501511946169389], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 782764), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 786442), params={'x': 1.9133887308361701}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[0.478579733599188], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 786676), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 790396), params={'x': 2.6917945747107215}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[2.044095285257505], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 790635), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 794373), params={'x': 3.4297186035222125}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[1.681137178483744e-05], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 794649), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 798406), params={'x': 2.004100167287421}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[0.5744635983220682], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 798647), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 802309), params={'x': 1.242066230913236}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[2.771262599987949], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 802543), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 806343), params={'x': 0.3352890340999284}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[0.008853447825725997], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 806582), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 810283), params={'x': 1.9059072381863196}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[0.3207380492201666], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 810520), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 814217), params={'x': 2.566337398747572}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[1.1702728465574943], datetime_start=datetime.datetime(2020, 12, 21, 5, 56, 50, 814440), datetime_complete=datetime.datetime(2020, 12, 21, 5, 56, 50, 818133), params={'x': 0.9182085013471892}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]
To get the number of trials:
len(study.trials)
Out:
100
By executing optimize() again, we can continue the optimization.
study.optimize(objective, n_trials=100)
To get the updated number of trials:
len(study.trials)
Out:
200
As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:
best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Out:
Found x: 2.004100167287421, (x - 2)^2: 1.681137178483744e-05
Total running time of the script: ( 0 minutes 0.834 seconds)