NeurIPS 2022: CityLearn Challenge
CityLearn-2-EvolveCity: A Swarm and Evolutionary Approach
Getting Started with Genetic Algorithm and Particle Swarm Optimization for CityLearn
This Notebook shows how to use evolutionary algorithms in the CityLearn reinforcement learning environment to optimize a rulebased agent for Energy Management in Buildings.
NatureCity: Swarm and Evolutionary Algorithms for CityLearn (Action24)¶
Author: Chia E Tungom
Email: bamtungom@protonmail.com
Date: Aug-02-2022This Notebook demostrates how we can use a metaheuristic particularly a Genetic Algorithm or Particle Swarm Optimization In the CityLearn Environment. To tackle the CityLearn Problem, it Lays a foundation for those aiming to explore an Evolutionary or Swarm optimization approach for Energy Management.
To achieve this I do the following:
- Formulate an objective function (single objective problem)
- Utilize Opensource implementations of GA and PSO (these Libraries contain other algorithms)
Note: The objective function is very very expensive and will take time to run one evaluation (e.g 10 particle running in 10 generations take approximately 10 minutes). This is because Swarm and evolutionary algorithms are population based approaches and evaluate the performaance of a function (one run of the environment) at evvery generation. More so, in one generation, there are given number of candidate solution corresponding to the warm population. Our implementation evaluate the environment after an entire run. A simpler approach could be to evolve the solution after every step (we dont implement this approach)
If you are still starting out with the CityLearn environment or RL, check this Notebook for an understanding of the environment.
Some parts of the Notebook are taken from here to define our environment.
To run this notebook successfully, you will need to download or clone the citylearn starter kit and place this notebook in the the root directory
Note that the objective function can be formulated as biobjective function The libraries used for implementation are pymoo and jmetalpy
Leave a question if you need clarification and/or give a like or bitcoin me :( if you find this useful
Lets Goooooo!!!
1. Build Environment¶
In [1]:
# import dependencies
import numpy as np
import time
from agents.orderenforcingwrapper import OrderEnforcingAgent
from citylearn.citylearn import CityLearnEnv
from citylearn.agents.sac import SAC
from agents import rbc_agent
/Users/chemago/opt/anaconda3/envs/CityLearn2022/lib/python3.7/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html from .autonotebook import tqdm as notebook_tqdm
In [2]:
# Custom configure enviroment
class Constants:
episodes = 3
schema_path = './data/citylearn_challenge_2022_phase_1/schema.json'
def action_space_to_dict(aspace):
""" Only for box space """
return { "high": aspace.high,
"low": aspace.low,
"shape": aspace.shape,
"dtype": str(aspace.dtype)
}
def env_reset(env):
observations = env.reset()
action_space = env.action_space
observation_space = env.observation_space
building_info = env.get_building_information()
building_info = list(building_info.values())
action_space_dicts = [action_space_to_dict(asp) for asp in action_space]
observation_space_dicts = [action_space_to_dict(osp) for osp in observation_space]
obs_dict = {"action_space": action_space_dicts,
"observation_space": observation_space_dicts,
"building_info": building_info,
"observation": observations }
return obs_dict
In [3]:
# CityLearn Environment
env = CityLearnEnv(schema=Constants.schema_path)
2. Define Objective Function [24 Hours Energy Control]¶
We model the problem as a 24 hour charge or discharge of electricity from the solar PV (time of day like night, morning, afternoon can also be used). This means that at every hour of the day, we want to determine how much charge or discharge of electric energy we want from the solar PV. Therefore our decision variable is a 24 length 1D array with upper and lower dimension of 1 and -1 respectively (see CityLearn action space).
There are 24 hours in a day so our intuition is that at every hour, we can decide to charge or discharge our battery which correspond to an action in the environment.
Our decison variables are [a, b, ... , x] where a is the action we take at hour 1 and x the action we take at hour 24.
In the Objective function (onbFunction defined in function) you can define
- Episodes: 1 is recommended for this case
- Days: Number of Days to run the simulation. We set 30 default corresponding to a months
- The number of hours in a day is 24 and our current setting searches for the same control for all days therefore for faster running, we can set the Days
- to 1 or
- 7 to generalize for a week
- The number of hours in a day is 24 and our current setting searches for the same control for all days therefore for faster running, we can set the Days
- RandnDay: True/False, We can also randomly use any day to e.g set day to one pick a random day in a year for trainiing or 7 days are randomly pick 7 days
Positive values stand for Charge while Neagive values stand for discharge.
In [4]:
def TakeAction24(Observation, Actions24):
""" takes an observation and Actions for 24 hours and returns
an action for the given hour
======================
Parameters
Observation: observation for one building
Actions: 1d numpy array of size 24
"""
hour = Observation[2] # Hour index is 2 for all observations
action = 0.0 # Default value
# for time, act in enumerate(range(actions)):
# # time plus one because hour starts from 0
# if hour == time + 1:
action = Actions24[hour-1]
action = np.array([action], dtype=float)
return action
def objFunction(Actions24, Env, Episodes = 1, Days = 30, RandomDay = True):
""" takes a 24 hour action array and returns it's evaluation performance
======================
Parameters
Action24: 1d numpy array of size 24
Env: CitiLearn Environment
RandomDay: bool
when training we to randomly pick a day (This should be convient for training evolutionary algorithms with Days = 1 in our current setting )
"""
episodes = Episodes
# Skipping to shorten training time
training_steps = 24*Days
for episode in range(1, episodes+1):
states = Env.reset()
done = False
score = 0
moves = 0
skipping = True
print(" STARTING EPISODE ", episode)
while (not done) and skipping:
#env.render()
try:
if RandomDay == True:
# Start Observation on some random day (NOT IMPLEMENTED)
action = [TakeAction24(Env.observation_space[building].sample(), Actions24) for building in range(len(states))]
states, reward, done, info = Env.step(action)
score+=reward
moves += 1
else:
# Take an action based on Observation
action = [TakeAction24(Observation, Actions24) for Observation in states]
states, reward, done, info = Env.step(action)
score+=reward
moves += 1
except:
print("UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON")
states = Env.reset()
continue
# print(done, skipping)
if moves >= training_steps:
# print(" TERMINATING AT SET STEP ", episode)
skipping = False
# print('Episode:{} Score:{} Out of:{}'.format(episode, score, moves))
print('Episode ::::: {} Evaluation :::: {}'.format(episode, Env.evaluate()))
Env.close()
return sum(env.evaluate())/2
# Acts24 = [random.uniform(-1,1) for _ in range(24)]
# About 6seconds for one evaluation
# objFunction(Acts24, env, Episodes=1, Days=5)
3. Optimization Using pyoo (A library for Multi-objective Optimization in Python)¶
To optimize using this library, we need to define our objective function as recommended by the developers. We start by adpating our objective function.
The documentation was a not very clear to me and I resorted to looking into the source code on github. This was especially useful for looing into the final results (the documentation and examples don't show muc)
In [5]:
from pymoo.core.problem import Problem
class ProBlem(Problem):
def __init__(self, Env, Episodes = 1, Days = 10, RandomDay = False):
# self.Actions24 = Actions24
self.Env = Env
self.Episodes = Episodes
self.Days = Days
self.RandomDay = RandomDay
super().__init__(n_var=24, n_obj=1, n_ieq_constr=0, xl=-1.0, xu=1.0)
def _evaluate(self, x, out, *args, **kwargs):
# print(x[0])
# print(len(x))
out["F"] = [objFunction(x[i], self.Env, self.Episodes, self.Days, self.RandomDay) for i in range(len(x))]
# out["G"] = 0.1 - out["F"]
problem = ProBlem(env, Days = 30, Episodes=1, RandomDay=False)
# Problem
# problem??
4. Genetic Algorithm¶
We expeimented on two variants of GA but NSGA2 was more promissing. Its also worth noting that NSGA2 was designed for Multi-Objective problems but works well for us here. If you run the algorithm Long enough, you will get results close to the current top of 0.67ish. An overnight run will suffice
Parameters for NSGA2 are
- population size: 100
- number of generation: 200 (the longer the better. u can also add a stopping criteria)
For details See: Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. A. M. T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2), 182-197.
Note: I ran into crashes coming from the CityLearn environment indicating that storage capacity cannot be less than zero (couldnt get why). I added a try/except line in my objective function to do away with this error by resetting the environmet. I don't fully understand the consequences (hit me up if you find out)
If your results we run in model( name for your optimization model), get the final results after running the algorithm you can run
model.pop[index].X: index is one member in the final population and X is it's genetic composition AKA solutionmodel.pop[index].F: F give the function value
In [6]:
from pymoo.algorithms.moo.nsga2 import NSGA2
from pymoo.problems import get_problem
from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter
from pymoo.algorithms.soo.nonconvex.ga import GA
from pymoo.optimize import minimize
In [7]:
# Using GA
# res = minimize(problem, GA(pop_size=100, verbose=True), termination=("n_gen", 10), seed=1)
# print('Threads:', res.exec_time)
# pop 10 and 10 gnerations completes in 9 mins
In [8]:
# Using NSGA2
algorithm = NSGA2(pop_size=100, verbose=True)
res = minimize(problem, algorithm, termination = ('n_gen', 200), seed=1, verbose=False)
STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.524832745891357, 1.7068377757963087) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.1319795592090927, 2.179550811583186) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5460837532771707, 1.545279737156711) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4903653672787063, 1.6285465200902383) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.158467136002888, 2.258112984352848) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7339243366102517, 1.6820512302489286) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9021745130920356, 1.8156255040959057) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0064084510512896, 1.9357315403364612) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3901385182919526, 1.4418648153683211) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5262900198422966, 1.688492384736818) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5778995134531437, 1.8313250730391726) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8329868458772032, 1.6896862170201121) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2594911926802281, 1.5336222570991007) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6897194150960384, 1.7036044259630547) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2640878212356255, 1.308488814190818) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.018615125527451, 1.8227697092779285) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3099363112707212, 1.5173021403526525) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4596983051161159, 1.5619199767219003) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3929757695296643, 1.736499375802727) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.3445547658220423, 2.2651948981942622) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9103304767314204, 1.7153375477712545) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8008483298461175, 1.5936718052540892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6533817007541292, 1.6925532233666438) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7357282712154156, 1.9211334393568247) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8983665992169583, 1.956545592767675) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.588234271980603, 1.6924599264351752) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8592622251379864, 1.8551220721699861) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.250074428584765, 2.2074995527934167) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4767608314656826, 1.660367340435604) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9241888746496987, 1.7938420713726284) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8200068506400195, 1.819154171648038) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7655910731574809, 1.7638972750086765) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.741489016583519, 1.6792825740362671) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7201868322670342, 1.9009294413213358) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5226583562559548, 1.6405725711352472) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9238126644203841, 1.9081769544419038) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.774403037813021, 1.654345685802231) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9144773649943758, 1.8796180246507093) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3936049671761503, 1.416769667589732) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4943891306300878, 1.5689104439278225) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.487049459207883, 1.6269103874649877) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7358093098813903, 1.6827593691045435) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8674158540378494, 1.7140232710627008) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7343836791021399, 2.045277010825979) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.2289407077532344, 2.0281751942328095) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8682283033417055, 1.7962307449846253) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.932775071554423, 1.7213082049151645) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4604447041123538, 1.483371003548367) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.507848776949875, 1.6198043027693443) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0619112713599486, 1.8903581713845947) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9413804395955363, 1.7330953382875849) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9007180601774247, 2.1985365408407227) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0113271514822437, 1.9996493519593148) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7294723800354237, 1.6206444118194876) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5644574611621616, 1.7846263342034907) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5112688742316667, 1.6611407442960733) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9240068054075492, 1.9020094666099745) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0457673502291467, 2.0489473416943746) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3551877969626682, 1.527182052761619) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8744314828350175, 1.715323479297415) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.1822048122147337, 1.985096740226956) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6894725137082973, 1.6205170880733284) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4228393021569055, 1.5379412811892519) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8470738768836774, 1.9065097508748063) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7326411710825456, 1.7961080951210047) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9476009418734028, 1.874213227954133) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9133026258270864, 1.9376513890526896) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4442893093104265, 1.673753398197652) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7142945800489, 1.8089875182771622) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9815757465840083, 1.8881391672657635) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5975224460440494, 1.6485096974992293) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7484088363699206, 1.7451180637553423) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.617391976360167, 1.6219627366427718) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8021307125908614, 1.7738077101573002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1105821135950045, 1.1450551366427348) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8265506998080232, 1.8474575559386959) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0177732217696844, 1.8603917262710399) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.2223358272766354, 2.1365824845930805) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.535741887985529, 1.4345109205259512) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8574970927082388, 1.7755632584084688) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.161433446487556, 2.0453326963444645) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9731502149156175, 1.8378337174289499) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9037731127827875, 1.746764526208836) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8660239161070271, 1.780317648695207) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0574489171103814, 1.8416240895002294) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.957112879638004, 1.8851397100224307) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.401465710213974, 1.45277916919727) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0952948391009074, 1.9251921171512931) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8996542472905065, 1.7834138471042842) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5012799678049151, 1.4389892021271988) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5416649847058304, 1.6874538579125662) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7340635118746235, 1.6150703419416381) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5638063689623674, 1.6096257074982285) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8537211567476064, 1.6009551343686377) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9447036725685742, 1.8739701370813677) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0757026894465818, 1.8117353406140566) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7528438384597125, 1.6900858259405651) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6389929854522258, 1.7460966350292781) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4619064200217091, 1.574842185519417) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7493306670481281, 1.7410451788352852) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6665780676181625, 1.7236808342575258) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5166424114545516, 1.638868060889415) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8793157688533106, 2.0764954824976067) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.311912789767544, 1.360555854644866) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3919328986397008, 1.4762288167633175) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7397849195112878, 1.7129771216879361) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2100525256357535, 1.3480124506299747) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4645586143149925, 1.5939941444425259) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6606210527209666, 1.7279680851521848) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5162386975011508, 1.6840727340845791) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4995428242800048, 1.6114193229516809) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5655505388981692, 1.616695178836432) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4811251228259743, 1.5011428551544908) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.940187897867869, 1.9064949244220504) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5994412235429345, 1.685430809113763) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0745753356515753, 1.9171932790687114) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7955433712885331, 1.5793258588234667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8902050648519, 1.8707228000129468) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6966433885681864, 1.8084342729195382) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8822731647264723, 1.8275334229756208) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7309559668905319, 1.6847060078988703) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.551423502552752, 1.634234253949685) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.2031671682601304, 2.1110073452422022) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5112549218956683, 1.6494416311996098) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4296505333716272, 1.5177631737706752) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8947531529541557, 1.7991619494388433) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9017648563308123, 1.8181788847695481) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4309531823165655, 1.5425411795871722) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9965157368642628, 1.6497399567949769) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7406509727785509, 1.5768017945006945) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9069595373284607, 1.9434431890294612) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4921124457447215, 1.6322267397892836) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8117610191482552, 1.7578440593619242) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2986117375736337, 1.4012056219125544) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5679745987544413, 1.6923524443545102) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.139368913173612, 1.9804041164805297) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9439872969345362, 1.8747438266679246) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4549276477530502, 1.4592488968469282) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3251197733789382, 1.6464440701958833) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5031516830467135, 1.560067191507632) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.289476187138525, 1.3652748916469537) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5739937495400005, 1.6943956934744753) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3198327535670356, 1.3647161789330933) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7181136847301637, 1.6868225367372691) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.472805290581467, 1.5498019460234966) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4715928461818455, 1.5730847257993807) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3585961951823735, 1.6324466033326754) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1098840112628257, 1.0747044756233686) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.37713250688384, 1.3556845524823096) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.415232800431626, 1.5867517731383465) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.806801601880402, 1.8584948679772348) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4575822734803872, 1.602542171998173) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.758995616329487, 1.815521385230467) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.899654156579755, 1.7834607674618705) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0179126698975245, 2.0137592106146145) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.927350950487531, 1.7987297942452525) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6215161761980317, 1.6256488920464234) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.343999825048105, 1.4325361048215) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9528577448757862, 1.8855439382516) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2295726846385404, 1.2967022592826232) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6212562619462858, 1.873340728770982) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4228393021569055, 1.5379412811892519) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7197498895777381, 1.7721010404158088) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5221813694687512, 1.6073758352783751) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.53559071226137, 1.6455428707819426) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7375723498828255, 1.6640260246557392) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8848498021179922, 1.8669277676640992) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8827509950942445, 1.7700370499144398) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7853009915165623, 1.714809334025446) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7124554869441968, 1.8409783839334557) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6258083087052981, 1.657993991499398) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6167458601955533, 1.621602939398673) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8056521632052895, 1.6678244066502401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3811628113168035, 1.3353109480365573) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0608388344535191, 1.1418201060822983) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5789857027005796, 1.8317980201275796) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6117778606939255, 1.7089641234839321) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7700356807973519, 1.816437145464378) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.426260656917837, 1.6326403066833544) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7255667333703857, 1.681995282801517) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7540675321658143, 1.6077655586193633) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5995089820622368, 1.826828720164778) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3620883211344859, 1.5385212634184804) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8269645013855658, 1.8270242423594576) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4914590805429886, 1.6369320845084652) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3811980854247419, 1.301602780643796) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6970746662549727, 1.9031675154504675) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3061513268986435, 1.4194881092028087) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9711950114638837, 1.8969405544368108) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9014833087037915, 1.667356766621863) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9047012772758025, 1.716070994635603) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6452395295633175, 1.5328975559351206) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1455825874689094, 1.2235801530496815) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.352040732886632, 1.4721720065264463) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7740202614503706, 1.8201694271899704) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5811803613069213, 1.825804292676262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8138775839914314, 2.043615206907926) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6056997699507218, 1.7076390351140094) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.259371834396783, 1.533540419608704) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.203102505045077, 1.5066134004272875) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7123431633853068, 1.8497964617671037) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7526187752996942, 1.6681469559522564) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.288851682466202, 1.5424500228240137) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5162386975011508, 1.6840727340845791) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.892149505491238, 1.6464009049385735) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3450495634391197, 1.3942444121480975) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4638472899256127, 1.5971028324941223) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1581385739786294, 1.2297344896658569) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3996803439168422, 1.5599150427623565) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2828211891366759, 1.4509593704776982) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5008502889472346, 1.7325251092028302) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.210031914969043, 1.34798998149455) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1027707374330353, 1.126168123679893) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4393808497247755, 1.526854581870447) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.402286700019241, 1.3982427716779087) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.839620906243752, 1.7629575188935511) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7923252367011542, 1.8871257658342855) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.807437857299448, 1.9457943226039593) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6665119075546517, 1.6335972805375012) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2486892629206372, 1.294989020686053) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.519427277445317, 1.6497111758734195) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4041713517583505, 1.5535635081188885) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4282907847201292, 1.482285552977186) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4690907798688415, 1.5658408144793023) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5907291780752466, 1.8272209071056102) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4050554964065631, 1.6736395564176538) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4272581464160305, 1.7100551201060885) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6213918899533724, 1.7185182539612769) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7736607057503835, 1.9019870268980055) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3571203431446452, 1.5457851181190776) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5173002712083974, 1.6719423450262845) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3199266745593463, 1.4759143796058278) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7772533346450736, 1.8739882745792837) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5278944117086906, 1.6289860235033584) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2702673983530275, 1.5123996299414058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9049121529303392, 1.8774810363444456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3163921330094044, 1.4679723527807866) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5144191635231192, 1.6248628954251128) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4402893076367094, 1.6736192341439047) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4314761120298103, 1.5031702747675648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1937266086116183, 1.1909509931227862) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6778567800628144, 1.5215511826323753) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5202144987191262, 1.6191051483868113) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.651496462403018, 1.621722284810489) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.745615157468271, 1.7393946802756335) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2905483747220812, 1.3257313301935083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1140437092527848, 1.0785551811340668) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4304948453111033, 1.6271773969056629) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3969400003937693, 1.4971949577078636) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.838356147331745, 1.8930967438886923) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3953999566168747, 1.5687527433090198) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1376934045087224, 1.1180116926295185) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.1005284080617597, 1.997181857624094) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.29639856233792, 1.373124301585291) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.247520456484481, 1.3378088031585595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5561791478224385, 1.6269944657131257) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.482812394287682, 1.5857744668217766) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7739084520959607, 1.8412974893086556) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5221760533053819, 1.6611069473667013) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4099608924575429, 1.5177372199479777) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.560243198293548, 1.7556446153167855) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3390887265576643, 1.431534016758071) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.21785250546121, 1.2884932887452567) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.648814708282664, 1.7214806564609109) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4716987046932082, 1.558684885364144) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4575142035261086, 1.5963178233949329) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4111535350603162, 1.5729077849547646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2863118951973156, 1.4637145974940977) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1084542856327482, 1.2148038625582005) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.010991217507171, 1.095942164484487) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4542917389012326, 1.535415413544523) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5337923935611413, 1.6395666203031325) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4158292740659582, 1.7030637588250332) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6741119885971794, 1.523570535814482) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.237519013880535, 1.2938658586167207) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4696105289591197, 1.6102123230083756) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4221358588481687, 1.5075583763555271) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6104681820731155, 1.7177668895509353) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2810105451429452, 1.4991266316147194) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6709712106864543, 1.954144655353131) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3326419237205058, 1.4294772457498566) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5011047153766857, 1.6368204622886653) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4495290980124462, 1.6572947143032766) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5329515707265249, 1.4702169746115648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7970194134466144, 1.6308760670834144) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4379138123650799, 1.548849642784296) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3811980921590696, 1.3016027845144258) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.052304493514755, 1.91540049385631) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3838736683710589, 1.5128704994361357) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5089921008203446, 1.5614864280681466) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0954140969968673, 1.2043401944353604) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7668408536736806, 1.8112298555928361) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2271803763598492, 1.3547188808859907) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.517169981191882, 1.4105735790089293) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6098251114451367, 1.6767022857975171) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1884738613511632, 1.3610085792378488) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7089452395031761, 1.647862760004308) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1252069259440995, 1.1894403971719456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5622677697388443, 1.3947389792462568) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8980176385621086, 1.6802011623861495) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.50665662887682, 1.6388171343280493) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4063899175106607, 1.535546563876065) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5081266654731056, 1.533930221033411) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5228590310180918, 1.6347123318790062) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.761007568659238, 1.8873079461287678) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5175998589450976, 1.6831208360043355) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3811980916710835, 1.3016027842339446) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2847755191925716, 1.3201138275878535) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3136734325524193, 1.630828155204197) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1730704030868868, 1.27075368069757) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4974845443163587, 1.6725455314321744) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4089855079943725, 1.3130992195605324) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3920993673859303, 1.4465147094145707) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.245321726140645, 1.2891919806275225) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4434746407275998, 1.5891500446902627) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.354728082431312, 1.4172726792244565) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4498107348220144, 1.3990223887653386) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.164268211578435, 1.2247083012507998) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.437985663627205, 1.549181514275991) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2145326342189222, 1.2616729458383646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3561238500985875, 1.548163366662674) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.139957324431731, 1.2183027765036327) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1662096031798521, 1.2852946910753318) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4464513893443738, 1.638795507436447) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4240601844262268, 1.5329517705331468) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5386505241427726, 1.6545377493149114) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3028729708166473, 1.4282989769844143) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3077586502736258, 1.391535535090644) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1085922397365109, 1.1753057784296916) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7321838833083787, 1.7710098351811077) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3391726803681505, 1.3711989882408735) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.089342318909889, 1.1544121755420806) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3373966829260653, 1.4510350229922275) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3907465300572257, 1.5048733538813557) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.265194192838656, 1.3624077903277996) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.39817723079563, 1.497938036201917) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0364455697026673, 1.9726160845887535) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4542251699579876, 1.4077632704744354) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.341081041647233, 1.4459682073740783) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2686406098369702, 1.3681049051208338) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4903861439835528, 1.4260013366478834) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2768312571371043, 1.3113538470210848) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1885144594622725, 1.4972544582600587) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0285562250067335, 1.1911068631621513) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3909155354176082, 1.449945196740468) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1983458886403635, 1.443550398035987) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4933262364308884, 1.6141442455110075) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9659172522877604, 1.2534801514954594) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1841905225068163, 1.262059195468517) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9985581175735244, 1.086014840148805) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4195791627453807, 1.6613979117848736) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5411866437212405, 1.6155837622457507) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3217175898624591, 1.4464694875874982) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7262541630085668, 1.5327581489869444) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1590770356371587, 1.2962726625064023) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3781332122460612, 1.2984896563294668) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.138077429377482, 1.1179930092605115) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5716550808952383, 1.7263685910425137) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3414035620649634, 1.4187141570887336) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.043152257226418, 1.0427555239137352) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2778799067216697, 1.3549263647467484) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3675166990698149, 1.649511978777963) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.284225566663942, 1.3481118314661917) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5835346679111917, 1.6748360054094409) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4196584040351354, 1.5700614109933764) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3370360026413934, 1.4175983829953507) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.359740994294722, 1.470150402901016) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1084542856327482, 1.2148038625582005) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3377354220802986, 1.4595589126810051) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2132630731744034, 1.208849698318002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.230208806627237, 1.375282897932246) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4371448038190944, 1.5808682771472375) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9927064501858469, 1.0862463557691435) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4464190130772527, 1.5006391197074826) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2051144202278903, 1.3842887636724441) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2340799201626143, 1.2564665655411178) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1457596905363314, 1.3751667562766416) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9856235334693674, 1.061888749951944) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2280358908717415, 1.3867205508672917) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.176935148302788, 1.140522065013941) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.420071138817855, 1.4701573140879494) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.193726607010204, 1.1909509916280543) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2596233626327975, 1.426367362999452) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4299724354726833, 1.4857996594455722) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8370500580299831, 1.8810487861578638) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2773162187866918, 1.3775038630776781) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4491410806294496, 1.522649104917256) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.61461128847435, 1.5087090799360463) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2756661365667676, 1.4417741460046019) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2358722047369683, 1.3165272727909527) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.079450546791489, 1.165071739522106) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3296205227005276, 1.3999619058985056) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3700963754567117, 1.4154198330947758) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7631828899466653, 1.6025472655357336) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5376721420060455, 1.6907457659417526) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4145407590920633, 1.5065058092162444) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4987533047613162, 1.6353425011083338) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7967497407157869, 1.7888339907902997) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3182472709918847, 1.406702162730272) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3661241216064535, 1.4276759804832857) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.172498745146777, 1.2325044287715445) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.948852417006679, 1.1540761181931842) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9649828591662741, 1.9513771914929008) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.24404688012507, 1.3260810294659322) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0713686392682165, 1.0773508560941265) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.628032852586953, 1.5780377553620155) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.216048076752868, 1.2636418918122716) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3364885047706332, 1.36588332346474) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.180266716883524, 1.3272171628890166) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1230399713561914, 1.1066000820909987) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000203084920114, 1.0000872428014385) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.186710072186473, 1.2427411032272662) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3332442682997223, 1.4954683782079874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7627882692085703, 2.0595284770311038) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3073040298009375, 1.4146487856136456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0680686076924863, 1.114245738681797) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4807334472255713, 1.5355080288197502) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.580581983110714, 1.4677299367258658) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1589763124970518, 1.3857450215448903) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1522072796589837, 1.1459030330007136) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6404015771481228, 1.687145171550228) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2374939234341749, 1.2938366715973413) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0583943623773677, 1.088023350908204) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.213364633227401, 1.2172629217388649) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4538798417097947, 1.7015048121109997) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4186194297180086, 1.5858266642456187) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6690658071237818, 1.71649283605559) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.178999150645325, 1.254948737903733) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.113250248447606, 1.0785719503477607) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2720911871385907, 1.3247335363372843) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3227091188887232, 1.306977300963902) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1017658894634688, 1.1254079404685402) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1674028471358269, 1.2633999182754088) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.399454372376918, 1.31628956989862) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5285003525131426, 1.6465273151559332) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9738074670890562, 1.0556216701591667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4012381522480792, 1.5197992929996713) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3896081010506192, 1.528751787072085) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2056865246699424, 1.277087612710514) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3289834132732052, 1.4323060220536843) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3887055638515182, 1.5075910454015173) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0577659977308231, 1.1255732166413028) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3723410872511135, 1.3865911208019703) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4425899887098166, 1.5282547757602283) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1627651017303005, 1.2168831557636353) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2099242509045198, 1.348438718349728) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5871877372927057, 1.4373274049551543) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2091021856543323, 1.2832633516848553) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0491884529157947, 1.0622972532992623) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.031610066577527, 1.0559581925252315) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4756643587828904, 1.749705749288251) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4821330963398376, 1.3996350350740767) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5728950716425238, 1.7237603824858312) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1006590724150527, 1.0594654279566802) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5517812874265509, 1.4115801820078087) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.149688123638645, 1.167327540848455) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.425783766806004, 1.3001257661649177) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9852198519912665, 1.061494892506612) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0548455332712352, 1.1351610926059148) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0130879321185886, 1.0979320338416476) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4891201383074868, 1.4510889195712324) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.087436575789147, 1.1102876054750814) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1396020372136297, 1.1643268716918327) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.13856569229442, 1.9998983042407734) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4636704629321051, 1.518740654509266) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.081702109498039, 1.212639471931836) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.037939467134611, 1.034440670582595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.985620939916065, 1.0618983409187308) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2979055258091532, 1.371785022799512) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0958344397561484, 1.0532975644060862) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0220919469089296, 1.039910921133406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1120354314964804, 1.137114499124826) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5399734920446835, 1.6401624101161603) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1378425868272688, 1.2561617653101627) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3938665469048928, 1.4692981165286911) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1360642869692357, 1.1817539901118372) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.599594502374825, 1.5655151923618216) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2842269659005363, 1.3768060743316017) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0613659108757638, 1.1422450146921632) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3986099384527149, 1.419674354166456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.08106973051699, 1.1241324065903355) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1953809818755137, 1.2594613620688602) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2710450926098082, 1.3647601644269525) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.251261899763205, 1.3082476744145388) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.321608359112897, 1.3369627112763294) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.369909989747395, 1.4604814516987168) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1746242173629335, 1.220821122148978) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.008600815109987, 1.0309102624018043) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1433287787024597, 1.2825068992436568) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.143083403109134, 1.1802285963750445) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7050950698569145, 1.9069430866110098) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1314884341936935, 1.293943053003809) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6342317899815784, 1.7559036835319526) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5191960914699518, 1.7244759594486152) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0189770771985924, 1.0419260045291616) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8645751043691636, 1.7599548802039195) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2634182781230785, 1.2921966221062404) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3623672263433897, 1.4477424814117) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0036449691654363, 1.0184174718673653) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1517838489988377, 1.4207443004073974) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0235931483469283, 1.099927889397832) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3227567734775738, 1.300243465181785) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0579412404487987, 1.0877818695819284) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.141702914104643, 1.1780335270300337) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0866935860513347, 1.1108823120380629) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9488524171396376, 1.1540761195942597) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1778686132115306, 1.1439987315136537) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1945287422127802, 1.2595417715135315) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0869569925814837, 1.1378265787425357) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.025551088649847, 1.0924723067123343) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9651795037027867, 1.0372700442319314) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.070973025544449, 1.124759336583048) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4171013710237508, 1.315146786853423) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3442879264042173, 1.4186113098743838) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8454045135957249, 2.048726908192926) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2597170732562741, 1.3480081148096839) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.056854142504352, 1.080458495663126) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0225187036321028, 1.0403827292567938) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000016632372664, 0.9999967408194587) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.249139833457908, 1.269613819976193) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941984040928144, 1.0622879314413631) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.057170310034955, 1.0838660441448804) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6879638012847726, 1.691601913446708) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6892894733558235, 1.7516391295647022) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2657302065845129, 1.3291307844797349) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0076221857696963, 1.010887529342634) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0110369916298863, 1.0069196758820942) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2473441291515839, 1.30002596500534) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.210603525082741, 1.3095124565368985) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2516717035785165, 1.2352629836673672) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0716598215416333, 1.1092074182917002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4140150221836936, 1.4795406150743378) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1383292453775182, 1.267502504561406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.245820976800964, 1.3350439215425773) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1488865338677403, 1.1439115563605924) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1000152911408843, 1.1881663583135051) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6928257147192012, 1.7237175983781792) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.491665065362328, 1.7194617819843758) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7250614703647011, 1.8773904132239871) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9738074670890562, 1.0556216701591667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1001103476112213, 1.058515869873802) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2382728518043857, 1.3478507233849057) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0230665790083904, 1.0409901582094545) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0913122886525288, 1.1150619603946401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6557620931336443, 1.8167740824918492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.256172679169253, 1.2249622128044328) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.987546331831983, 1.9365299159860083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009190613248756, 1.024548780731301) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4145607948927805, 1.3260156620457264) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1416670973827394, 1.2295336198763587) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.22857882769453, 1.319886974863151) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3087379156367287, 1.33864821963651) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.193430887257921, 1.1906712069583714) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0279205022176732, 1.0346821508794564) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1085922397365109, 1.1753057784296916) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0606348646365347, 1.0966087325272353) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0857048954595128, 1.10798758540514) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086884562049796, 1.0240627900805013) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1226367036116651, 1.0930512266684942) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0677945891200036, 1.0923838676403874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0700847088650962, 1.153953905044766) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0426999219085318, 1.1001192173042595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2174892275830156, 1.2681189078893371) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.039431301166405, 1.0768339646433518) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.3633888476663105, 2.0651857831144986) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0387199139880843, 1.0330557460869298) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1984925707981613, 1.342681136705142) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0819059530347916, 1.2130468029383499) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5105324148522856, 1.5064272748381218) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0637979994271536, 1.2334191609637015) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3804632530776413, 1.282225760122347) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7891689191436735, 1.8672681820747605) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9860594886782702, 1.0731416732663317) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1106892044113257, 1.1110776181236728) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.133555471990277, 1.1789444222991796) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0609413116904702, 1.1432974892627557) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0716494305127886, 1.2412763841457801) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0093005221465805, 1.015108770973406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0276624990816674, 1.0620643378830223) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6056317564653693, 1.6543814344764252) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3091810534828912, 1.377575625370183) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0634351293192081, 1.0769623439574851) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3621225146049591, 1.545872721203336) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.958846805139947, 1.0230049020289091) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.901165028084583, 1.8163028848824083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.551345182841608, 1.6885986403641053) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.167906857421411, 1.320447068603874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0957733646572474, 1.0542870121252816) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.071036683838371, 1.1045290039100588) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1128674362112978, 1.0779101894490986) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2540558249639593, 1.3906515021515626) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1401782238603204, 1.1633489539189938) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4648903109052682, 1.5017598129200391) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086854553918623, 1.0240902358515251) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.248948606229635, 1.2075460792652186) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0638711306186068, 1.077485929574595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.302366621589939, 2.14537014754687) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6800782204436746, 1.601782563108804) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1324207762644822, 1.2374258969227758) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0106239647709434, 1.0317680864118683) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9398006103566571, 1.150477172942554) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.003711483109492, 1.0184673907891648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0150423857205677, 1.0103067669238535) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9899286762176144, 1.0831523331781618) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0772381946711351, 1.0684834189593408) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5788816509521195, 1.7085125524272937) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086490183228989, 1.100993076618948) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1098840112628257, 1.0747044756233686) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0269037878637026, 1.040319696531097) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2989810246822493, 1.516018871935026) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.09579761936854, 1.0542768476270743) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7312283105258133, 1.6029147242376325) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0041341968737916, 1.0942381626961162) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9652025531199022, 1.0372713852031648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.028971096588657, 1.0496419169018198) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001600167111078, 1.0006364886795711) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5901000545161017, 1.6520210340818942) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0608388344607216, 1.1418201060888773) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6294095853926036, 1.6163785825672696) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9239194629816694, 1.0620409046996342) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9865180053344625, 1.0631700785996852) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0111045677411543, 1.0068501866530697) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0300263800398297, 1.0460945887316897) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.055656008159491, 1.0793102682927262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0592993873762195, 1.1000761948454916) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0763465311021605, 1.150805770834621) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.027312208814914, 1.0466914317467562) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0075629889992679, 1.0117655642478243) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1032720384376888, 1.1258049207286795) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9721553832545388, 1.1562210739966874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.09677874206375, 1.112563844095475) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.144192901363899, 1.2383234015846594) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0491884529157947, 1.0622972532992623) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1201481309769188, 1.23198314524209) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0321506842244292, 1.1028910037854294) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940822219328275, 1.0622150536023378) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0089889001401338, 1.0289256214349285) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.2886972882148124, 2.1481992195754844) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0221349171549556, 1.039958039867031) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.00889678874725, 1.0100369440115566) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0316100774920902, 1.0559582052343706) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7255033424144446, 1.7393040874114574) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4041550187895262, 1.4979801563587698) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0300152256075086, 1.0418258516807768) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0677222492216774, 1.1547546627711922) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9738074676474453, 1.055621670863076) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9554967094702291, 1.2407425382454345) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9852198519912659, 1.0614948925066114) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0225651148434265, 1.0404811287603104) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.023945530468186, 1.1002953897407484) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0366718740572467, 1.0219022700185652) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.89145582510874, 2.0073119818172453) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0256218506525012, 1.092572813082167) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2543424804576606, 1.1584906153804997) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2303746771822694, 1.268150140148696) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3436037993659378, 1.544638789573766) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.054195786747068, 1.1270852408171206) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1132501376936867, 1.0785984664464343) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0191178952071054, 1.0277573500979316) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0040125812148393, 1.0185725847316023) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5090777892285159, 1.6837647168160437) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0548455332712352, 1.135161092605915) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5489217678977465, 1.703872187792144) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0267775706948479, 1.0391766345286284) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086440515245, 1.0133289241223302) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0003622527749174, 1.0006421743934348) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0314807279277802, 1.0687991423520553) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0087316658505554, 1.0156950719736757) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.064672009863228, 1.0744669379189278) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0656820405033327, 1.1329682694466814) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9627709511752979, 1.159799109152461) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.03791917484076, 1.034419506409441) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.056679464796781, 1.080091081084917) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.968253524679665, 1.0560952996256234) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0279655698819599, 1.0414197049581009) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0756470202712114, 1.073197941794778) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0225187036321035, 1.0403827292567942) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0091356279595576, 1.010301773199098) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0089244991442203, 1.0244131377391028) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.958907810383543, 1.0229990123743389) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0101261847400955, 1.015592302490348) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.856487759188739, 1.987802491829756) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0957733646572474, 1.0542870121252816) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4323900975722248, 1.38764558181673) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0002030869213232, 1.0000872458552943) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001817460282, 0.9999968348942765) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6761448836707815, 1.8000159191048122) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0110369777520172, 1.0069196052774705) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0076420270566468, 1.0109130650092806) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3143444888358118, 1.4364660301512269) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6269458399195869, 1.7588137833190511) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0150423857205677, 1.0103067669238535) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9503313698708773, 1.0210101877141475) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0221103411565742, 1.0399310912541044) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.022161740185914, 1.0440362161741663) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0212291948123313, 1.037812238414688) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0230473136805212, 1.0409687987569014) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000574743026291, 1.0001944692240694) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9790271258063736, 1.0621447449651071) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0045672201764262, 1.0206986161366172) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.981552788172939, 1.0654062102517186) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5786147994618658, 1.5796723157571968) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9505085876017155, 1.0266160225205738) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8992468483830982, 1.8381705583042731) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.007640550260995, 1.0109111691283788) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1023798800220563, 1.1027946169389353) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0378383854026931, 1.0916541944229017) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.662171547715608, 1.4809388858784158) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3884009154247585, 1.4693833732295243) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.00820081147347, 1.0048237589520899) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9891217013593645, 1.0452298135042355) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2967977505086272, 1.4205821265807699) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0267734505196762, 1.189527234470338) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0095041942875886, 1.0304585780094506) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0984425260911028, 1.1801463996773247) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.008949565268555, 1.0100975605869684) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0555547511592458, 1.2005282123293055) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2019556929161193, 1.4631989134731263) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.85401212533698, 1.697045646823639) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0465953184091767, 1.078970690899583) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6315200339640263, 1.7412508264771518) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.475741136999491, 1.6184227747004782) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9813546998577536, 2.0012193652412913) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.1175332031727656, 2.0604185415667806) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.908707020100311, 1.062610305602706) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0092028625440435, 1.0245464529546735) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5536231422112985, 1.6445809489282637) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.504961884357026, 1.4765656818367083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0040416763463311, 1.0188538547942232) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0229109649916208, 1.040825047374595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9243708792845878, 1.0618213332611868) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0100659861217105, 1.0346741609410086) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001738283730812, 1.0007021852863212) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.765619624354735, 1.671608274339667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000256539704562, 1.00011472593805) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1749660006132376, 1.3348151157119925) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5196245050708794, 1.5016510294807845) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4372234986294643, 1.556964093531671) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0495427967759612, 1.0637613514438269) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8942162937965823, 1.0623074342290058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9566233368769345, 1.022180800314833) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.010297978789749, 1.015862871232645) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.022537171977591, 1.0404032050948724) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0089244991024624, 1.0244131377024719) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0002005714269862, 1.0000861425492609) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0225855613154375, 1.0474590050600343) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.008282093745896, 1.015020385021123) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3292441637907877, 1.3420852968778196) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0355842499392884, 1.0403347265719545) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5598901570117432, 1.754258521302419) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.00756367750926, 1.0117730012140096) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.589460202631202, 1.6310416833075057) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009182414588414, 1.0245368362440739) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3500679823016972, 1.5849241373278118) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6300716452223947, 1.6608549895984739) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8934598111960491, 1.0613851373263503) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0172232674741366, 1.0309227835834553) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4026965025537432, 1.3017255685461213) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0145845858779112, 1.126713426855627) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086854557919522, 1.0240882973827123) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0332645848178301, 1.0452853283629715) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9591073336357664, 1.0230972090860468) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0090239324043384, 1.010197809009051) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0438961669928477, 1.0729439648575128) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.181806091706719, 1.1839318741128002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8563512541739648, 1.9976868998238977) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9501169477878597, 1.0205649913958266) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.836137114254394, 1.7153939255824424) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3134774434799432, 1.342577163042404) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8197120532555924, 2.0141723235965414) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9207919331237537, 2.050377207587237) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9639522648751613, 1.036055644122443) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0086948638130546, 1.0241059801816847) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2725033245342992, 1.3217851085178995) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1689277456189513, 1.1614566237272046) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164099, 1.0210066614435458) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0091441437902997, 1.010337914035154) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0092865647406306, 1.0150552663051189) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0244815605193682, 1.041186509326272) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0268617336683088, 1.0402377936518334) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.0203691919367213, 2.0355428292104714) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8893296347449184, 1.8067703531209003) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000285414250728, 1.000127735592676) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.014727668091586, 1.024994903364542) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.374735755938457, 1.3683210931928274) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9591167474096071, 1.023255555647784) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.014727668091586, 1.024994903364542) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.916975185408699, 1.0965767497411336) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009283349316657, 1.0166808687633724) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0078702046233545, 1.0262933124211526) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000256539704562, 1.00011472593805) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8485257789765346, 2.121358072841311) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9601905509580709, 1.0334768742054787) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.5255282929440686, 2.452911024745238) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001813631912162, 1.0000782589566193) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8983651890409553, 1.0766186783966203) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.958846805139947, 1.0230049020289091) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9688327513277963, 1.044563674185145) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000285414250728, 1.000127735592676) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.888421415046983, 2.0389624328064775) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.897469495829563, 1.0334486669474874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0076257075690669, 1.0108914598331338) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8447784384874368, 1.8537593361734914) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8348029147999825, 1.7705575036765941) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.011180184215307, 1.0069192384807035) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9443577252528829, 1.09761943095273) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.962539015690692, 1.0367422623126292) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.003763196359698, 1.01869747451309) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000442574487853, 1.0000290664078346) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9713441551507028, 1.0418621810901223) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1649562462636027, 1.1543387918993817) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9576015593182938, 1.029787566065412) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.956985634947714, 1.017789008078671) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4398772285772838, 1.4578505988350332) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1887359576087102, 1.4116640096821176) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940847901867173, 1.0622175913578324) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6004502807844898, 1.8319691222586834) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6641820329897186, 1.780828152694232) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.955791832116544, 1.0181799290622577) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0002016077843499, 1.0000906155009845) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6708890920970974, 1.8389841661831505) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.731113267044244, 1.880056161733783) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.227388537978634, 2.1583684897174633) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869128255280491, 1.0393251197518067) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0091441437902997, 1.010337914035154) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5831518940461633, 1.6998967959198485) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6603639268468309, 1.6072962270464162) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950108053305285, 1.0207871078265653) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9274518154699893, 2.095257601452129) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.292504788859244, 1.4883772934654753) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557918317153393, 1.0181799286637723) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941983994502497, 1.0622879255884905) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0090071784623644, 1.0099373959556541) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164096, 1.0210066614435453) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0090127923955599, 1.010150670921568) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0002030869213232, 1.0000872458552943) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.278879179351104, 2.0825058086425807) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0112492158592294, 1.007657265605051) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8913331541256028, 1.0569426743997046) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (2.4361590056863096, 2.374190931569668) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939289404560917, 1.062022404926285) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557918321601032, 1.0181799291171718) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0090991943468435, 1.0102877550462999) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9243708792845878, 1.0618213332611868) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4331094728613554, 1.5112456252710835) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0192733415750903, 1.0275285681001012) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0089160640918784, 1.0100617539516896) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6042447677558862, 1.7629027148367047) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6634026124917787, 1.5925843101227983) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914921705024802, 1.0585524353231401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009144143565107, 1.0103379138387492) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9097744112621657, 1.0500497370526887) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000001477143548, 0.9999966273037646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9505136962575907, 1.0213377649219801) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0090239324043373, 1.0101978090090498) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8623825201586335, 1.841775191472733) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9026042075368054, 1.0690367396043892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.009486289062839, 1.0166388049448345) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.321108065322959, 1.3022205430125817) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4627883481240596, 1.3156113591698202) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950278454731098, 1.0210040021918243) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3730902106247738, 1.4271836797815871) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.00772034451788, 1.011000459386809) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1914682869757884, 1.2941662133722462) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9566051304967601, 1.0221569090939595) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001797235145369, 1.0005184894104755) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2927623396592858, 1.3793261908995886) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4512984572338474, 1.5343143802245203) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7618646009565007, 1.8747397733176263) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0093810934189602, 1.016564710112363) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9706302205069914, 1.0458661138935021) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4013784259898834, 1.495704806512474) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6000330454581484, 1.6493104962012994) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9866790255995396, 1.0615916381646795) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.840027281222045, 1.7316649697960953) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8945271508179691, 1.062425004847966) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842479077887365, 1.045410572513111) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000031083525063, 0.999997622328312) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0002203665049088, 1.0007179948702902) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5738124503212934, 1.5372375552568045) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.520338878533676, 1.5683495339813136) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9595739055422712, 1.0361899292515682) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941085492117471, 1.0621058219322854) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000011370020554, 1.000003900558499) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000574743026291, 1.0001944692240694) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6659089512776009, 1.6411694441435993) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9505085876017155, 1.0266160225205738) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941940722453688, 1.062285730482262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9026042075368054, 1.0690367396043892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9699595875812708, 1.047341440198091) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5299505481929392, 1.7556035094271705) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8925953072126493, 1.7360014096095309) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8257252605429963, 1.987568762431884) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9087070204729161, 1.0626103059155712) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6183932239248628, 1.5333785020280855) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557917578271535, 1.0181798461719413) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9624879491822841, 1.036692099157249) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9814308983958775, 1.0504867838097345) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9091874092458865, 1.0502540809236933) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001583606407198, 1.0006397026853333) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4560704394660193, 1.5245406763062734) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0082320736540555, 1.0047539818872901) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0000561676542408, 1.0001900480586785) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9501120740489738, 1.0207923943122854) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3302168684292093, 1.3910388870841244) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9564842773957657, 1.0179328280678963) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5370607748602756, 1.4814698259141177) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9566636828156103, 1.0222389933366538) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.2149201540394599, 1.2410130607210792) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1870061638011018, 1.1882924568036257) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.894502858306569, 1.0623901471374817) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557918309581602, 1.0181799277089096) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9791820258980188, 1.1297320818407426) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9556375843670228, 1.0179063885260198) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9669608449343777, 1.0389959071458281) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9505820232553542, 1.0214431078598631) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9553444300188888, 1.0133820227730423) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9569465632576678, 1.017535783432386) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.5983409781557905, 1.6311671623174848) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.000003304326776, 0.999997741871058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.9067261609115833, 1.7311009781764974) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9970148176562125, 1.138200910026389) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.956561128439479, 1.01928379408489) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843121783984705, 1.045122452388375) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914949601852116, 1.0571491736459837) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0581542682436056, 1.1298324564688413) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939624888794047, 1.0620575287811318) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9261894013752994, 1.0614450422166941) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.311894127129303, 1.596997589078736) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.398730083603971, 1.615434483494464) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9571174216446804, 1.0135612308021646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9556477291833532, 1.0182447555539234) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9595912987209998, 1.0264966554531383) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869123216684076, 1.0393239217323598) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.7993807443116585, 2.0052457968459505) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164085, 1.021006661443544) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8983879074194269, 1.076641128801191) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914921705024802, 1.0585524353231401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845384562069515, 1.0436712889104103) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8518724464847682, 2.092757728915635) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1776382188472128, 1.4300494802833006) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8011933368830362, 2.0928842843942275) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974695085832005, 1.0334885872816622) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0041697272247563, 1.0872653555042004) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.957310268432746, 1.0193575402556054) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.955794419721498, 1.018182816515744) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950700372825176, 1.0215944566750284) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9572241068979425, 1.022062159031538) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0001754844408781, 1.0006989712784398) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.6583159448476927, 1.7924984382408327) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164058, 1.0210066614435407) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8947970054494894, 1.0604894032113408) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3537714409095722, 1.3894619321159891) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941938296181465, 1.062472353424943) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.8853649310472507, 2.1105484702237076) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9591173347273355, 1.0232174044910312) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3454490354419428, 1.4105764754211916) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1702023921991176, 1.3658928807594075) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940822237354685, 1.0622150558748473) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941986977444691, 1.0622882503047892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8975475595273333, 1.0753400811218452) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9590699161253581, 1.0232043634269357) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9503125159091969, 1.0210420120912211) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.003889578630017, 1.1606345666182971) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950113403784784, 1.0205619035430495) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.3430936954250827, 1.5506968430785084) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8972369216559433, 1.0335345934747369) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.4194257987799546, 1.5864786233336352) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0152618855570221, 1.165209463789838) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502156200769863, 1.0209100452926638) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9601913299052982, 1.0382918044546101) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9704251472344442, 1.042113399407608) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8975476727079061, 1.075340222537085) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8975735902692433, 1.0335841108005055) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8986678384036879, 1.0320941678923272) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502807565864843, 1.0210065707247646) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9548962356323156, 1.0143130983249151) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868394865215237, 1.039244521458028) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8946529452885819, 1.063178560620024) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9709853608924994, 1.0336262984865476) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8938362321354126, 1.0619225875032998) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8945813186778627, 1.0625432662957484) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0537649697138565, 1.1253204289809702) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8834031453562833, 1.0252116825829065) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9002114935261128, 1.052843316232599) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939868854382584, 1.0626553015777687) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846995776717301, 1.0438764624438224) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9601866562524861, 1.0382872372565464) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.918361006993246, 1.072692087375352) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9097744112621657, 1.0500497370526887) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9599373167647364, 1.0380274564539875) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941986985350194, 1.0622882509983287) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940857801832358, 1.0621659191013397) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.0, 1.0) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8956671061768354, 1.0618203772255705) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9551180174642577, 1.0184940119441308) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941041899465134, 1.062238724206648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974693167493978, 1.0334484671193227) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9084983756494442, 1.0630820590596464) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.1177881200122328, 1.3080483398718796) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9090788392903091, 1.050292387682496) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939462854196408, 1.0621300567461451) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9539848444971053, 1.0190333946692782) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502693469209685, 1.02099383920014) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164097, 1.0210066614435456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8951580173308704, 1.0632771055645738) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.896816318101885, 1.0570175214717819) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9582164623913728, 1.021791310090982) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.894502858306569, 1.0623901471374817) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502057766536014, 1.0208950929329204) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9501940966090602, 1.026164744959707) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8983651890409508, 1.0766186783966145) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9601823415880952, 1.0382830220396266) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9564363832233063, 1.0257595124077998) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9507955453045585, 1.0240585241267968) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.895107307874063, 1.069015804437798) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9557997933043807, 1.0181888126403205) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9032981133494074, 1.0681160304151667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.894198716109187, 1.0622882715933644) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886904674726375, 1.039465005504385) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9538358391697054, 1.0188676631255702) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9560997016241446, 1.0180441011597698) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843144776016898, 1.0451250179827514) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8942127496510244, 1.062219119981917) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8871011419850323, 1.0390668424158374) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.880905825285222, 1.0428735791152401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502037881623938, 1.0206625892287897) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950280837822858, 1.0210066619559381) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.898142949053932, 1.0273557070345178) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9126549682316623, 1.0932099898540915) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9554341646703016, 1.0325367939663324) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9563788644556133, 1.0179604873085566) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9556471819636251, 1.0182441449590736) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.899299100178376, 1.032994101336617) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9029986408836063, 1.0677838934625534) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8907454142599518, 1.0563427426943064) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9644780586275306, 1.1558413123215767) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9556375843670228, 1.0179063885260198) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9501169477878321, 1.0205649913957933) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8947970054495682, 1.0604894032114363) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9025628566009269, 1.0704267990929364) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842255304503367, 1.0454908877096365) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8944549755591761, 1.0610267983402195) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9503469916101253, 1.0167045624163822) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8916160225362895, 1.058634376177002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.957310469764952, 1.0193577649242986) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502674684905084, 1.020991742812167) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9221678165985119, 1.076276161902389) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164081, 1.0210066614435436) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8991665529711603, 1.0328512775486034) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870520603304433, 1.0396862636042665) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940847901867173, 1.0622175913578324) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9000456921822038, 1.0324194710975032) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941428478931885, 1.062232943148117) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9505136962575907, 1.0213377649219801) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808374164083, 1.0210066614435438) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8930905379743387, 1.059912229716592) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974695085832032, 1.0334885872816653) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842370070455763, 1.0451281007031168) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8854949161055692, 1.039102962113158) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9538215046161871, 1.0203195545601644) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845228466670344, 1.0436566417808824) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.890616300144348, 1.0549532598741442) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841505763172492, 1.0448847099839396) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8942204821106046, 1.0624040475435306) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9114526452581333, 1.0492939459825077) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9014796619112142, 1.0761924353252783) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9558429193258363, 1.0182369352211798) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502808303033987, 1.0210066534118285) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9502850609903386, 1.021011374475186) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845514649331024, 1.043529453162475) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8847000813038893, 1.0436279506451003) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9570691982054613, 1.1103112911695283) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9013204863392861, 1.0832920031898172) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868378432623664, 1.0392416657718972) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940702658270657, 1.0622413499918526) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8946607951806524, 1.0609528659393674) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8915267267745466, 1.0570885903196658) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940069920219259, 1.0626010499641199) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8988966338847353, 1.0290562752880255) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940857789804284, 1.062165917584331) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8940838379347354, 1.0623559996721639) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863315674186475, 1.038442243920968) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8944574385417562, 1.0189216305173685) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8804519575823039, 1.0425383212352792) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941940722453688, 1.062285730482262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.894107162630076, 1.0621040894177067) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.893621727962765, 1.061556070617722) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870245342920887, 1.0392143832100182) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9501169478514208, 1.0205649914759567) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941986977444691, 1.0622882503047892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883080506751018, 1.0260443788445517) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843144776016898, 1.0451250179827514) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839594384364455, 1.0425774520522129) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9116216355475159, 1.085318412600921) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841505763172515, 1.0448847099839418) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8942457346827394, 1.0623341252258116) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.897469495829563, 1.0334486669474874) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9102393965122879, 1.0507837133831952) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843170066088211, 1.0450352466927582) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846995967702157, 1.0438764837780505) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9002084297744006, 1.0528389199769097) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.950116949035539, 1.020564992968726) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844868425842727, 1.028058842244608) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8942348032083767, 1.062327611527961) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845226107850152, 1.043656742267697) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8965515811994822, 1.0622047763435993) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8934383742359895, 1.0612902851389145) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8931188494060625, 1.0599398762864045) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841602105652381, 1.0450304519593647) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870333881273703, 1.0398664751039377) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9124698816652121, 1.0489539015522962) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838724877749433, 1.0416416165207394) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8943893414750063, 1.062149554765627) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848499124824625, 1.0363897077555162) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842257259764116, 1.0454911050238098) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8996622175873834, 1.0329178942483495) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8992991001299203, 1.0329941012943542) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8911133713921698, 1.0583498777492444) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884536172266802, 1.0436687403990905) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843008045554382, 1.0451077908452195) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8962467859000214, 1.055858900483028) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8840876876524356, 1.0452930898134833) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8872712482438897, 1.039406924402545) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8847781467586744, 1.0431543303867161) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9000456918017351, 1.0324194707637366) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.895933381136212, 1.0521004817597785) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8986696773519718, 1.0320968067070329) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9558301752152614, 1.0182183499901714) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8975916782296633, 1.033604294409082) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842370072956964, 1.045128100921271) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8936448942088875, 1.0616736388480328) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914921701608283, 1.0585524350234214) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841408209618895, 1.0448738243521225) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.897469332334775, 1.0334486166146322) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8895902889473792, 1.0373244996700266) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844993612079805, 1.0436305177351837) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9813550906160847, 1.1098582569016473) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868663061843037, 1.039365606911665) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939647127190081, 1.0620599531532042) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868520564431657, 1.0394490132952843) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846554700874092, 1.0437883950477789) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869123212073634, 1.039323921327908) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974496490815799, 1.0334265202245176) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9000456919962718, 1.0324194709343946) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8837253799288081, 1.0338634703834906) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941069393649306, 1.0626509324205353) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8941572672994204, 1.0624325362536797) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842255302163166, 1.045490887414613) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863376820217886, 1.0402088785033003) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8937862504646655, 1.0600292041076598) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8975735899758811, 1.0335841105446382) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9534115813263999, 1.123342048390804) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841392031911509, 1.0449026537509514) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845246117574453, 1.0436142307738507) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914671050687435, 1.0585257440058378) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9178373185632005, 1.0473066087363918) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8973686286109105, 1.0337008490627486) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8939765147277854, 1.062111734933252) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8814599105397722, 1.0447945675036054) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9334423471315889, 1.126911387094484) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868739516404095, 1.039508318666753) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8979534219825587, 1.0651009851999302) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8916160225362895, 1.058634376177002) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974695087150142, 1.0334885874047053) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8964636943182209, 1.0559571274534332) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842370070455763, 1.0451281007031168) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841216471812222, 1.0447895367657338) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845425775133958, 1.0436085598150364) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870245172520231, 1.0389825381508841) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870401480436075, 1.039581714734841) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868841696245217, 1.0392331844300793) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841403881450398, 1.0448733421219132) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870251297649668, 1.0396362031950366) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844414873055246, 1.0280764331526626) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844737621629787, 1.043688377371377) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8814614148480985, 1.044796246153705) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845267858637098, 1.0436601895367428) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884219607191592, 1.0430138413858436) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844684952850032, 1.0383096969404442) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841576335763275, 1.0448863695284558) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8840391623696698, 1.0452482758546946) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846640211169294, 1.0438367863901072) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868603382669491, 1.0393866012891804) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884180908631619, 1.0448922283064215) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8997090583650919, 1.032565374034569) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870362565866272, 1.0389980281267344) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8840291791149988, 1.0453048693540814) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8835800163582026, 1.0424493278057945) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886856063760331, 1.0394879414307447) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838121971270286, 1.0355157748833328) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9255949844662335, 1.0626131441546731) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974640427525638, 1.0334422572355222) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8847000689814379, 1.0438775736386503) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8891288961363409, 1.0465636640176579) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862880933071776, 1.0384446793191695) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868375533618861, 1.0394525283465947) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848499124824625, 1.0363897077555162) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845398842016197, 1.0436731717913503) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843273167749219, 1.045081535540481) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8840370007111185, 1.022493919106171) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869046747264594, 1.0394650055044872) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869159906292576, 1.0393196961342464) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842699791157562, 1.0449803877080743) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8816319246464079, 1.0306295586308563) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841408207651842, 1.0448738241805566) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886878371740889, 1.0393790725927041) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868653103474409, 1.0394271257654797) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868718130327133, 1.0393110992361045) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833349795200912, 1.0255822626728397) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884523802723792, 1.0436574088524633) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868075025140963, 1.0397921548266529) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8829825865699247, 1.0259113676917708) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841477506941997, 1.044913196789047) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8814599105397722, 1.0447945675036054) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883204348915901, 1.0330085174431654) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8806266253237603, 1.0427645810528077) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8894008769442711, 1.036979775680204) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863541760344726, 1.0385258663896895) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.897200256513114, 1.0429276763929074) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870649887714022, 1.0396830350670634) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842594950042909, 1.045027289953092) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9521455524184239, 1.0221118564220961) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868640699725098, 1.0393631116044628) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8957989239335423, 1.0206426422460786) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870457960656776, 1.039593424406626) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8849364703023895, 1.0433708685692302) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833601863438498, 1.03384497461026) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850245273467933, 1.03677481327519) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.880905825285222, 1.0428735791152401) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869194542872769, 1.0394345756679813) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8971691847604705, 1.032803812547496) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838280223480993, 1.0418895004546864) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842120490389063, 1.0320071018388561) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846269060691648, 1.0435230472671055) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846533779171868, 1.0440166564909756) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974403111587477, 1.0334644304168354) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8990810668573138, 1.0328536932385453) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845828630732562, 1.0441773864300856) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841654847166786, 1.0450366620346174) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869953644591441, 1.0395498153719949) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9005845375444042, 1.0325576012142559) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848228082558551, 1.0431517208845484) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845653032998577, 1.0437163485198022) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841491100237233, 1.0448827852614155) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845185371732659, 1.0436789061037213) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8969811692111858, 1.0336098810862542) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.887271247510772, 1.039406923729551) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8867595941338793, 1.0393579671691955) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (1.07966712644957, 1.1843918968521627) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869128255280486, 1.0393251197518063) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8972261716669117, 1.0335225960977896) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8842255304503492, 1.0454908877096516) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8865250553294518, 1.0387807151483246) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884125191128995, 1.0449786677872521) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844987494658294, 1.0436298351180213) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8872690757153521, 1.0393034580679124) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862570942491317, 1.0382259536884817) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8847572810192503, 1.0439344255601406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838576766964145, 1.0424314351634083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844969881782313, 1.0435568806963669) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841304694340802, 1.045014907688456) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845787593296526, 1.0442464914510472) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8871323467216152, 1.0391001318009236) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8986678385767901, 1.0320941681105362) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869975134809187, 1.0394765833673683) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874039512068537, 1.0240345831162683) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8877746284151872, 1.034815231026067) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844868497093711, 1.0444417607289582) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863007010272281, 1.0384685724471603) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836677687159668, 1.035450191692939) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843071999870654, 1.042835218919069) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843453665780665, 1.0279620697397762) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874039512068537, 1.0240345831162683) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850929541779191, 1.0368702028546122) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8810555402701352, 1.0312717098325557) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870092677324964, 1.039369527965617) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883080506751018, 1.0260443788445517) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8837810159879984, 1.040093988399115) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843523838432643, 1.0440564288299463) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8894228840304867, 1.0386181414715459) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869281604699416, 1.039486235999586) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883847903936893, 1.0338918073297667) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836296048035176, 1.0422811255069206) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8871542477979518, 1.039392575545263) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845109858961587, 1.0444223078288253) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8809018109728631, 1.0428690996408878) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8866100258201463, 1.038915554047133) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8956119528474671, 1.027772133820594) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.881197751894625, 1.033411874365739) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8821904119990687, 1.030302992232298) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864799041678686, 1.022881174465728) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863575115401833, 1.0385685444810406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8867596063620274, 1.0393483027233914) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886522936926914, 1.038777344163882) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8822213539645855, 1.0354804612831485) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883959438436387, 1.0425774520521596) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9061654464880206, 1.0643668887768918) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839172241779049, 1.031880811560815) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8851878533275718, 1.0246708439080188) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9376118441015455, 1.100316473057421) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883955427075993, 1.04253763230495) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8858732043733644, 1.0309519001029928) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850079784299527, 1.0367547567032593) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886975439950094, 1.0395185619913496) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870333867940885, 1.0398664738575873) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8867269691727351, 1.0284340908329648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8875166384729484, 1.0238354196096289) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886872960962994, 1.03940035569736) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8871503471009436, 1.0394325136590077) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884849912476245, 1.0363897077500617) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886947292523965, 1.039416692967769) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8797496336133791, 1.0431888817709927) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845911488693399, 1.0436351874752188) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870688132957278, 1.0391240198382945) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883839313387906, 1.0479413912177828) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868701799941592, 1.039394672779692) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8957328659463455, 1.0205170539991448) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8914533122296399, 1.0358892108822018) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883876416554687, 1.0416460004771249) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.886328691489413, 1.0384379815886058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869965339812815, 1.0394779746485405) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88178641974889, 1.039808613138718) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8804519575823039, 1.0425383212352792) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864711694553102, 1.038721607509162) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8856840334787076, 1.0243880293521692) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88641705824979, 1.0308786893057682) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868922170037916, 1.0393543847703948) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848566649778343, 1.0364014420266998) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874904954193316, 1.0387054482888916) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862261016074253, 1.0384758997780277) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8964186875550347, 1.0245044441654427) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8984481156794475, 1.0487561262726952) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868779755442596, 1.0393791202694795) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8865144809298917, 1.0260979992964847) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8806660436095406, 1.0377276178759598) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838843017505087, 1.0416774481848603) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8812344045626206, 1.0395617874388396) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868390410094851, 1.0392437467119648) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863206431412547, 1.0384898413449337) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833331126168856, 1.025580013388065) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8857130546744113, 1.037007440454426) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844323237453654, 1.0280605092116128) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870170185671551, 1.0396590361267295) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869064405070886, 1.0396398923746468) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974072033333113, 1.0334374426348913) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868387736829476, 1.039452491910502) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870252456816653, 1.038984102940454) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8807213543533097, 1.0284419140613277) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8869219682423782, 1.0393351573721359) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870116799101543, 1.0395384103751195) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8818269591881492, 1.029879539322214) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8849275119615388, 1.0378894800650615) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870639338624404, 1.0393195049475927) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9150098617563833, 1.0663411351436822) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8888758093935458, 1.045108720455998) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868378432534189, 1.039241665764048) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864736395418353, 1.0387214001295224) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8806256403216461, 1.0425901841352745) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8870300188998725, 1.0390844019092682) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838688630209862, 1.0223833011525654) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841254204534181, 1.03547317542759) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8806266883183438, 1.042590630097335) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862669608609823, 1.0382102879814323) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8849685459853902, 1.0310377215935262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883431367710808, 1.0254038995439723) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8868343684517165, 1.0397701443822638) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862924489136382, 1.021852756957845) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8802325023131706, 1.0386051982927567) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833208932288326, 1.0254572688985601) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841254204534181, 1.03547317542759) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833960205879398, 1.0347818412495706) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848444724311246, 1.0363802616395608) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843736809349437, 1.0229440078983232) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8810940126070648, 1.0313411667690613) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8857918743670979, 1.0244745697814246) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8829008720070219, 1.0417916099260798) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850257855680215, 1.0260286236360203) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8797285504068043, 1.0281378487750228) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8805192412868336, 1.0375381519739815) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8857818062003725, 1.0236209475482687) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839117957055439, 1.0319567783163264) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8974278103063263, 1.0448933786341863) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8808565309785706, 1.0262295212048622) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8812344058977594, 1.0395617891219944) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8857566870327165, 1.030724720963988) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9008517992043875, 1.14016885457395) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8872739896382066, 1.0249781748987283) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844905018904644, 1.028173686831477) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850186672415757, 1.0247154310037414) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8813156360864791, 1.0441035949812352) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.881817639512068, 1.0397460888921104) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8831206002857801, 1.0260905487198027) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830808413786386, 1.0263544597562932) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8951563612849178, 1.0363503061889159) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8801187060936172, 1.0285424794593505) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836255784767675, 1.0237856240668897) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8853234937153627, 1.0245515992460166) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8976546290202823, 1.0204612054677062) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8803767022550749, 1.0381703079575026) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8848243975146199, 1.03080461985632) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8829268887453925, 1.0515352194687886) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8823400039474092, 1.0405499577033483) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8810122706089247, 1.0311773514731) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.899878790506009, 1.0308252556631272) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8816745096558878, 1.0334645099652346) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836996715325873, 1.0264873743345018) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8835800681549406, 1.0353143043938255) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874038675332423, 1.0240365056039387) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8970691058248013, 1.0312143684583175) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830662966255733, 1.0305070738490316) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8798452404477423, 1.0433138420261467) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8845124595596596, 1.0282902418007491) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8818176040676019, 1.029642282460642) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8821904119990687, 1.030302992232298) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8856840334787076, 1.0243880293521692) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8850779197016599, 1.0368513688619263) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863734003864407, 1.0218887540326498) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8861525229019205, 1.0305574579489853) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.880556612786355, 1.028392803469554) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884849914183206, 1.0363814077308688) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8865775112934304, 1.0386754948661256) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864108577712083, 1.0386877209285064) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8860406224749614, 1.0305339897456605) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844544647362024, 1.0314040257782406) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8809935367298692, 1.040652155530783) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8817447532382219, 1.0297684244147511) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8824517944806263, 1.0403512991684685) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8944905409329525, 1.018879348889758) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8840222544473272, 1.0430973865360902) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8872980653077914, 1.0263440168459188) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8993383789195616, 1.0269598602933792) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8815482210365153, 1.0277867340491118) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8859471489258909, 1.0378214518678608) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.896398511913371, 1.0244442126859563) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8957995474555421, 1.0218821315836872) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.882806632369967, 1.022386793794319) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838157165947296, 1.0224218078448892) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8822020516151738, 1.0354706079804974) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874819399889983, 1.0240958770319044) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874313615404985, 1.0238839090286322) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.887460038958368, 1.0241167669701143) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884945758022629, 1.0270397798007884) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836438978623783, 1.0269351622622565) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8810180524731769, 1.026313627857682) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8805360970910437, 1.0430020832501985) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844653217648661, 1.0281782870143084) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8804990836485151, 1.0281252230598963) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8880670898084847, 1.035773640787236) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8832755272877548, 1.025498587926587) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.881300428261898, 1.033487494999639) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836632794565139, 1.0298609852700185) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.881786120269805, 1.0398076082948509) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830929691185837, 1.0261067861694473) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836034388250953, 1.0354481065629202) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8813524425533105, 1.027366499952146) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8873653126844468, 1.0300286630187048) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8835975064531558, 1.0337026064186619) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88424041955985, 1.0316555341025693) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.881297494298276, 1.0397183183428083) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88641705824979, 1.0308786893057682) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.884849912476245, 1.0363897077500617) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8837703139426802, 1.0425447356451951) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8833806056421495, 1.0338076423762204) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8816319246464079, 1.0306295586308563) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8861403777858053, 1.0220759835815074) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862964538822913, 1.0384774895744242) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8800703464054904, 1.0281259072666897) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8852337208224543, 1.0247310330229955) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8801235724999943, 1.0284722273655693) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8829219277756155, 1.0224791679638434) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8837279920038196, 1.025384896633747) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846274750806976, 1.0282193663856836) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838157504015557, 1.0227899609595161) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.879860611645739, 1.0289916977068714) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8805243097451915, 1.028464607854174) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8922305414931471, 1.051981252311652) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8847290850952502, 1.0235397042957322) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8821348667493571, 1.035722153039107) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8853234937153627, 1.0245515992460166) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836186176457813, 1.0238029635977772) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8831954933480487, 1.0257160769897635) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8852451661081575, 1.0256420580313257) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844932282195671, 1.0261857442154578) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843149129848474, 1.028085513300432) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8810122705383889, 1.031177351411579) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839764324631005, 1.022736328929449) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838901087873526, 1.022636944668754) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8804573688391676, 1.0285451169756699) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839858733436812, 1.0224040760519006) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8806812328446679, 1.0295270731999482) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8818382403982071, 1.029567125282607) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8818269591881492, 1.029879539322214) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8829192618439978, 1.0249345367926537) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8797889610314171, 1.0279876730827446) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8863728183837453, 1.021885656606352) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.880869539816131, 1.0274482980290522) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8832341100915341, 1.0233391597455004) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88590272868021, 1.031598193147425) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8805557462311357, 1.0283858637713112) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8827555255510684, 1.0283964315347045) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830171754299031, 1.026179400751388) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.885011665638233, 1.0267080264845487) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844987802167745, 1.0279938323274105) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.882423929639966, 1.0252425881921148) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862434245003847, 1.021711781404769) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8838837586333645, 1.022385604786707) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8813764925661375, 1.0326583967518554) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846162298357588, 1.0278933288901146) STARTING EPISODE 1 UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON Episode ::::: 1 Evaluation :::: (0.8847623922108264, 1.034287372771509) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8832772777153879, 1.0265434470611374) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844412982539935, 1.028223715430959) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830096229351724, 1.026318906113262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836458246290728, 1.0253592434397152) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8800736204756704, 1.0281220496354941) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.9050572821287955, 1.0462252697249053) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8821934977306434, 1.0303053964559374) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8857886837810696, 1.0245434029538567) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844025749163604, 1.0283811976500434) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8834891825209601, 1.0260152056110132) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8880165171146122, 1.022610014316876) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8830415218752963, 1.0263067946696354) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8836217773002284, 1.025174005173988) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864062168366843, 1.0217961913997526) STARTING EPISODE 1 UNABLE TO ACT RIGHT NOW >>> RESETING ENVIRONMENT AND MOVING ON Episode ::::: 1 Evaluation :::: (0.9150802925369884, 1.0886651105217835) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8872181106590594, 1.0239515503145886) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8874041851473254, 1.024029325084619) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8821441546583499, 1.0313017415756547) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8849685459853902, 1.0310377215935262) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8861792336737766, 1.0243937510979058) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843863387519391, 1.0224604061040612) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8894877622046623, 1.0252789895730734) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8817447509952839, 1.0297684216940026) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8892447930091479, 1.0236897671204293) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8996754818072406, 1.0223861778014818) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8846318331268077, 1.0280168799426017) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8858289504395894, 1.0226964230422662) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8864193132174071, 1.0230073171349423) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8834519611392884, 1.0251250377015788) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8849803114182522, 1.0260161489673698) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862907025238627, 1.0218527989674056) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.885009188142692, 1.0255843879829065) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8843311626945676, 1.0258640881181795) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841930462872625, 1.0226852946445635) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8841913429002656, 1.0274420696289301) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8862386698713425, 1.0222489647645912) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8860418282010637, 1.0228487286090473) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.88347428501444, 1.0254724556534198) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8800570037814501, 1.028613364171055) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8839044510378904, 1.0236153848039866) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8858590566469442, 1.0250747259801765) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8844677926831401, 1.0230651634252528) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.8825725351108615, 1.0180308004099439) STARTING EPISODE 1 Episode ::::: 1 Evaluation :::: (0.883603269550208, 1.0256726144720323) STARTING EPISODE 1
In [ ]:
import plotly.express as px
fig = px.line(x= [Time for Time in range(1,25)], y=res.pop[0].X, line_shape="spline", render_mode="svg")
fig.show()
5. Particle Swarm Optimization (PSO) (my favorite algorithm)¶
PSO is a popular metaheuristic with success in a handful of applications. We implement the single objective version provided by pymoo.
We set the following paramers for PSO
- population size: 100
- inertia wieght w: 0.4 (helps with exploration)
- cognitive coefficient c1 and social coefficient c2: 0.9 and 1.2 (higher value for social coefficients indicates social learning outweighs cognitive lelarning)
In [ ]:
from pymoo.algorithms.soo.nonconvex.pso import PSO
from pymoo.optimize import minimize
algorithm = PSO(pop_size=50, w = 0.4, c1 = 0.6, c2 = 1.2, verbose=True)
res = minimize(problem, algorithm, termination=("n_gen", 200), seed=1)
print('Threads:', res.exec_time)
In [ ]:
for member in res.pop:
print(f' Solution Variables ::: {member.X} Soliton Objecitve ::: {member.F}')
print(f'Optimal Solution ::: {res.opt[0].F}')
In [ ]:
from jmetal.algorithm.multiobjective import NSGAII
from jmetal.operator import SBXCrossover, PolynomialMutation
from jmetal.problem import ZDT1
from jmetal.util.termination_criterion import StoppingByEvaluations
problem = ZDT1()
algorithm = NSGAII(
problem=problem,
population_size=100,
offspring_population_size=100,
mutation=PolynomialMutation(probability=1.0 / problem.number_of_variables, distribution_index=20),
crossover=SBXCrossover(probability=1.0, distribution_index=20),
termination_criterion=StoppingByEvaluations(max_evaluations=25000)
)
algorithm.run()
In [ ]:
from jmetal.util.solution import get_non_dominated_solutions, print_function_values_to_file, print_variables_to_file
front = get_non_dominated_solutions(algorithm.get_result())
from jmetal.lab.visualization import Plot
plot_front = Plot(title='Pareto front approximation', axis_labels=['x', 'y'])
plot_front.plot(front, label='NSGAII-ZDT1')
# save to files
# print_function_values_to_file(front, 'FUN.NSGAII.ZDT1')
# print_variables_to_file(front, 'VAR.NSGAII.ZDT1')
In [ ]:
solutions = algorithm.get_result()
print(f' VARIABLES {solutions[1].variables} \n OBJECTIVES {solutions[1].objectives} \n CONSTRAINTS {solutions[1].constraints}')
7. Naive Sample Model submission¶
- You can define a new agent using the control parameters from your optimization algorithm
- In the agent folder, name ur new agent e.g RuleOptimizedPSO
- Copy the best control variables found by ur algorithm and paste them in them as a variable in the start inside your rbPSO_policy
- An example is shown below
- You can then define this agent as ur agent in the "user_agent.py" file
- Go ahead and submit
Note: You can aslo use ur algorithm to find better parameters for rbc_policy (Try it )
```import numpy as np
def rbPSO_policy(observation, action_space): """ rule based 24 hour PSO policy """
Actions24 = [-0.13273342 -0.43600161 -0.91979483 -0.94664411 -0.86072348 -0.77316203
-0.17414943 -0.11285137 -0.86327844 0.00431361 0.02807022 -0.21779452
-0.96210762 0.19775465 0.12472641 0.06825766 -0.05606408 -0.09025377
-0.86401095 -0.61281611 -0.51690614 -0.65854083 -0.08919786 -0.76346844]
hour = Observation[2] # Hour index is 2 for all observations
action = 0.0 # Default value
# for time, act in enumerate(range(actions)):
# # time plus one because hour starts from 0
# if hour == time + 1:
action = Actions24[hour-1]
action = np.array([action], dtype=float)
action = np.array([action], dtype=action_space.dtype)
assert action_space.contains(action)
return action
class RuleOptimizedPSO:
def __init__(self):
self.action_space = {}
def set_action_space(self, agent_id, action_space):
self.action_space[agent_id] = action_space
def compute_action(self, observation, agent_id):
"""Get observation return action"""
return rbPSO_policy(observation, self.action_space[agent_id])
```Content
Comments
You must login before you can post a comment.