A robot is going to clean an office with layout as Figure 1. However, this robot has no prior knowledge of office layout. The robot equips sensors to perfectly sense the status of 4 immediate neighboring grids (left, right, up, down) and move accordingly. The black grids are prohibited to enter (i.e. wall). The robot has memory to store the grids being visited and being sensed, with relative position to the entry point to form its own reference (i.e. map). (a) Suppose all colored grids are not accessible. Please develop the RL al- gorithm to visit all white grids and leave from the entrance (indicated by green and red triangles) of the office. Can RL guarantee cleaning all grids? If so, how many steps does the robot take? If not, can you develop a nearly trivial method to ensure cleaning all grids? (b) Is TD-n learning good in this case, why? Figure 1: Layout of an Office A robot is going to clean an office with layout as Figure 1. However, this robot has no prior knowledge of office layout. The robot equips sensors to perfectly sense the status of 4 immediate neighboring grids (left, right, up, down) and move accordingly. The black grids are prohibited to enter (i.e. wall). The robot has memory to store the grids being visited and being sensed, with relative position to the entry point to form its own reference (i.e. map). (a) Suppose all colored grids are not accessible. Please develop the RL al- gorithm to visit all white grids and leave from the entrance (indicated by green and red triangles) of the office. Can RL guarantee cleaning all grids? If so, how many steps does the robot take? If not, can you develop a nearly trivial method to ensure cleaning all grids? (b) Is TD-n learning good in this case, why? Figure 1: Layout of an Office