Continuous Environment An Environment is continuous if its set of states or the Agents set of Actions is non-finite and uncountable.