+
Skip to content

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Apr 9, 2025

Stack from ghstack (oldest at bottom):

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Apr 9, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2898

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 3 Pending, 3 Unrelated Failures

As of commit 66a7806 with merge base 8ce11a8 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens pushed a commit that referenced this pull request Apr 9, 2025
ghstack-source-id: 1697be8
Pull Request resolved: #2898
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 9, 2025
@vmoens vmoens added the enhancement New feature or request label Apr 9, 2025
@vmoens vmoens added the Environments Adds or modifies an environment wrapper label Apr 9, 2025
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Apr 9, 2025
ghstack-source-id: 41c6b99
Pull Request resolved: #2898
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Apr 9, 2025
ghstack-source-id: b85faa1
Pull Request resolved: #2898
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Apr 9, 2025
ghstack-source-id: e089186
Pull Request resolved: #2898
@vmoens vmoens merged commit 66a7806 into gh/vmoens/127/base Apr 9, 2025
53 of 70 checks passed
vmoens pushed a commit that referenced this pull request Apr 9, 2025
ghstack-source-id: e089186
Pull Request resolved: #2898
@vmoens vmoens deleted the gh/vmoens/127/head branch April 9, 2025 15:34
pip3 install ale-py
pip3 install sympy
pip3 install "gymnasium[accept-rom-license,mujoco]<1.0" mo-gymnasium[mujoco]
pip3 install "gymnasium[accept-rom-license,mujoco]>=1.1" mo-gymnasium[mujoco]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, gymnasium[accept-rom-license] shouldn't exist for >=1.0

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks I edited that!

@JonDum
Copy link

JonDum commented Apr 11, 2025

Hey @vmoens thanks for getting this out. After a lot of struggle, I was finally able to test this out on yesterday's nightly build using a reference from the sota/decision_transformer example and I'm running into an error that has me stuck thinking it's part of this transition to gymnasium 1.1.

I'm largely just adapting this:

def make_base_env(env_cfg, from_pixels=False, device=None):
set_gym_backend(env_cfg.backend).set()
env_library = LIBS[env_cfg.library]
env_name = env_cfg.name
frame_skip = env_cfg.frame_skip
env_kwargs = {
"env_name": env_name,
"frame_skip": frame_skip,
"from_pixels": from_pixels,
"pixels_only": False,
}
if env_library is DMControlEnv:
env_task = env_cfg.task
env_kwargs.update({"task_name": env_task})
env = env_library(**env_kwargs, device=device)
return env

Docs are sparse on custom environments but I was able to piece together through other examples that I needed to set the backed to gymnasium then register the env with EnvBase like so:

def make_base_env(env_cfg):
    set_gym_backend(env_cfg.backend).set()
    EnvBase.register_gym(
        "MyCustomEnv"
        entry_point=MyCustomEnv
        backend="gymnasium",
    )
    env = GymEnv(env_cfg.name)
    return env

where MyCustomEnv is

import gymnasium as gym import numpy as np
from gymnasium.spaces import Box

class MyCustomEnv(gym.Env):
    metadata = {
			"render_modes": [],
			"autoreset_mode": gym.vector.AutoresetMode.DISABLED,
    }

    def __init__(self, config: dict):
        self.observation_space = Box(-1.0, 1.0, (5,), dtype=np.float32)
        self.action_space = gym.spaces.Discrete(3)

    def reset(self, seed=None, options=None, **kwargs):
        super().reset(seed=seed, options=options, **kwargs)
        info = {}
				...
        return observation, info

    def step(self, action: np.int32):
        self.num_steps += 1
        reward = 1
        return (observation, reward, terminated, truncated, info)

I get the error:

TypeError: register() got an unexpected keyword argument 'autoreset'
 .venv/lib/python3.12/site-packages/torchrl/envs/common.py:2601  
 in _register_gym                                                                                 
                                                                                                  
   2598 │   │   │   transform=transform,                                                          
   2599 │   │   │   **kwargs,                                                                     
   2600 │   │   )                                                                                 
 ❱ 2601 │   │   return gymnasium.register(                                                        
   2602 │   │   │   id=id,                                                                        
   2603 │   │   │   entry_point=entry_point,                                                      
   2604 │   │   │   reward_threshold=reward_threshold,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request Environments Adds or modifies an environment wrapper

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载