这是indexloc提供的服务，不要输入任何密码
Skip to content

Improve interface of BasePolicy.compute_action #1170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 18, 2024
Merged

Improve interface of BasePolicy.compute_action #1170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
408f052
Improve interface of BasePolicy.compute_action, supporting array-like
opcode81 Jul 8, 2024
42642b0
Merge branch 'master' into improve-compute-action
MischaPanch Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
- policy:
- introduced attribute `in_training_step` that is controlled by the trainer. #1123
- policy automatically set to `eval` mode when collecting and to `train` mode when updating. #1123
- Extended interface of `compute_action` to also support array-like inputs #1169
- `highlevel`:
- `SamplingConfig`:
- Add support for `batch_size=None`. #1077
Expand Down
7 changes: 4 additions & 3 deletions tianshou/policy/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import torch
from gymnasium.spaces import Box, Discrete, MultiBinary, MultiDiscrete
from numba import njit
from numpy.typing import ArrayLike
from overrides import override
from torch import nn

Expand Down Expand Up @@ -289,7 +290,7 @@ def soft_update(self, tgt: nn.Module, src: nn.Module, tau: float) -> None:

def compute_action(
self,
obs: arr_type,
obs: ArrayLike,
info: dict[str, Any] | None = None,
state: dict | BatchProtocol | np.ndarray | None = None,
) -> np.ndarray | int:
Expand All @@ -300,8 +301,8 @@ def compute_action(
:param state: the hidden state of RNN policy, used for recurrent policy.
:return: action as int (for discrete env's) or array (for continuous ones).
"""
# need to add empty batch dimension
obs = obs[None, :]
obs = np.array(obs) # convert array-like to array (e.g. LazyFrames)
obs = obs[None, :] # add batch dimension
obs_batch = cast(ObsBatchProtocol, Batch(obs=obs, info=info))
act = self.forward(obs_batch, state=state).act.squeeze()
if isinstance(act, torch.Tensor):
Expand Down
Loading