这是indexloc提供的服务，不要输入任何密码
Skip to content

Feature/improvements batch #1181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jul 31, 2024
Merged

Feature/improvements batch #1181

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
167d4bb
Batch: important new functionality
MischaPanch Apr 30, 2024
d3905c3
Batch: fixed hasnull return
MischaPanch May 3, 2024
08db707
Merge branch 'refs/heads/thuml-master' into feature/improvements-batch
MischaPanch Jul 28, 2024
ce6e34b
Improved typing, bugfix in Batch.apply_array_func
MischaPanch Jul 29, 2024
2480a62
Improved typing in Batch and Protocol, added get and pop explicitly
MischaPanch Jul 29, 2024
539c159
Fixed Batch.to_torch (was previously copying the array)
MischaPanch Jul 29, 2024
94a66a0
Spelling
MischaPanch Jul 29, 2024
cbba461
More input validation in Batch.cat_
MischaPanch Jul 29, 2024
61c4ffd
Fixed batch schema comparison in cat_
MischaPanch Jul 30, 2024
2a87a9a
Typo [skip ci]
MischaPanch Jul 30, 2024
2d4266d
Tests: fixed bug in MoveToRightEnv (info dict had wrong structure)
MischaPanch Jul 30, 2024
defcffb
Fix in tensor type handling in bcq and cql learn
MischaPanch Jul 31, 2024
a9a8af4
Tests: fixed info of MoveToRightEnv
MischaPanch Jul 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/spelling_wordlist.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,4 +270,4 @@ v_s
v_s_
obs
obs_next

dtype
18 changes: 11 additions & 7 deletions examples/inverse/irl_gail.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import datetime
import os
import pprint
from typing import SupportsFloat
from typing import SupportsFloat, cast

import d4rl
import gymnasium as gym
Expand All @@ -16,6 +16,7 @@
from torch.utils.tensorboard import SummaryWriter

from tianshou.data import Batch, Collector, ReplayBuffer, VectorReplayBuffer
from tianshou.data.types import RolloutBatchProtocol
from tianshou.env import SubprocVectorEnv, VectorEnvNormObs
from tianshou.policy import GAILPolicy
from tianshou.policy.base import BasePolicy
Expand Down Expand Up @@ -185,12 +186,15 @@ def dist(loc_scale: tuple[torch.Tensor, torch.Tensor]) -> Distribution:

for i in range(dataset_size):
expert_buffer.add(
Batch(
obs=dataset["observations"][i],
act=dataset["actions"][i],
rew=dataset["rewards"][i],
done=dataset["terminals"][i],
obs_next=dataset["next_observations"][i],
cast(
RolloutBatchProtocol,
Batch(
obs=dataset["observations"][i],
act=dataset["actions"][i],
rew=dataset["rewards"][i],
done=dataset["terminals"][i],
obs_next=dataset["next_observations"][i],
),
),
)
print("dataset loaded")
Expand Down
6 changes: 4 additions & 2 deletions test/base/env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,14 +147,16 @@ def step(self, action: np.ndarray | int): # type: ignore[no-untyped-def] # cf.
if self.index == self.size:
self.terminated = True
return self._get_state(), self._get_reward(), self.terminated, False, {}

info_dict = {"key": 1, "env": self}
if action == 0:
self.index = max(self.index - 1, 0)
return (
self._get_state(),
self._get_reward(),
self.terminated,
False,
{"key": 1, "env": self} if self.dict_state else {},
info_dict,
)
if action == 1:
self.index += 1
Expand All @@ -164,7 +166,7 @@ def step(self, action: np.ndarray | int): # type: ignore[no-untyped-def] # cf.
self._get_reward(),
self.terminated,
False,
{"key": 1, "env": self},
info_dict,
)
return None

Expand Down
Loading