add PSRL policy #202
add PSRL policy #202
Conversation
There was a problem hiding this comment.
Again, please refer to https://tianshou.readthedocs.io/en/latest/contributing.html to correct the errors in the unit test, including code-style checking and other unit tests.
| self.rew_mean = (self.rew_mean * self.rew_count + rew_sum) / sum_count | ||
| self.rew_square_sum += rew_square_sum | ||
| raw_std2 = self.rew_square_sum / sum_count - self.rew_mean ** 2 | ||
| self.rew_std = np.sqrt(1 / ( |
There was a problem hiding this comment.
I don't understand this. Can you explain it more?
There was a problem hiding this comment.
The line of calculating self.rew_std is strange.
| state: Optional[Any] = None, | ||
| info: Dict[str, Any] = {}, |
There was a problem hiding this comment.
These two arguments are not used.
There was a problem hiding this comment.
That's to be consistent with model API
There was a problem hiding this comment.
What do you mean by "model API"? PSRLModel is the first model and I don't see it has any base-class.
|
I can roughly get the main idea of PSRL, but still cannot understand some of the details, especially how to update the |
Add PSRL policy in tianshou/policy/modelbase/psrl.py. Co-authored-by: n+e <trinkle23897@cmu.edu>
Add PSRL policy in
tianshou/policy/modelbase/psrl.py.