ShmemVectorEnv Implementation #174

ChenDRAG · 2020-08-01T08:22:30Z

Following OpenAI baseline, I implement ShmemVectorEnv which is an optimized version of SubprocVectorEnv that uses shared variables to communicate observations. ShmemVectorEnv has exactly the same API as SubprocVectorEnv, but when obs.shape is too large, the usage of shared variables can save us a lot of time.

This pr include:

Basic Implementation of ShmemVectorEnv
update in test_env.py to test ShmemVectorEnv
some improvement in of test_env.py for generalization
some fix in test/base/env.py and test_env.py

I borrow from openai baseline to implement ShmemVectorEnv Class. Please refer to https://github.com/openai/baselines/blob/master/baselines/common/vec_env/shmem_vec_env.py

test/base/test_env.py

tianshou/env/vecenv/shmemenv.py

Trinkle23897 · 2020-08-01T12:30:33Z

Also need to modify the documentation somewhere.

codecov-commenter · 2020-08-02T00:30:00Z

Codecov Report

Merging #174 into dev will increase coverage by 0.46%.
The diff coverage is 88.46%.

@@            Coverage Diff             @@
##              dev     #174      +/-   ##
==========================================
+ Coverage   88.93%   89.40%   +0.46%     
==========================================
  Files          38       39       +1     
  Lines        2214     2321     +107     
==========================================
+ Hits         1969     2075     +106     
- Misses        245      246       +1

Flag	Coverage Δ
#unittests	`89.40% <88.46%> (+0.46%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
tianshou/env/vecenv/shmemenv.py	`88.23% <88.23%> (ø)`
tianshou/env/__init__.py	`100.00% <100.00%> (ø)`
tianshou/env/vecenv/subproc.py	`84.88% <100.00%> (ø)`
tianshou/data/collector.py	`93.90% <0.00%> (+0.03%)`	⬆️
tianshou/data/buffer.py	`92.59% <0.00%> (+0.11%)`	⬆️
tianshou/trainer/onpolicy.py	`94.02% <0.00%> (+16.41%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 996e2f7...46496e7. Read the comment docs.

Trinkle23897 · 2020-08-02T03:53:33Z

How about #166?

duburcqa · 2020-08-02T07:22:39Z

It would be nice to refactor this module, to avoid many code duplications. But it could be done in another PR.

Trinkle23897 · 2020-08-02T07:23:40Z

It would be nice to refactor this module, to avoid many code duplications. But it could be done in another PR.

Yes, I think so. Another PR will reconstruct these vecenv codes to support wait_num.

ChenDRAG · 2020-08-03T11:08:00Z

recurse dict is supported now. Ready for review.

How about #166?
I think this should be done in another PR, 'cause it is not related to Shmemenv?

tianshou/env/__init__.py

tianshou/env/vecenv/shmemenv.py

test/base/test_buffer.py

test/base/test_env.py

youkaichao

Nice PR for memory efficiency and speed with large observations 👍

* add shmem vecenv, some add&fix in test_env * generalize test_env IO * pep8 fix * comment update * style change * pep8 fix * style fix * minor fix * fix a bug * test fix * change env * testenv bug fix& shmem support recurse dict * bugfix * pep8 fix * _NP_TO_CT enhance * doc update * docstring update * pep8 fix * style change * style fix * remove assert * minor Co-authored-by: Trinkle23897 <463003665@qq.com>

ChenDRAG and others added 7 commits August 1, 2020 15:25

add shmem vecenv, some add&fix in test_env

730ec9d

generalize test_env IO

b96cc74

pep8 fix

6f280bb

comment update

2519ba7

style change

e829e6c

pep8 fix

7ffe797

Merge branch 'dev' into scheme

d4b06bf

ChenDRAG changed the title ~~Scheme~~ ShmemVectorEnv Implementation Aug 1, 2020

Trinkle23897 reviewed Aug 1, 2020

View reviewed changes

test/base/test_env.py Outdated Show resolved Hide resolved

test/base/test_env.py Outdated Show resolved Hide resolved

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

ChenDRAG and others added 2 commits August 1, 2020 17:04

style fix

14487f2

minor fix

a2764b0

ChenDRAG commented Aug 1, 2020

View reviewed changes

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

fix a bug

3a3edcc

youkaichao reviewed Aug 1, 2020

View reviewed changes

tianshou/env/vecenv/shmemenv.py Show resolved Hide resolved

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

youkaichao reviewed Aug 1, 2020

View reviewed changes

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

ChenDRAG added 2 commits August 1, 2020 22:52

test fix

5151495

merge

bdb68b1

Merge branch 'dev' into scheme

c6c7c67

Trinkle23897 and others added 6 commits August 2, 2020 18:24

Merge branch 'dev' into scheme

46496e7

change env

d1a6853

testenv bug fix& shmem support recurse dict

ea62f83

bugfix

3d38795

pep8 fix

ea084a9

_NP_TO_CT enhance

7fe5b4f

ChenDRAG added 4 commits August 3, 2020 21:13

doc update

7b32cd0

docstring update

8de6bbe

pep8 fix

d92b8e6

style change

205265a

ChenDRAG requested review from Trinkle23897, youkaichao and duburcqa August 3, 2020 15:01

youkaichao reviewed Aug 3, 2020

View reviewed changes

tianshou/env/__init__.py Outdated Show resolved Hide resolved

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

tianshou/env/vecenv/shmemenv.py Outdated Show resolved Hide resolved

style fix

f7b9d95

ChenDRAG requested a review from youkaichao August 4, 2020 02:06

remove assert

10ff4ad

youkaichao reviewed Aug 4, 2020

View reviewed changes

test/base/test_buffer.py Show resolved Hide resolved

test/base/test_env.py Outdated Show resolved Hide resolved

minor

eacde86

youkaichao approved these changes Aug 4, 2020

View reviewed changes

Trinkle23897 approved these changes Aug 4, 2020

View reviewed changes

youkaichao merged commit f2bcc55 into thu-ml:dev Aug 4, 2020

ChenDRAG deleted the scheme branch August 4, 2020 07:58

Trinkle23897 added enhancement Feature that is not a new algorithm or an algorithm enhancement and removed enhancement Feature that is not a new algorithm or an algorithm enhancement labels Aug 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ShmemVectorEnv Implementation #174

ShmemVectorEnv Implementation #174

Uh oh!

ChenDRAG commented Aug 1, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Trinkle23897 commented Aug 1, 2020 •

edited by youkaichao

Loading

Uh oh!

codecov-commenter commented Aug 2, 2020 •

edited

Loading

Uh oh!

Trinkle23897 commented Aug 2, 2020

Uh oh!

duburcqa commented Aug 2, 2020

Uh oh!

Trinkle23897 commented Aug 2, 2020

Uh oh!

ChenDRAG commented Aug 3, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

youkaichao left a comment

Uh oh!

Uh oh!

ShmemVectorEnv Implementation #174

ShmemVectorEnv Implementation #174

Uh oh!

Conversation

ChenDRAG commented Aug 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Trinkle23897 commented Aug 1, 2020 • edited by youkaichao Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Aug 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Trinkle23897 commented Aug 2, 2020

Uh oh!

duburcqa commented Aug 2, 2020

Uh oh!

Trinkle23897 commented Aug 2, 2020

Uh oh!

ChenDRAG commented Aug 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

youkaichao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ChenDRAG commented Aug 1, 2020 •

edited

Loading

Trinkle23897 commented Aug 1, 2020 •

edited by youkaichao

Loading

codecov-commenter commented Aug 2, 2020 •

edited

Loading

ChenDRAG commented Aug 3, 2020 •

edited

Loading