这是indexloc提供的服务,不要输入任何密码
Skip to content

Add profile workflow #143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
Aug 2, 2020
Merged

Add profile workflow #143

merged 48 commits into from
Aug 2, 2020

Conversation

ChenDRAG
Copy link
Collaborator

@ChenDRAG ChenDRAG commented Jul 17, 2020

  • add a workflow to profile batch
  • buffer profiling
  • collector profiling

youkaichao and others added 11 commits July 11, 2020 09:44
* make sure the key type of Batch is string, and add unit tests

* add is_empty() function and unit tests

* enable cat of mixing dict and Batch, just like stack
* minor polish

* improve and implement Batch.cat_

* bugfix for buffer.sample with field impt_weight

* restore the usage of a.cat_(b)

* fix 2 bugs in batch and add corresponding unittest

* code fix for update

* update is_empty to recognize empty over empty; bugfix for len

* bugfix for update and add testcase

* add testcase of update

* fix docs

* fix docs

* fix docs [ci skip]

* fix docs [ci skip]

Co-authored-by: Trinkle23897 <463003665@qq.com>
…ml#130)

* re-implement Batch.stack and add testcases

* add doc for Batch.stack

* reuse _create_values and refactor stack_ & cat_

* fix pep8

* fix docs

* raise exception for stacking with partial keys and axis!=0

* minor fix

* minor fix

Co-authored-by: Trinkle23897 <463003665@qq.com>
* remove multibuf

* reward_metric

* make fileds with empty Batch rather than None after reset

* many fixes and refactor
Co-authored-by: Trinkle23897 <463003665@qq.com>
* Enable selecting worker for vector env step method.

* Update collector to match new vecenv selective worker behavior.

* Bug fix.

* Fix rebase

Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
* code refactor; remove unused kwargs; add reward_normalization for dqn

* bugfix for __setitem__ with torch.Tensor; add Batch.condense

* minor fix

* support cat with empty Batch

* remove the dependency of is_empty on len; specify the semantic of empty Batch by test cases

* support stack with empty Batch

* remove condense

* refactor code to reflect the shared / partial / reserved categories of keys

* add is_empty(recursive=False)

* doc fix

* docfix and bugfix for _is_batch_set

* add doc for key reservation

* bugfix for algebra operators

* fix cat with lens hint

* code refactor

* bugfix for storing None

* use ValueError instead of exception

* hide lens away from users

* add comment for __cat

* move the computation of the initial value of lens in cat_ itself.

* change the place of doc string

* doc fix for Batch doc string

* change recursive to recurse

* doc string fix

* minor fix for batch doc
@Trinkle23897 Trinkle23897 changed the base branch from master to dev July 17, 2020 09:49
@Trinkle23897 Trinkle23897 changed the title Add batch profile workflow WIP: Add profile workflow Jul 17, 2020
@Trinkle23897 Trinkle23897 linked an issue Jul 17, 2020 that may be closed by this pull request
@ChenDRAG ChenDRAG closed this Jul 17, 2020
@Trinkle23897 Trinkle23897 reopened this Jul 17, 2020
@ChenDRAG
Copy link
Collaborator Author

conflict solved

ChenDRAG and others added 6 commits July 17, 2020 22:01
* add doc for len exceptions

* doc move; unify is_scalar_value function

* remove some issubclass check

* bugfix for shape of Batch(a=1)

* keep moving doc

* keep writing batch tutorial

* draft version of Batch tutorial done

* improving doc

* keep improving doc

* batch tutorial done

* rename _is_number

* rename _is_scalar

* shape property do not raise exception

* restore some doc string

* grammarly [ci skip]

* grammarly + fix warning of building docs

* polish docs

* trim and re-arrange batch tutorial

* go straight to the point

* minor fix for batch doc

* add shape / len in basic usage

* keep improving tutorial

* unify _to_array_with_correct_type to remove duplicate code

* delegate type convertion to Batch.__init__

* further delegate type convertion to Batch.__init__

* bugfix for setattr

* add a _parse_value function

* remove dummy function call

* polish docs

Co-authored-by: Trinkle23897 <463003665@qq.com>
@Trinkle23897 Trinkle23897 changed the base branch from dev to test July 20, 2020 07:45
@Trinkle23897 Trinkle23897 changed the base branch from test to dev July 20, 2020 07:45
@Trinkle23897 Trinkle23897 mentioned this pull request Jul 20, 2020
9 tasks
@Trinkle23897 Trinkle23897 changed the title WIP: Add profile workflow Add profile workflow Jul 25, 2020
@ChenDRAG
Copy link
Collaborator Author

another fix, my bad

@Trinkle23897 Trinkle23897 changed the title Add profile workflow WIP: Add profile workflow Jul 25, 2020
@codecov-commenter
Copy link

codecov-commenter commented Jul 26, 2020

Codecov Report

Merging #143 into dev will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##              dev     #143   +/-   ##
=======================================
  Coverage   88.63%   88.63%           
=======================================
  Files          38       38           
  Lines        2226     2226           
=======================================
  Hits         1973     1973           
  Misses        253      253           
Flag Coverage Δ
#unittests 88.63% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99a1d40...d1ec533. Read the comment docs.

@Trinkle23897 Trinkle23897 changed the title WIP: Add profile workflow Add profile workflow Aug 1, 2020
@duburcqa
Copy link
Collaborator

duburcqa commented Aug 1, 2020

I will have a look tomorrow.

@Trinkle23897 Trinkle23897 merged commit 996e2f7 into thu-ml:dev Aug 2, 2020
@ChenDRAG ChenDRAG deleted the addtest branch August 4, 2020 07:37
@ChenDRAG ChenDRAG restored the addtest branch August 4, 2020 07:38
BFAnas pushed a commit to BFAnas/tianshou that referenced this pull request May 5, 2024
* add a workflow to profile batch
* buffer profiling
* collector profiling

Co-authored-by: Trinkle23897 <463003665@qq.com>
Co-authored-by: Huayu Chen(陈华玉) <chenhuay17@gamil.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Batch & Buffer profiling
5 participants