+
Skip to content

Proof-of-concept implementation of CLI list command #3273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

Erotemic
Copy link

There have been a few times where I've wished I could use the CLI to list or describe the data in a folder / collection / item / file.

I have a basic implementation of this feature. It works well enough, but it could be better.

I''ve currently tested with the following commands to list items in the VIAME girder collection:

    girder-client --api-url https://data.kitware.com/api/v1 list 58b747ec8d777f0aef5d0f6a

    girder-client --api-url https://data.kitware.com/api/v1 list 58c49f668d777f0aef5d7960

    girder-client --api-url https://data.kitware.com/api/v1 list 5c423a5f8d777f072b0ba58f
    
    girder-client --api-url https://data.kitware.com/api/v1 list 5dd3181eaf2e2eed3505827c

    girder-client --api-url https://data.kitware.com/api/v1 list 5aac22638d777f068578d53c

This results in the following output:

(py38) joncrall@Ooo:~/code/girder$     girder-client --api-url https://data.kitware.com/api/v1 list 58b747ec8d777f0aef5d0f6a
Listing collection: 58b747ec8d777f0aef5d0f6a - VIAME
=== folder ===
ID                       TYPE   NAME                    
5dfcf2efaf2e2eed35a52a19 folder 3d_models
5c82ca1c8d777f072b8a85f9 folder annnotations
58d4c1178d777f0aef5d8920 folder binary_releases
5ba3d4348d777f06b9fee5b3 folder dependencies
58c49f668d777f0aef5d7960 folder detection_models
58e5149b8d777f0aef5d8d76 folder documentation
58c49f708d777f0aef5d7961 folder example_images
5b6efbb18d777f06857c32a7 folder hidden
5d362147877dfcc902244b0b folder jenkins_status
5aac22638d777f068578d53c folder other
595d10518d777f16d01e99cb folder videos
(py38) joncrall@Ooo:~/code/girder$ 
(py38) joncrall@Ooo:~/code/girder$     girder-client --api-url https://data.kitware.com/api/v1 list 58c49f668d777f0aef5d7960
Parent collection: 58b747ec8d777f0aef5d0f6a - VIAME
Listing folder: 58c49f668d777f0aef5d7960 - detection_models
=== folder ===
ID                       TYPE   NAME                    
5c82be098d777f072b8a66d8 folder microsoft
5dd3181eaf2e2eed3505827c folder netharn
5c82be028d777f072b8a66ae folder xnor
=== item ===
ID                       TYPE   NAME                    
5cb74eeb8d777f072b40f904 item   models-arctic_seal_tf-v1.1.1.tar.gz
5cb74eeb8d777f072b40f906 file   models-arctic_seal_tf-v1.1.1.tar.gz
5d606146d35580e6dcb70c06 item   models-arctic_seal_transforms-v0.1.tar.gz
5d606146d35580e6dcb70c08 file   models-arctic_seal_transforms-v0.1.tar.gz
5d6edf99d35580e6dcddbc3b item   models-arctic_seal_transforms-v0.2.tar.gz
5d6edf99d35580e6dcddbc3f file   models-arctic_seal_transforms-v0.2.tar.gz
5cb80b008d777f072b44a652 item   models-arctic_seal_yolo-v1.2.tar.gz
5cb80b008d777f072b44a654 file   models-arctic_seal_yolo-v1.2.tar.gz
5cbb84d08d777f072b53ab46 item   models-arctic_seal_yolo-v1.3.tar.gz
5cbb84d08d777f072b53ab48 file   models-arctic_seal_yolo-v1.3.tar.gz
5afe6c018d777f15ebe1d6f2 item   models-blue_marble_context-v1.0.0.tar.gz
5afe6c018d777f15ebe1d6f3 file   models-blue_marble_context-v1.0.0.tar.gz
5afe6e1e8d777f15ebe1d6fc item   models-default_caffe_descriptor-v1.0.0.tar.gz
5afe6e1e8d777f15ebe1d6fd file   models-default_caffe_descriptor-v1.0.0.tar.gz
5afe74fc8d777f15ebe1d70c item   models-default_faster_rcnn-v1.0.0.tar.gz.tar.gz
5afe74fc8d777f15ebe1d70d file   models-default_faster_rcnn-v1.0.0.tar.gz.tar.gz
5afe74f68d777f15ebe1d709 item   models-default_yolo_v2-v1.0.0.tar.gz
5afe74f78d777f15ebe1d70a file   models-default_yolo_v2-v1.0.0.tar.gz
5b5e5aab8d777f06857c17d0 item   models-default_yolo_v2-v2.0.0.tar.gz
5b5e5aab8d777f06857c17d1 file   models-default_yolo_v2-v2.0.0.tar.gz
5b5f00d18d777f06857c17fa item   models-default_yolo_v2-v2.1.0.tar.gz
5b5f00d18d777f06857c17fb file   models-default_yolo_v2-v2.1.0.tar.gz
5afe74ef8d777f15ebe1d706 item   models-habcam-v1.0.0.tar.gz
5afe74ef8d777f15ebe1d707 file   models-habcam-v1.0.0.tar.gz
5b5e27cc8d777f06857c17c2 item   models-habcam-v2.0.0.tar.gz
5b5e27cc8d777f06857c17c3 file   models-habcam-v2.0.0.tar.gz
5cdf04218d777f072bb4fc37 item   models-habcam-v2.1.0.tar.gz
5cdf04218d777f072bb4fc39 file   models-habcam-v2.1.0.tar.gz
5dd41087af2e2eed350a3bb3 item   models-habcam-v2.2.0.tar.gz
5dd41087af2e2eed350a3bb5 file   models-habcam-v2.2.0.tar.gz
5e12b04aaf2e2eed35d42979 item   models-mdnet_tracking_models-v1.0.0.tar.gz
5e12b04aaf2e2eed35d4297b file   models-mdnet_tracking_models-v1.0.0.tar.gz
5cdc4ff88d777f072bad5473 item   models-mmdet_detection-v1.0.0.tar.gz
5cdc4ff88d777f072bad5475 file   models-mmdet_detection-v1.0.0.tar.gz
5cdda0b78d777f072bb09919 item   models-mmdet_detection-v1.1.0.tar.gz
5cdda0b78d777f072bb0991b file   models-mmdet_detection-v1.1.0.tar.gz
5b0455de8d777f15ebe1efbe item   models-mouss-generic-v1.0.0.tar.gz
5b0455de8d777f15ebe1efbf file   models-mouss-generic-v1.0.0.tar.gz
5b7216e68d777f06857c4557 item   models-mouss-v2.0.0.tar.gz
5b7216e68d777f06857c4558 file   models-mouss-v2.0.0.tar.gz
5bad56158d777f06b908c473 item   models-mouss-v2.1.0.tar.gz
5bad56168d777f06b908c475 file   models-mouss-v2.1.0.tar.gz
5e9ba3919014a6d84e18ff5d item   models-netharn_detection-v1.0.0.tar.gz
5e9ba3919014a6d84e18ff5f file   models-netharn_detection-v1.0.0.tar.gz
5eaa2cff9014a6d84e3dc9a3 item   models-netharn_detection-v1.0.2.tar.gz
5eaa2cff9014a6d84e3dc9a5 file   models-netharn_detection-v1.0.2.tar.gz
5e3b1124af2e2eed356ee701 item   models-pyrosome-v1.0.1.tar.gz
5e3b1124af2e2eed356ee703 file   models-pyrosome-v1.0.1.tar.gz
5d2f87ac877dfcc90212d12c item   models-pysot_tracking_models-v1.0.0.tar.gz
5d30aed3877dfcc902188274 file   models-pysot_tracking_models-v1.0.0.tar.gz
5afe6d678d777f15ebe1d6f6 item   models-pytorch_descriptors-v1.0.0.tar.gz
5afe6d678d777f15ebe1d6f7 file   models-pytorch_descriptors-v1.0.0.tar.gz
5bad56178d777f06b908c47c item   models-pytorch_descriptors-v1.1.0.tar.gz
5bad56178d777f06b908c47e file   models-pytorch_descriptors-v1.1.0.tar.gz
5afe6d6d8d777f15ebe1d6f9 item   models-pytorch_tracking-v1.0.0.tar.gz
5afe6d6d8d777f15ebe1d6fa file   models-pytorch_tracking-v1.0.0.tar.gz
5b6890a38d777f06857c1f22 item   models-pytorch_tracking-v1.1.0.tar.gz
5b6890a38d777f06857c1f23 file   models-pytorch_tracking-v1.1.0.tar.gz
5afe74e68d777f15ebe1d703 item   models-yolo_v2_seed_model-v1.0.0.tar.gz
5afe74e68d777f15ebe1d704 file   models-yolo_v2_seed_model-v1.0.0.tar.gz
5c1a59468d777f072bdbaca2 item   models-yolo_v2_seed_model-v1.1.0.tar.gz
5c1a59468d777f072bdbaca4 file   models-yolo_v2_seed_model-v1.1.0.tar.gz
5c423a5f8d777f072b0ba58f item   models-yolo_v2_seed_model-v1.2.0.tar.gz
5c423a5f8d777f072b0ba591 file   models-yolo_v2_seed_model-v1.2.0.tar.gz
5c115d918d777f2179e5ae9a item   models-yolo_v3_seed_model-v1.0.0.tar.gz
5c115d918d777f2179e5ae9c file   models-yolo_v3_seed_model-v1.0.0.tar.gz
5c1a59488d777f072bdbacac item   models-yolo_v3_seed_model-v1.1.0.tar.gz
5c1a59488d777f072bdbacae file   models-yolo_v3_seed_model-v1.1.0.tar.gz
5c423a608d777f072b0ba599 item   models-yolo_v3_seed_model-v1.2.0.tar.gz
5c423a618d777f072b0ba59b file   models-yolo_v3_seed_model-v1.2.0.tar.gz
(py38) joncrall@Ooo:~/code/girder$ 
(py38) joncrall@Ooo:~/code/girder$     girder-client --api-url https://data.kitware.com/api/v1 list 5c423a5f8d777f072b0ba58f
Parent folder: 58c49f668d777f0aef5d7960 - detection_models
Listing item: 5c423a5f8d777f072b0ba58f - models-yolo_v2_seed_model-v1.2.0.tar.gz
=== file ===
ID                       TYPE   NAME                    
5c423a5f8d777f072b0ba58f item   models-yolo_v2_seed_model-v1.2.0.tar.gz
5c423a5f8d777f072b0ba591 file   models-yolo_v2_seed_model-v1.2.0.tar.gz
(py38) joncrall@Ooo:~/code/girder$     
(py38) joncrall@Ooo:~/code/girder$     girder-client --api-url https://data.kitware.com/api/v1 list 5dd3181eaf2e2eed3505827c
Parent folder: 58c49f668d777f0aef5d7960 - detection_models
Listing folder: 5dd3181eaf2e2eed3505827c - netharn
=== folder ===
ID                       TYPE   NAME                    
=== item ===
ID                       TYPE   NAME                    
5e8de0932660cbefbaa0258a item   deploy_MM_CascadeRCNN_etvvhzni_007_IPEIQA_multiclass.zip
5e8de0932660cbefbaa0258c file   deploy_MM_CascadeRCNN_etvvhzni_007_IPEIQA_multiclass.zip
5dd3eb8eaf2e2eed3508d601 item   deploy_MM_CascadeRCNN_myovdqvi_035_MVKVVR_fix3.zip
5dd3eb8eaf2e2eed3508d604 file   deploy_MM_CascadeRCNN_myovdqvi_035_MVKVVR_fix3.zip
5eb9c21f9014a6d84e638b47 item   deploy_MM_CascadeRCNN_rgb-fine-coi-v40_ntjzrxlb_007_FVMWBU.zip
5eb9c21f9014a6d84e638b49 file   deploy_MM_CascadeRCNN_rgb-fine-coi-v40_ntjzrxlb_007_FVMWBU.zip
5dcf2e68af2e2eed35fb5b11 item   deploy_scallop_MM_CascadeRCNN_myovdqvi_035_MVKVVR_fix2.zip
5dcf2e68af2e2eed35fb5b13 file   deploy_scallop_MM_CascadeRCNN_myovdqvi_035_MVKVVR_fix2.zip
(py38) joncrall@Ooo:~/code/girder$ 
(py38) joncrall@Ooo:~/code/girder$     girder-client --api-url https://data.kitware.com/api/v1 list 5aac22638d777f068578d53c
Parent collection: 58b747ec8d777f0aef5d0f6a - VIAME
Listing folder: 5aac22638d777f068578d53c - other
=== folder ===
ID                       TYPE   NAME                    
5ee6a5149014a6d84ec02d66 folder sealion_edits
5b314cb78d777f2e62258d62 folder transfer
5b35be588d777f2e6225928a folder transfer2
5b4521cd8d777f2e62259d5a folder transfer3
=== item ===
ID                       TYPE   NAME                    
5d66b64dd35580e6dcca814b item   older-annotator-tool.zip
5d66b64ed35580e6dcca814d file   older-annotator-tool.zip
5b64a3a98d777f06857c1ab5 item   short-sequences.tar.gz
5b64a3a98d777f06857c1ab6 file   short-sequences.tar.gz
5aac227e8d777f068578d53e item   viame-testing.tar.gz
5aac227e8d777f068578d53f file   viame-testing.tar.gz

It would be nice to harden up this CLI, or even make it easier to implement on the CLI side. It would be nice to have a query-by-name feature, so you could traverse girder in a similar way that you would traverse a file system. It would also be nice to list extra column information like sha512 hashes if they exist.

There are probably a lot of directions you could go with this, but I think the girder-cli is lacking in that it has no way to query what data exists. So this is an attempt to fix that.

@jcfr
Copy link
Contributor

jcfr commented Jul 3, 2020

I think the girder-cli is lacking in that it has no way to query what data exists. So this is an attempt to fix that

I support this. Thanks for working on it 🙏

Ideally:

  • two types of output:
  • select type of data to list
  • recursive vs direct child

An other command to add would be metadata

@Erotemic
Copy link
Author

Was there a reason this was closed? Can this get re-opened?

I find myself needing this again. I rebased this PR on the latest master, and I've done a bit of cleanup. For my current use-case I want to add the ability for users to specify a few more information columns, which I don't think will be difficult.

I've added initial logic for a json machine readable output, but it needs a bit of work.

I have a backup of the original PR here: https://github.com/Erotemic/girder/tree/dev/cli_list_old

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载