+
Skip to content

Conversation

will-moore
Copy link
Member

@will-moore will-moore commented Mar 3, 2025

Since we are starting to use externalInfo in IDR and possibly for other users - see https://github.com/ome/omero-cli-zarr/pull/167/files, it may be useful to display this in web. Glencoe already use this.

Updated description based on discussions below:

  • We don't show anything new by default since ExternalInfo is considered to be "debug" info only.

  • You can show it by using the browser dev Console to show elements with debug class with: $(".debug").show()

  • We aim to load externalInfo along with the original objects (instead of an extra lazy loading call)

  • getObject() and getExternalInfo() load externalInfo omero-py#453 updates getObject() etc to include externalInfo

  • We use that PR to display in the right panel (nothing is shown if externalInfo is not found)

  • NB: if the omero-py PR is missing, then this will fail silently since obj.getExternalInfo will not be found. No need for this PR to depend on getObject() and getExternalInfo() load externalInfo omero-py#453

Screenshot 2025-03-14 at 17 20 13

cc @knabar @dominikl

@will-moore will-moore marked this pull request as draft March 3, 2025 15:50
Copy link
Member

@sbesson sbesson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@will-moore thanks for starting this conversation. As noted above, Glencoe is using ExternalInfo quite extensively so we are interested in ways to expose this metadata via the OMERO.web API and/or UI.

Starting with with your last question, my immediate feeling is that it would be most beneficial to include this functionality in the OMERO.web JSON API. A few reasons for that:

  • as explained in https://omero.readthedocs.io/en/stable/developers/Web.html, the /webclient component should be considered as internal only. If the intent is to expose this metadata as JSON for consumption (both internal and external), api/ is the location of choice
  • omero-marshal already has support for ExternalInfo encoding/decoding so there is no complex cross-component work
  • the usage of ExternalInfo is not limited to Image and can be applied to any object. Glencoe uses this on Mask objects to represent labels for instance. Having api/v0/<images,rois,...>/<id> loading and exposing the external info in a unified manner would be consistent.

On the query itself, my impression is that externalInfo should be treated the same way as other details. I don't expect the additional join to be expensive in terms of query time given these are relatively shallow objects with only a few attributes but that should certainly be reviewed as par of the functional testing.

The UI questions is probably the one where I am the less opinionated and I have a suspicion the answer might be "it depends". For Zarr data, we have used ExternalInfo as an mechanism to store the location of the data. The closest equivalent would be the legacy OMERO pyramids which are not exposed in the UI. The most relevant existing icon would be the ../../. in the right-hand panel which lists the Fileset entries.
However other ExternalInfo objects might be more naturally suited either as Image details or as additional attributes

@knabar
Copy link
Member

knabar commented Mar 4, 2025

Agree with @sbesson; I would probably not create a separate view for retrieving externalInfo, but get it with the other object details on an existing call. Any performance hit should be minimal compared to having a whole separate call.

@will-moore will-moore marked this pull request as ready for review March 13, 2025 16:09
@will-moore will-moore changed the title Add /webclient/extinfo/image/ID/ for extInfo as JSON externalInfo display Mar 13, 2025
@will-moore
Copy link
Member Author

@sbesson This is now deployed on idr-testing. e.g. https://idr-testing.openmicroscopy.org/webclient/?show=image-15160031

@joshmoore
Copy link
Member

As a quick sidenote before I disappear for a week, I'd be more than happy if we hide the acronym "LSID" from users.

@dominikl
Copy link
Member

👍 Looks good to me. What does LSID actually stand for, and what is the ExternalInfo actually used for generally @joshmoore ?

@joshmoore
Copy link
Member

LSID stands for "Life Science Identifier". https://www.lsid.info/ et al. https://zenodo.org/records/46804 is a pretty good read.

The ExternalInfo object wasn't used a lot previously but was intended to hold info of the objects that felt "too detailed" to expose in the main object. It also helps that if you have an opaque identifier that you don't have to check every table to find the object but you can look in the one external info table.

@will-moore
Copy link
Member Author

https://en.wikipedia.org/wiki/LSID.
@joshmoore "Hide LSID from users" - what about CLI users? - in ome/omero-cli-zarr#167 we have --lsid and in general we try to make the UI reflect the data model.

Do we use another term or just e.g.

http://path/to/my/image.zarr
entityType: com.glencoesoftware.ngff.multiscales
entityId: 3

@joshmoore
Copy link
Member

in general we try to make the UI reflect the data model.

"Attribues"? "Attachments"? 😄

what about CLI users?

I certainly worry about CLI users less.

But if we need to keep it, that's fine, just more than the other terms, "LSID" was a mistake.

@will-moore
Copy link
Member Author

As discussed in web meeting, we probably don't want ExternalInfo to be so much "in your face" since it's not of interest to most users.
Could look at hiding it under ../../ or elsewhere.

@will-moore
Copy link
Member Author

Deployed latest changes to idr-testing and updated screenshot above.

@will-moore
Copy link
Member Author

To help testing, you can now set ExternalInfo via the CLI - see ome/omero-py#453

@will-moore
Copy link
Member Author

In IDR (and regular OMERO if/when omero-zarr-pixel-buffer becomes adopted) we want it to be nice and easy for a user to get the OME-Zarr URL for an image they are looking at, as they can do for e.g. https://idr.github.io/ome-ngff-samples/.

We probably don't want to show "External Info" or com.glencoesoftware.ngff.multiscales but for Images where the extInfo EntityType is com.glencoesoftware.ngff.multiscales, we should show the OME-Zarr URL with a handy "copy to clipboard" button.
This could be in addition to the ExternalInfo display as above, which would support all Objects (not just images) and other EntityTypes.

@will-moore
Copy link
Member Author

As discussed in web meeting today:

  • Exposing externalInfo to users is not useful in IDR since this is a path/to/zarr rather than a public s3.
  • It is only useful for debugging etc, so let's hide it with a .debug: {display: none} css then we can easily show all such UI elements with:
$(‘.debug’).show()

@will-moore
Copy link
Member Author

As discussed with @jburel...

Since we have addressed s3 backend issues in omero-zarr-pixel-buffer, we can expect to use genuine public URLs for external-info, so this should be easier for the user to find.

This also gives us the chance to add "Open with" various zarr viewers.
However, the stored externalInfo lsid doesn't include endpoint info. E.g. s3://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0062A/6001240.zarr/?anonymous=true can't be used in web based viewers. Instead we need e.g. https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0062A/6001240.zarr. Can we assume that we just replace s3 with https to get this URL (and remove ?anonymous=true)?

@sbesson
Copy link
Member

sbesson commented Sep 11, 2025

Note that everything that is discussed is conditional to the fact the externalInfo:
1- is of the correct entityType and entityId
2- is pointing at a remote resource rather than a local path

However, the stored externalInfo lsid doesn't include endpoint info. E.g. s3://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0062A/6001240.zarr/?anonymous=true can't be used in web based viewers

Not sure I understand. Unlike the s3://<bucket>/<prefix> URI used e.g. by tools like the AWS CLI, to be usable by omero-zarr-pixel-buffer, the externalinfo.lsid must include an endpoint after s3://

Can we assume that we just replace s3 with https to get this URL

At least, this should be correct for Amazon S3. The service supports to types of requests: Path-style requests and Virtual-hosted–style requests. So https://s3.<region-code>.amazonaws.com/<bucket-name>/<zarr_prefix> is a valid path-style request.
Other S3 compatible object stores might behave differently.

and remove ?anonymous=true

If the intent is to create an HTTP URL using by any a third-party application, then such transformation is probably only applicable to public data. No query parameter in a S3 lsid value indicates that credentials are expected to read the data.

@jburel
Copy link
Member

jburel commented Sep 11, 2025

@will-moore we cannot change the value to https or remove anonymous as @sbesson indicated.

We only discussed showing external info to help identifying the "registered" one not reformatting it. to use it in the web, the application will need to parse it

@will-moore
Copy link
Member Author

I understand that omero-web will need to parse it - I'm just asking if it will always work to replace s3 with https?
That's what I've implemented above and it seems to work for various lsids.

@joshmoore
Copy link
Member

I'm just asking if it will always work to replace s3 with https?

Definitely not in the general case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载