+
Skip to content

openGRIS - Standard Project Contribution and Onboarding - APPROVED #326

@bansalr

Description

@bansalr

1. Describing The Contribution

Business Problem

Describe the business problem the contribution solves

Large financial organizations have a collection of distributed compute resources - physical severs, virtual instances, cloud compute (AWS, GCP, ....) and grids (e.g. IBM Symphony). The second problem is that capacity on these grids is often statically allocated and and workloads cannot dynamically resource up and down. There are multiple challenges around utilizing different compute resources efficiently which means grids often runs at low utilization (30% or below) and also worsen climate impact.

Proposed Solution

Describe the type of contribution (project or working group) and how it solves that business problem

openGRIS is an open Standard for Grid Resource Scheduling with client and worker standards to tie resources together and share them in a cost-effective and climate friendly way. openGRIS aims to provide a standard to allow different hardware and compute pools to be effectively addressed by applications across the organization - production batch processes, on-demand compute from notebooks, etc. The goal is on one side to easily unlock "elastic-compute" on demand and on the other hand enable pools of compute to be more effectively utilized. openGRIS draws inspiration from services AWS Lambda and BigQuery and software tools like Dask and Spark to propose a cloud provider and hardware independent standard.

Scope

Clearly define the scope of the standard. In cases where the scope is expected to be defined by the standard project participants this should be one of the first tasks the group completes. Guidelines can be found in the Community Specification documentation.

openGRIS will specify a definition for compute payloads for routing to compute backends. This definition will include elements which have come up in discussion with FinOS members (Intel, Nvidia, IBM and AWS) and can include directives for GPU and wide Vector instructions.

openGRIS will also specify policies which can define how those payloads are routed to various compute backends. These policies can have time and cost preferences. Finally, the definition of backends will be left out of the scope of the initial standard but the standard can be later expanded to definition and categorization of compute backends.

Tentative Roadmap and Current State

Describe the short and medium term goals and phases of the project. What does success look like for this project?

Category Milestone Target Date Status Comments
Proposal Standard 1.0 6/30/24 Done
Implementation Reference Implementation 7/30/24 Done
Implementation Open Source Scaler 9/30/24 Done github.com/citi/scaler
Proposal 1.0 Discussion with Nvidia, AWS, Intel, IBM 9/30/24 Done
Proposal Formalize Standard 2.0 2/15/25 Done Standard 2.0
Implementation Interface to IBM Symphony 1/30/25 Done Done
Implementation EC2 Static Compute Adapter 4/30/25 Pending
Implementation Multi-language bindings: C++ 6/30/25 Pending
Implementation Time based Policies for Scheduling 9/30/25 Pending
Implementation Dynamic Scaling for Symphony and EC2 10/30/25 Pending
Implementation Multi-Environment support via Docker/Podman 11/30/25 Pending
Implementation AWS HTC Grid Adapter TBD Pending

|
See detailed architecture and roadmap here.

Existing Materials

If materials already exist, provide a link to them that Foundation staff can access - if it's in a private GitHub.com repositories, you should invite the finos-admin user with R/O permissions to those repositories

  • GitHub / GitLab Repository
    • Project Name (enter here) OpenGRIS
  • Is Continuous Integration used? If so, which system is used?
  • Was the project ever released? (yes / no)
    • If so, are releases public? (yes / no)
    • And what's the latest released version?
  • Existing Project Documentation ( URL / microsite / PDF etc detail here) no
  • Does the name have a registered trademark? (yes / no) no
  • Is there a logo? (yes / no) no
  • High-Level Presentation prepared for Technical Oversight Committee (~15 mins) yes
  • Are meetings currently held for the project? (yes / no + details) no
  • Are meeting minutes, agenda and attendance tracked? (yes / no + details) no

Development Team

Maintainers

Who will be the project maintainer(s)? Provide full name, affiliation, work email address, and GitHub / GitLab username.

Name Affiliation Work Email Address Github / GitLab username
Zhuo Yin Citi zhuo.yin@citi.com @sharpener6

Confirmed contributors

If applicable, list all of the individuals that have expressed interest in and/or are committed to contributing to this project, including full name, affiliation, work email address, and GitHub.com username

Name Affiliation Work Email Address Github / GitLab username
Raphael Javaux Citi raphael.javaux@citi.com @rafa-be
George Lewis Citi george.lewis@citi.com @magniloquency

Target Contributors

Describe the contributor profile (background, position, organization) you would like to get contributions from.
Engineers at Financial Institutions and software and hardware vendors (AWS, IBM, Google, HP, etc.)

Project Communication Channel(s)

  • Contributor to ask maintainers which communications channels they'd like to use:
  • Asynchronous
    • GitHub Issues (public)
    • GitHub Discussions (public)
    • Mailing-list (groups.io)
    • FINOS Slack Channel (consisting of the above described contributors)
      • General (public) (supply channel name)
      • Leadership (private) (supply channel name)
  • Synchronous
    • Recurring meetings

Approval (Lead: FINOS Infra)

  • Assign issue to Executive Director (@mindthegab) to trigger voting
    (optional). If additional socialization is required, the Executive Director may bring standards projects to the FINOS Governing Board
  • FINOS accepts the contribution/new standard project (and the contribution process can move forward)

Assets transfer (optional - Lead: FINOS Infra)

  • Check GitHub repository transfer requirements:
    • finos-admin has Admin to all repositories to transfer
    • finos-admin ia allowed to transfer repositories out of the org
    • if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
  • Transfer all code assets as GitHub repositories under github.com/finos
  • Invite GitHub usernames to GitHub FINOS Org
  • Create <project-name>-maintainers GitHub team and invite users
  • Configure finos-admins and finos-staff team permissions

Infra setup (Lead: FINOS Infra)

  • Update release coordinates and code namespace to include finos (best effort)
  • Update project badge
  • Update project README
  • Aggregate mailing lists to community@finos.org

Metadata update (Lead: FINOS Infra)

  • Add logo to FINOS landscape
  • Add maintainers emails to finos-project-maintainers@finos.org list
  • Add maintainers GitHub usernames to the project-maintainers Team
  • Onboard project on LF systems (SFDC, Insights, EasyCLA, Groups.io)

Mailing list (optional)

  • Create mailing-list

Announcement (Lead: FINOS Contrib POC)

  • Work with FINOS marketing to send out announcement to announce@finos.org , checkout announcement template at the Contribution page.
  • Notify FINOS Contrib POC and FINOS marketing manager once the announcement has been sent out (FINOS infra)

Marketing collateral and Social (Lead: FINOS Marketing)

  • Update FINOS marketing collaterals to update numbers and include the new project
  • Post on FINOS social media
  • Post on LF social media
  • Email brief announcement to lf_staff@linuxfoundation.org (Optional depending applicability of contribution)

Onboarding and training (Lead: FINOS Infra)

  • FINOS Standards Project Governance
  • FINOS Standards Project Lifecycle

Press Release (OPTIONAL - Lead: FINOS Marketing)

  • Identify quotes for press release
  • Draft press release
  • Send embargoed press release to reporters

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载