-
Notifications
You must be signed in to change notification settings - Fork 32
Description
1. Describing The Contribution
Business Problem
Describe the business problem the contribution solves
Large financial organizations have a collection of distributed compute resources - physical severs, virtual instances, cloud compute (AWS, GCP, ....) and grids (e.g. IBM Symphony). The second problem is that capacity on these grids is often statically allocated and and workloads cannot dynamically resource up and down. There are multiple challenges around utilizing different compute resources efficiently which means grids often runs at low utilization (30% or below) and also worsen climate impact.
Proposed Solution
Describe the type of contribution (project or working group) and how it solves that business problem
openGRIS is an open Standard for Grid Resource Scheduling with client and worker standards to tie resources together and share them in a cost-effective and climate friendly way. openGRIS aims to provide a standard to allow different hardware and compute pools to be effectively addressed by applications across the organization - production batch processes, on-demand compute from notebooks, etc. The goal is on one side to easily unlock "elastic-compute" on demand and on the other hand enable pools of compute to be more effectively utilized. openGRIS draws inspiration from services AWS Lambda and BigQuery and software tools like Dask and Spark to propose a cloud provider and hardware independent standard.
Scope
Clearly define the scope of the standard. In cases where the scope is expected to be defined by the standard project participants this should be one of the first tasks the group completes. Guidelines can be found in the Community Specification documentation.
openGRIS will specify a definition for compute payloads for routing to compute backends. This definition will include elements which have come up in discussion with FinOS members (Intel, Nvidia, IBM and AWS) and can include directives for GPU and wide Vector instructions.
openGRIS will also specify policies which can define how those payloads are routed to various compute backends. These policies can have time and cost preferences. Finally, the definition of backends will be left out of the scope of the initial standard but the standard can be later expanded to definition and categorization of compute backends.
Tentative Roadmap and Current State
Describe the short and medium term goals and phases of the project. What does success look like for this project?
Category | Milestone | Target Date | Status | Comments |
---|---|---|---|---|
Proposal | Standard 1.0 | 6/30/24 | Done | |
Implementation | Reference Implementation | 7/30/24 | Done | |
Implementation | Open Source Scaler | 9/30/24 | Done | github.com/citi/scaler |
Proposal | 1.0 Discussion with Nvidia, AWS, Intel, IBM | 9/30/24 | Done | |
Proposal | Formalize Standard 2.0 | 2/15/25 | Done | Standard 2.0 |
Implementation | Interface to IBM Symphony | 1/30/25 | Done | Done |
Implementation | EC2 Static Compute Adapter | 4/30/25 | Pending | |
Implementation | Multi-language bindings: C++ | 6/30/25 | Pending | |
Implementation | Time based Policies for Scheduling | 9/30/25 | Pending | |
Implementation | Dynamic Scaling for Symphony and EC2 | 10/30/25 | Pending | |
Implementation | Multi-Environment support via Docker/Podman | 11/30/25 | Pending | |
Implementation | AWS HTC Grid Adapter | TBD | Pending |
|
See detailed architecture and roadmap here.
Existing Materials
If materials already exist, provide a link to them that Foundation staff can access - if it's in a private GitHub.com repositories, you should invite the finos-admin user with R/O permissions to those repositories
- GitHub / GitLab Repository
- Project Name (enter here) OpenGRIS
- Is Continuous Integration used? If so, which system is used?
- Was the project ever released? (yes / no)
- If so, are releases public? (yes / no)
- And what's the latest released version?
- Existing Project Documentation ( URL / microsite / PDF etc detail here) no
- Does the name have a registered trademark? (yes / no) no
- Is there a logo? (yes / no) no
- High-Level Presentation prepared for Technical Oversight Committee (~15 mins) yes
- Are meetings currently held for the project? (yes / no + details) no
- Are meeting minutes, agenda and attendance tracked? (yes / no + details) no
Development Team
Maintainers
Who will be the project maintainer(s)? Provide full name, affiliation, work email address, and GitHub / GitLab username.
Name | Affiliation | Work Email Address | Github / GitLab username |
---|---|---|---|
Zhuo Yin | Citi | zhuo.yin@citi.com | @sharpener6 |
Confirmed contributors
If applicable, list all of the individuals that have expressed interest in and/or are committed to contributing to this project, including full name, affiliation, work email address, and GitHub.com username
Name | Affiliation | Work Email Address | Github / GitLab username |
---|---|---|---|
Raphael Javaux | Citi | raphael.javaux@citi.com | @rafa-be |
George Lewis | Citi | george.lewis@citi.com | @magniloquency |
Target Contributors
Describe the contributor profile (background, position, organization) you would like to get contributions from.
Engineers at Financial Institutions and software and hardware vendors (AWS, IBM, Google, HP, etc.)
Project Communication Channel(s)
- Contributor to ask maintainers which communications channels they'd like to use:
- Asynchronous
- GitHub Issues (public)
- GitHub Discussions (public)
- Mailing-list (groups.io)
- FINOS Slack Channel (consisting of the above described contributors)
- General (public) (supply channel name)
- Leadership (private) (supply channel name)
- Synchronous
- Recurring meetings
Approval (Lead: FINOS Infra)
- Assign issue to Executive Director (@mindthegab) to trigger voting
(optional). If additional socialization is required, the Executive Director may bring standards projects to the FINOS Governing Board - FINOS accepts the contribution/new standard project (and the contribution process can move forward)
Assets transfer (optional - Lead: FINOS Infra)
- Check GitHub repository transfer requirements:
- finos-admin has
Admin
to all repositories to transfer - finos-admin ia allowed to transfer repositories out of the org
- if the repository is owned by a user (and not an org), the user must be able to transfer the repository to finos-admin
- finos-admin has
- Transfer all code assets as GitHub repositories under github.com/finos
- Invite GitHub usernames to GitHub FINOS Org
- Create
<project-name>-maintainers
GitHub team and invite users - Configure
finos-admins
andfinos-staff
team permissions
Infra setup (Lead: FINOS Infra)
- Update release coordinates and code namespace to include
finos
(best effort) - Update project badge
- Update project README
- Aggregate mailing lists to community@finos.org
Metadata update (Lead: FINOS Infra)
- Add logo to FINOS landscape
- Add maintainers emails to finos-project-maintainers@finos.org list
- Add maintainers GitHub usernames to the project-maintainers Team
- Onboard project on LF systems (SFDC, Insights, EasyCLA, Groups.io)
Mailing list (optional)
- Create mailing-list
Announcement (Lead: FINOS Contrib POC)
- Work with FINOS marketing to send out announcement to announce@finos.org , checkout announcement template at the Contribution page.
- Notify FINOS Contrib POC and FINOS marketing manager once the announcement has been sent out (FINOS infra)
Marketing collateral and Social (Lead: FINOS Marketing)
- Update FINOS marketing collaterals to update numbers and include the new project
- Post on FINOS social media
- Post on LF social media
- Email brief announcement to lf_staff@linuxfoundation.org (Optional depending applicability of contribution)
Onboarding and training (Lead: FINOS Infra)
- FINOS Standards Project Governance
- FINOS Standards Project Lifecycle
Press Release (OPTIONAL - Lead: FINOS Marketing)
- Identify quotes for press release
- Draft press release
- Send embargoed press release to reporters