+
Skip to content

Load data from external sources #43

@facundoolano

Description

@facundoolano

Right now data can be loaded as template variables from yml files in the data/ root dir. (It would be trivial to extend it to support other formats, eg JSON files)

I'd like to be able to similarly load data files from remote locations. This would allow for the static site generator to ingest content from other applications. The particular cases I have in mind are creating a bookmarks/linkblog section from feedi favorites, and generating a "reads" section from a reading management app (a subset of goodreads).

Some notes:

  • the content would be downloaded from a remote url set in config.yml
  • supported formats should be: json, yaml, rss, atom.
    • it would require some effort turning an arbitrary rss/atom feed into a dynamic structure available as template variables (without exposing go parser internals)
  • the logic to fetch/refresh the data should be:
    • fetch content the first time jorge serve runs, but not upon file refresh.
    • alternatively, the data could be cached so jorge serve only fetches when there isn't a local copy.
    • fail with a warning but don't stop the local server.
    • fetch each time jorge build runs, and crash on error. (i.e. don't generate a production build with stale data)
  • this would require some support for secret management:
    • support loading secret variables from env variables or from local .env files (outside source control)
    • add syntax to use the variables in url and header specs in config.yml
    • use url and header spec when fetching data (thinking of personal feed url and bearer token scenarios)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载