GitDocumentDB

Offline-first DocumentDB that Syncs with Git

Use GitDocumentDB to ...

📗 Store JSON documents into Git repository.

🎨 Manage Git repository by document database API.

🚀 Synchronize, diff and patch automatically with a remote repository.

     (No need to resolve conflicts manually.)

🔄 CI/CD through GitHub.

🐪 Travel history of database.

You do not need knowledge of Git to start. However, you make the most of GitDocumentDB if you understand Git.

API

Overview

https://gitddb.com/docs/api/API_overview

Specifications

https://github.com/sosuisen/git-documentdb/blob/doc-v1.0/docs-api/index.md

Usage

Getting started

Prerequisite

Node.js 10 or later

Installation

npm i git-documentdb

NOTE:
GitDocumentDB uses native addon (libgit2).
If you receive errors about installation, you probably miss building tools and libraries.
In Ubuntu 18:

sudo apt update
sudo apt install build-essential libssl-dev libkrb5-dev libc++-dev 

In Windows 10:
The list below shows typical environments.

• Node.js 12, Python 2.7.x, and Visual Studio 2017 Community (with Desktop development with C++).
• npm config set msvs_version 2017

If you are still encountering install problems, documents about NodeGit and Building NodeGit from source may also help you.

Import

import { GitDocumentDB } from 'git-documentdb';

const gitDDB = new GitDocumentDB({
  db_name: 'db01', // Git working directory
});

Basic CRUD

  // Open a repository at /your/path/to/the/example/git-documentdb/db01/.git
  const result = await gitDDB.open(); 
  // Create and open the repository if not exits.
  if (!result.ok) await gitDDB.createDB(); 

  // Create
  await gitDDB.put({ _id: 'nara', flower: 'cherry blossoms', season: 'spring' }); 
  // Git adds 'nara.json' under the working directory and commits it.

  console.log(`$ gitDDB.put({ _id: 'nara' ... }) # Create`);
  console.log(await gitDDB.get('nara')); // { _id: 'nara', flower: 'cherry blossoms', season: 'spring' }
  
  // Update document if exists.
  console.log(`\n$ gitDDB.put({ _id: 'nara' ... }) # Update`);
  await gitDDB.put({ _id: 'nara', flower: 'double cherry blossoms', season: 'spring' });
  // Git adds an updated file and commits it.

  // Read
  const doc = await gitDDB.get('nara');
  console.log(doc);
  // { flower: 'double cherry blossoms', season: 'spring', _id: 'nara' }

  // Delete
  await gitDDB.delete('nara');
  // Git deletes a file and commits it.
  
  console.log(`\n$ gitDDB.delete('nara') # Delete`);
  console.log(await gitDDB.get('nara')); // undefined

Revisions

  // get(id, 2) returns two revisions before
  const oldDoc = await gitDDB.get('nara', 2); 
  console.log(oldDoc);
  // { _id: 'nara', flower: 'cherry blossoms', season: 'spring' }

Synchronization

  // Please enter your GitHub account name.
  const github_repository = 'https://github.com/enter_your_accunt_name/git-documentdb-example.git';
  // See https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token
  const your_github_personal_access_token = 'Enter your personal access token with checked [repo]';
  await gitDDB.sync({
    live: true,
    remote_url: github_repository,
    connection: { type: 'github', personal_access_token: your_github_personal_access_token },
  });

(You can find more examples in examples/src/sync.ts)

Prefix search

 /**
    Create documents under sub-directories

    git-documentdb
    └── db01
        ├── nara
        │   ├── nara_park.json
        │   └── tsukigase.json
        └── yoshino
            └── mt_yoshino.json

  */
  // Put documents by using filepath.
  await gitDDB.put({ _id: 'nara/nara_park', flower: 'double cherry blossoms' });
  await gitDDB.put({ _id: 'nara/tsukigase', flower: 'Japanese apricot' });
  await gitDDB.put({ _id: 'yoshino/mt_yoshino', flower: 'cherry blossoms' });

  // Read
  const flowerInYoshino = await gitDDB.get('yoshino/mt_yoshino');
  console.log(flowerInYoshino);
  // { flower: 'cherry blossoms', _id: 'yoshino/mt_yoshino' }

  // Prefix search
  
  // Read all the documents whose IDs start with the prefix.
  const flowersInNara = await gitDDB.allDocs({ prefix: 'nara/' });
  console.dir(flowersInNara, { depth: 3 });
  /* flowersInNara = 
  {
    total_rows: 2,
    commit_sha: 'xxxxx_commit_sha_of_your_head_commit_xxxxx',
    rows: [
      {
        id: 'nara/nara_park',
        file_sha: '7448ca2f7f79d6bb585421c6c29446acb97e4a8c',
        doc: { flower: 'double cherry blossoms', _id: 'nara/nara_park' }
      },
      {
        id: 'nara/tsukigase',
        file_sha: '1241d69c4e9cd7a27f592affce94ec60d3b2207c',
        doc: { flower: 'Japanese apricot', _id: 'nara/tsukigase' }
      }
    ]
  }
  */
 
  // destroy() closes DB and removes
  // both the Git repository and the working directory if they exist.
  await gitDDB.destroy();

Collections

  // Try it again by another way.
  await gitDDB.create();
  // Use collections to make it easier
  const nara = gitDDB.collection('nara');
  const yoshino = gitDDB.collection('yoshino');
  await nara.put({ _id: 'nara_park', flower: 'double cherry blossoms' });
  await nara.put({ _id: 'tsukigase', flower: 'Japanese apricot' });
  await yoshino.put({ _id: 'mt_yoshino', flower: 'cherry blossoms' });

  // Read
  const flowerInYoshinoCollection = await yoshino.get('mt_yoshino');
  console.log(flowerInYoshinoCollection);
  // { flower: 'cherry blossoms', _id: 'mt_yoshino' }

  // Read all the documents in nara collection
  const flowersInNaraCollection = await nara.allDocs();
  console.dir(flowersInNaraCollection, { depth: 3 });
  /* flowersInNaraCollection = 
  {
    total_rows: 2,
    commit_sha: 'xxxxx_commit_sha_of_your_head_commit_xxxxx',
    rows: [
      {
        id: 'nara_park',
        file_sha: '7448ca2f7f79d6bb585421c6c29446acb97e4a8c',
        doc: { flower: 'double cherry blossoms', _id: 'nara_park' }
      },
      {
        id: 'tsukigase',
        file_sha: '1241d69c4e9cd7a27f592affce94ec60d3b2207c',
        doc: { flower: 'Japanese apricot', _id: 'tsukigase' }
      }
    ]
  }
  */
  await gitDDB.close();

(You can find more examples in examples/src/collection.ts)

Examples:

See examples directory.

$ cd examples
$ npm i
$ npm start
$ npm run sync
$ npm run collection

Continuous Deployment (CD) using GitDocumentDB

https://github.com/sosuisen/sosuisen-my-inventory-gatsby

App using GitDocumentDB

https://github.com/sosuisen/inventory-manager

Roadmap

• v0.1 Basic CRUD 🐾

• v0.2 Group and Search 🐾

  • Collections 🐾
  • Prefix search 🐾

• v0.3 Synchronization 🐾

  • Synchronization with GitHub 🐾
  • Revisions 🐾
  • Automated conflict resolution 🐾
  • Automated JSON diff and patch 🐾
  • Automated combining of inconsistent repositories 🐕(Here now)

• v0.4 Work on both Node.js and browser

  • Connect with SSH key pair
  • Connect to GitHub with OAuth

• until v1.0

  • Operate other data types
  • Replication
  • Grep
  • Transaction (bulk)
  • Tag (Redo/Undo)
  • Indexed Search
  • GitLab and Bitbucket
  • Push server
  • Migration
  • Plugins

(https://github.com/sosuisen/git-documentdb/projects/2)