这是indexloc提供的服务,不要输入任何密码
Skip to content

shakuhachi - code explanation with voice. #62

@JSerZANP

Description

@JSerZANP

Motivation

Think about recording a video explaining a piece of code

  1. need screen recording
  2. need editing
  3. upload somewhere
  4. embed as video widget

The problem is Video is too heavy a tool for such use case, what we actually want is

  1. a voice companion for the code snippet
  2. a cursor to indicate where we are.

This actually could be achieved by recording two stream(voice stream, action stream) and put them together.

Design

1. recording

1. voice stream

Just use MediaStream api

https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API/Using_the_MediaStream_Recording_API

2. action stream

We only need to record the important actions, including

  1. mouse click
  2. selection
  3. Shaku-specfic events

Shaku-specific events are events that triggers visual changes on code snippets, e.g.

  1. highlight a line
  2. toggle @ fold

Events must have location information attached to specific lines & characters.

2. Edit (low priority)

It should be feasible to cut the spaces between actions and voice.
If not able to do it automatically, we can at least edit manually.

3. Generating

Voice data and the event streams are bundled together, with time info on one single timeline.

3. Rendering

Just play the voice and events along the timeline.

For fast forward, we just increase the play speed.
For rewind, we need to start from the very start.

We need controls on the UI.

  1. play
  2. pause
  3. progress bar

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions