Sketch

No need for those 4 create buttons(seems unnecessary), we can display empty notes/flashcards like the mock(but not created on db yet). A button inside the editor to autopopulate with AI is better, when they add content, update db. And a floating button to chat with AI, that doesn’t effect notes/flashcards, but each response can have buttons to pull to the flashcard or note,

Missions

Abhi

Ashwin

• OpenAI embeddings for embedded videos (createContentW…)

• For Creat Notes/Flashcards from Content

  • CreateFlashcardSetFromContentInput/CreateFlashcardSetInputFromNote (need graphql input similar to createFlashcardSetFromNoteInput)

  • Add contentId to updateNoteInput, updateFlashcardSetInput and create schemas too

• Allow chapters to be created from createContentWithTranscription

• Pulling chapters from YT

Not urgent

• dynamodbTrigger lambda for contentTable to get rid of the s3 items

Raas

• Extension UI

  • Login

  • Video

  • tabswitcher

• Delete Media From Home(maybe other functionalities too -> list public media on profile)

  • Trash content

  • Restore

  • Permanently delete

• New Editors with embed ai

• Chat Section for Media Page

• Create AI notes/flashcards (set up flow)

• Chapters in transcription box

• Create aux function to create AI notes/flashcards using the next js server side (ai/knowtai API)

  • Notes one is already done, just needs to be hooked up

Karim

• Extension UI

  • Flashcards Tab

  • PDF Tab

  • Article

Overall Remaining Work

Video Flow Remaining work

• Custom controls

• Transcription generating indicator

• Audio Player

• Embed Transcription for other iframes

• Upload cancellation

• Audio player slider disappearing on larger screen after adding chapters

• Being able to click on transcript and scroll to part of the video

• Spritesheet prev over captions

• Clicking playback controls shouldn't pause the video

• Uplaod audio to s3 with right extension + update dynamodb too

• Deepgram speed

• Ffmpeg timing out early? , only 1-2 frame , remaining empty (http://localhost:3000/content/8e62bf5f-9ee2-4a71-8f1a-66726b6693c1 , http://localhost:3000/content/abbfd493-e2e9-4127-a46a-d60639be9da9 ) I think it’s something else, cos, for some videos, even if it’s longer, getting result properly!

• Support mkv,mov,webm,mp3,wav

• Audio with type “VIDEO” on db (was on front-end, sorry:))

• Change deepgram model to nova

  • Maybe pick language beforehand? If it has an effect on the cost

• Create embeddings -> 1 embedding per paragraph in the transcript

• Set up an API to find relevant information based on a question input. This should do semantic search on the embeddings vectors, and return a subset of the relevant information. Then, we can pass that info to the openai on the next js server.

• Get subtitles to stream text

• See if converting mp4 to mp3 before sending to deepgram speeds up processing

• Backend - Update graphql scheme for front-end, can’t call transcript

• List content on home page

• Supported file types alert based on browsers when playing

• Content card home page-> basic functionalities like other cards

• Next Big Step -> OpenAI