Transcribed Audio and Video Remixing

Transcribed Audio and Video Remixing

Hello all!

Here at The Hyperaudio Project we’ve been working on a concept for remixing transcribed audio (and video) for about 8 years now. In that time we’ve built a working prototype (Hyperaud.io) and branched off to create commercial tools (Trint) and work with organisations such as Studs Terkel Radio Archive and Al Jazeera, honing the technology as we went. We’ve even trialled Hyperaudio in schools as a way to teach media literacy, with some initially promising results.

We’re now ready to take these learnings and create the next version of Hyperaudio. We’re currently not considering hosting the content ourselves, preferring to embed YouTube videos or audio from SoundCloud or various media hosted at the Internet Archive and other embeddable media platforms.

So why are transcripts so important to us? In a word – accessibility. We’ve found that when you attach a transcript to a piece of media you instantly open it up for easier search, navigation, sharing and remixing. The key is to capture word-timings and link them to the media to create interactive transcripts which we are now seeing adopted and embraced by startups such as Descript.

Our focus is on encouraging people to not only add content but also to remix it. Using tools like the Hyperaudio Pad this becomes as easy as moving transcript text from one window to another, we then take the corresponding start and stop times of that text to create a playlist of snippets. Sprinkle a few effects here and there and even a bit of background music and you have a relatively professional looking remix.

There are a number of forms remixes can take – from video essays to podcast digests and we’d be looking to reward the creators of the content going into remixes as well as the remixers themselves. Since we have the duration of each excerpt or snippet making up the remix we can reward everybody proportionally. Here Probabilistic Revenue Sharing looks like the way to go.

There’s still a lot to work out, but the nice thing about Hyperaudio is that from every snippet you can jump directly to the piece it was snipped from, giving you instant context and an ability to consume more of the content you’re interested in.

Sharing of specific parts of audio and video also becomes easier when you can highlight a piece of text and click share – from there we could post the link back with the text to social media and even a small clip. In theory we can track who shared the content from the URL and reward both content creator and sharer – something that others might be thinking about doing perhaps?

I’m curious to hear what people think of these ideas and if others are thinking of doing something similar – or if there would be potential collaboration opportunities with podcasting or other audio/video platforms.

Mark

4 Likes

Nice to see you here Mark! I have been/am a big fan of Hyperaudio!

1 Like

Thanks Chris! It’s certainly been an interesting journey!

As far as the licensing of content is concerned we plan to offer the full range of Creative Common licenses – where attribution is required we can automatically add that to any remixes in the form of credits.

3 Likes

That is very cool! We are definitely interested in how these elements like WM, CC licenses, remix trails play together and start to create new ways creators/consumers relate.

1 Like