Transcribed Audio and Video Remixing

Transcribed Audio and Video Remixing

Hello all!

Here at The Hyperaudio Project we’ve been working on a concept for remixing transcribed audio (and video) for about 8 years now. In that time we’ve built a working prototype (Hyperaud.io) and branched off to create commercial tools (Trint) and work with organisations such as Studs Terkel Radio Archive and Al Jazeera, honing the technology as we went. We’ve even trialled Hyperaudio in schools as a way to teach media literacy, with some initially promising results.

We’re now ready to take these learnings and create the next version of Hyperaudio. We’re currently not considering hosting the content ourselves, preferring to embed YouTube videos or audio from SoundCloud or various media hosted at the Internet Archive and other embeddable media platforms.

So why are transcripts so important to us? In a word – accessibility. We’ve found that when you attach a transcript to a piece of media you instantly open it up for easier search, navigation, sharing and remixing. The key is to capture word-timings and link them to the media to create interactive transcripts which we are now seeing adopted and embraced by startups such as Descript.

Our focus is on encouraging people to not only add content but also to remix it. Using tools like the Hyperaudio Pad this becomes as easy as moving transcript text from one window to another, we then take the corresponding start and stop times of that text to create a playlist of snippets. Sprinkle a few effects here and there and even a bit of background music and you have a relatively professional looking remix.

There are a number of forms remixes can take – from video essays to podcast digests and we’d be looking to reward the creators of the content going into remixes as well as the remixers themselves. Since we have the duration of each excerpt or snippet making up the remix we can reward everybody proportionally. Here Probabilistic Revenue Sharing looks like the way to go.

There’s still a lot to work out, but the nice thing about Hyperaudio is that from every snippet you can jump directly to the piece it was snipped from, giving you instant context and an ability to consume more of the content you’re interested in.

Sharing of specific parts of audio and video also becomes easier when you can highlight a piece of text and click share – from there we could post the link back with the text to social media and even a small clip. In theory we can track who shared the content from the URL and reward both content creator and sharer – something that others might be thinking about doing perhaps?

I’m curious to hear what people think of these ideas and if others are thinking of doing something similar – or if there would be potential collaboration opportunities with podcasting or other audio/video platforms.

Mark

7 Likes

Nice to see you here Mark! I have been/am a big fan of Hyperaudio!

1 Like

Thanks Chris! It’s certainly been an interesting journey!

As far as the licensing of content is concerned we plan to offer the full range of Creative Common licenses – where attribution is required we can automatically add that to any remixes in the form of credits.

3 Likes

That is very cool! We are definitely interested in how these elements like WM, CC licenses, remix trails play together and start to create new ways creators/consumers relate.

1 Like

Hi Mark. Really interested in what you’re doing, and it definitely caught my eye as I was browsing the forum!

I’ve recently been working with Happen Films to create 1000(ish) word articles for their new video podcast series, working from a transcription in Google docs to create something that gives access to a different audience, those with less time, etc (examples here https://happenfilms.com/blog). Not only can I see value in what you’ve developed in easing that process, but in also creating shorter video and audio versions, plus snippets for social media, etc.

But what struck me most once I realised what your system offers is the potential to create a platform / service where it’s possible for the content creators to gain the benefits of revenue generated by shorter versions using the web monetization protocol.

Like Medium but for audio, video and written content. My partner works on websites within the accessibility space, so I’m also aware of the impact providing proper transcriptions can have, and that not everyone has the time or capability to listen / watch an hour long podcast. It seems to me that so much good content is not as available or accessible as it could be.

I’d also love for the original content creators to ‘own’ the url of their content. For example, I encourage the people I work with to host the full content on their own website, and use something like Medium as a ‘feeder’ with shorter versions, or overview articles. Otherwise the value of those links that are shared across the internet is in the hand of a third party - Medium could disappear and you’ve no way to set up redirects for your content.

When it comes to remixing, there’s an interesting discussion around who owns the rights and who gets the revenue, and any ‘service’ offering to do this content creation in partnership with the originator could either be a paid service or based on a share of revenue. Lots of options to consider.

Podcasters with decent sized teams can afford to do this work themselves (though they tend not to fully utilise the capability or maximise accessibility), but many don’t have the capacity. To use a platform like yours to generate the content could shortcut the process sufficiently to make it viable.

I love the idea of a platform which acts as a ‘feeder’ for the original content, containing a range of different variants that can be accessed by a wider range of people.

Anyway, just wanted to respond and let you know of an idea sparked by your post. If you want to know more about me, you can find me on LinkedIn (https://www.linkedin.com/in/tukkrr/) or check out my website fromleftfield.co.nz. Although I’m focusing more on branding, storytelling and positioning these days, my background is content writing, hence the interest.

Cheers,
Nick.

2 Likes

Hi Nick,

Thanks for responding with this thoughtful post. I’m so glad that you can see the potential in attaching transcripts to media, from an accessibility standpoint and further – as a way to quickly put together short pieces or remixes. And yes we have high hopes that a model that rewards both original content creator, remixer/sharer can make a viable alternative to the ad-based model.

I think the idea that the original content creator owns the URL to their content is an important one. In fact we’re committed to making this easier by providing open source libraries such as Hyperaudio Lite and creating interactive transcript plugins for platforms like Wordpress (still work in progress). In fact at least initially we won’t host any audio or video preferring for people to supply a URL whether it be YouTube or self-hosted. So I hope that Hyperaudio will very much become a ‘feeder’ for original content – perhaps encouraging users to follow the 100-20 rule or similar.

Another thing we’re doing in this space is providing subdomains for an example see this prototype at chattanooga.hyperaud.io and only today I was talking to somebody about the possibility of having custom URLs. It doesn’t quite solve the problem of Hyperaudio disappearing, but it does give a sense of ownership and I see no reason why we couldn’t offer a server installable version of Hyperaudio if this is what people want – after all, it’s all open source.

I really like the look of the Happen Films content and more generally would be happy to transcribe any content you have, create an interactive transcript from it as a trial and perhaps collaborate further as we move forward.

Thanks again for the feedback – it’s always useful to see things from a content creator’s viewpoint.

Mark

2 Likes

Love the stamina of the Gig City/Hyperaudio collaboration!

1 Like

Yes – that was a great collaboration! The folk at Chattanooga Public Library were so engaged and supportive! More recently we collaborated with a school here in Italy using schools.hyperaud.io – which was a lot of fun. My favourite bit is the showcasing of their work at the end :slight_smile:

In case folk are interested, I went into more detail in this blogpost https://maboa.it/rethinking-remixing/

3 Likes

Liking the thinking, @maboa. And good to see the site itself web monetized.

1 Like

Enjoyed the article, thanks Mark. Also wanted to post a link to a potentially aligned GFTW application that’s been posted recently by Oliver Sauter of Memex, an app that allows people to curate collections of web-based research and content. Seems very aligned with the idea of creating remixes of video discussed here. Memex is proposing development of a ‘social’ aspect to their app that allows curators to share their collections, and generate revenue though web monetisation for both the curators and the original content generators.

Here’s the link to Oli’s proposal:

2 Likes

In case folk are interested - we wrote about the ethical considerations of creating the new Hyperaudio platform here https://maboa.it/hyperaudio-ethical-considerations-of-building-a-platform/

2 Likes

I was checking out bookshop.org since they are doing interesting work supporting local bookstores. Noticed that they had an affiliate program where people are essentially paid to promote books, gaining 10% of every sale. Kinda interesting and something I think we’re going to experiment with the next version of Hyperaudio. This could be an interesting model for other projects too.

1 Like