Rust library for Media over QUIC
Go to file
Wanjohi 43af19ba94
Fix for audio breaking after a certain time into the stream
After some time into the audio stream, the audio breaks and an error message is displayed in the console stating that the frame offset exceeds the total frames. 
``
caught RangeError: Failed to execute 'copyTo' on 'AudioData': Frame offset exceeds total frames (1024 >= 1024).
``
This is my proposed solution to the issue.  I have tested and verified that my fix works and the audio now runs smoothly without any break.

Hope it helps :)
2023-05-02 15:57:58 +03:00
cert Simplify the fingerprint code. 2023-04-13 13:34:34 -07:00
media Initial work on supporting renditions. 2022-11-08 13:18:35 +00:00
player Fix for audio breaking after a certain time into the stream 2023-05-02 15:57:58 +03:00
server Simplify the fingerprint code. 2023-04-13 13:34:34 -07:00
.gitignore Remove the need to use Chrome Canary. 2023-03-26 12:35:33 -07:00
LICENSE Initial public release. 2022-06-29 09:17:02 -07:00
NOTICE Initial public release. 2022-06-29 09:17:02 -07:00
README.md Generate a proper certificate for WebTransport. 2023-03-28 14:40:43 +09:00

README.md

Warp

Segmented live media delivery protocol utilizing QUIC streams. See the Warp draft.

Warp works by delivering each audio and video segment as a separate QUIC stream. These streams are assigned a priority such that old video will arrive last and can be dropped. This avoids buffering in many cases, offering the viewer a potentially better experience.

Limitations

Browser Support

This demo currently only works on Chrome for two reasons:

  1. WebTransport support.
  2. Media underflow behavior.

The ability to skip video abuses the fact that Chrome can play audio without video for up to 3 seconds (hardcoded!) when using MSE. It is possible to use something like WebCodecs instead... but that's still Chrome only at the moment.

Streaming

This demo works by reading pre-encoded media and sleeping based on media timestamps. Obviously this is not a live stream; you should plug in your own encoder or source.

The media is encoded on disk as a LL-DASH playlist. There's a crude parser and I haven't used DASH before so don't expect it to work with arbitrary inputs.

QUIC Implementation

This demo uses a fork of quic-go. There are two critical features missing upstream:

  1. WebTransport
  2. Prioritization

Congestion Control

This demo uses a single rendition. A production implementation will want to:

  1. Change the rendition bitrate to match the estimated bitrate.
  2. Switch renditions at segment boundaries based on the estimated bitrate.
  3. or both!

Also, quic-go ships with the default New Reno congestion control. Something like BBRv2 will work much better for live video as it limits RTT growth.

Setup

Requirements

  • Go
  • ffmpeg
  • openssl
  • Chrome Canary

Media

This demo simulates a live stream by reading a file from disk and sleeping based on media timestamps. Obviously you should hook this up to a real live stream to do anything useful.

Download your favorite media file:

wget http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4 -O media/source.mp4

Use ffmpeg to create a LL-DASH playlist. This creates a segment every 2s and MP4 fragment every 10ms.

./media/generate

You can increase the frag_duration (microseconds) to slightly reduce the file size in exchange for higher latency.

TLS

Unfortunately, QUIC mandates TLS and makes local development difficult.

If you have a valid certificate you can use it instead of self-signing. The go binaries take a -tls-cert and -tls-key argument. Skip the remaining steps in this section and use your hostname instead.

Otherwise, we use mkcert to install a self-signed CA:

./generate/cert

With no arguments, the server will generate self-signed cert using this root CA. This certificate is only valid for 2 weeks due to how WebTransport performs certificate fingerprinting.

Server

The Warp server supports WebTransport, pushing media over streams once a connection has been established. A more refined implementation would load content based on the WebTransport URL or some other messaging scheme.

cd server
go run main.go

This can be accessed via WebTransport on https://localhost:4443 by default.

Web Player

The web assets need to be hosted with a HTTPS server. If you're using a self-signed certificate, you may need to ignore the security warning in Chrome (Advanced -> proceed to localhost).

cd player
yarn install
yarn serve

These can be accessed on https://localhost:4444 by default.

If you use a custom domain for the Warp server, make sure to override the server URL with the url query string parameter, e.g. https://localhost:4444/?url=https://warp.demo.