Classic linear television is perfect for big live transmissions but offers little to no possibilities for interactivity or viewer-specific personalisation of content. On the other hand, streaming services allow for a high degree of personalisation, something that has made it increasingly popular.
Hybrid Multiview Broadcast by NativeWaves (NW) combines the best of both worlds: Individually selectable camera angles and soundtracks as well as personalised information on the second screen, all in perfect sync to the TV programme that is on the main screen (Fig 1).
The status quo of linear television
Today, depending on the format of a TV station, the average age of the typical TV viewer is between 45 and 65 years1. For those younger than 30, the classical, linear television experience is nearly nonexistent, used at best for watching big live events not available online or for downloading shows and clips from TV station websites2.
One of the reasons for the indifference of younger viewers is the lack of options for personalisation of content. For example: even if a major TV event is produced with more than 20 cameras, only one camera signal can be on air – possibly the one that is least interesting to the viewer at a certain moment. Especially with sports, such as football, ice hockey, basketball and motorsports, but also with concerts or theatre transmissions, even the best TV director can only deliver a subjective selection of the entire event; the rest of the footage is lost forever.
The same holds true for sound – even the most popular TV commentators can be annoying if they favour the opposing team, are hardly audible, speak the wrong language or are simply irritating.
But how can Hybrid Multiview Broadcast by NativeWaves help in these situations?
Internet technologies for a more enjoyable linear television experience
There were many attempts in the history of linear television to individualise content and to let the viewer choose their own TV experience. Starting in the late 1990s, a few TV stations have tried to offer multiple camera angles during Formula 1 broadcasts transmitted on specific TV channels via satellite (DBS, DVB-S). There was major public interest at the start but that soon turned to disappointment, because every change between channels could last up to a few seconds, thereby interrupting the programme, and becoming more of a disturbance than enjoyment.
A number of decision makers in the television industry still view the internet as a rival that only alienates viewers. But what if the worldwide web could be the best friend of linear television? What if the ‘second screen’ on a smartphone or tablet is not a distraction, but the perfect complement to the linear programme? And all with comparatively low technical and financial investment? This is where the solution from NativeWaves comes in: Hybrid Multiview Broadcast.
Mastering technical challenges
Anyone wanting to stream multiple channels of audio-visual content over the internet – in parallel and perfectly synced to a linear TV channel – needs detailed knowledge of typical TV production methods and transmission paths. Assuming an outside production that uses an OB truck and a satellite link for contribution to the station, the latency between the camera on the production site and the TV set at home will be around five to seven seconds. The latencies for DTV transmission via cable (DVB-C), satellite (DVB-S, DBS) or terrestrial (DVB-T) depend on how they are distributed, but are pretty close to each other. Use of statistical multiplexers and complex methods of distribution extend the time needed for the signal to arrive at a viewer’s home. Nevertheless, compared to typical video streaming over the internet with latencies of around 60 seconds that are seen as “normal”, these TV latencies are considered minor.
The secret: Ultra Low Latency Streaming (ULLS)
Extensive research and design at NativeWaves have enabled a technology for very fast audio and video streaming that is usable on all content delivery networks (CDN) and on the internet. NativeWaves’s ultra-low latency streaming (ULLS) is based on common but precisely managed HTTP streaming technologies. The transmission latency is a maximum of just three (3) seconds, half as long (or double as fast) as the quickest “classic” ways of TV distribution (DTV, DVB). This speed of transmissions is achieved by using very fast and industry-proven codecs for sound and high-quality video encoding (eg H.264, H.265, AAC) as well as a highly optimised end-to-end streaming workflow over the station’s CDN.
Signal flow for low latency and perfect sync
NativeWaves provides efficient, agile and versatile technologies that enable Hybrid Multiview Broadcasting.
In addition to the “normal” TV programme (PGM), the OB truck or studio produces multiple video signals (together with audio) which can be routed to a NativeWaves encoder. Multiple encoders can be used in parallel to add redundancies or increase the number of simultaneous streams. For example, one encoder in a default 4U 19-inch configuration can handle up to 16 streams. Typically, the signals are derived from the CCUs (camera control unit) or vision mixer outputs, and from the audio mixing desk, and are routed to the encoders using embedders and a switching matrix. Video sources could be cameras on racing vehicles (Formula 1, Moto GP); in goal, player and trainer cameras (football, basketball, ice hockey); or backstage and musician cameras at concerts. These signals can be augmented by live data, such as the pulse rate and running distance of athletes and other statistical data. (Typical signal flow is illustrated in Fig 2).
The NativeWaves encoder is a specially equipped computer with video input and output cards. Typically, the audio and video connections are made via 6G UHD-SDI. After processing and encoding, data is streamed into the NW cloud, where all signals are synchronised and time-aligned to each other. Depending on the configuration, different synchronisation strategies (eg via audio, video or timestamps) are used to virtually connect the additional streams to the PGM stream. The synchronisation happens automatically but manual correction is also supported. The NW cloud provides this synchronisation information of an event to the viewer.
As soon as the user opens the (previously installed) broadcaster/event app incorporating NativeWaves technology on his smartphone or tablet, the mobile device will “listen” to the TV programmme for about five seconds, then send the audio to the NW cloud. This serves as a “sync endpoint”, where all parameters for the real-time synchronisation of audio are calculated. The result is then sent back to the mobile device to synchronise all streams to the main TV programme with precise accuracy. The synchronisation is so accurate that the audio on the TV is perfectly lip-synced to the video on the second screen. All necessary modules can be easily integrated into the app using the NativeWaves SDK (software developer kit) (Fig 3).
Numerous possibilities for management and utilisation
Since the NativeWaves Solution (NWS) supports scalable multi-stream infrastructures, it allows for cost and quality optimisation of data distribution. All deployed technologies are agnostic and work with practically every CDN, a fact that allows for easy implementation of Hybrid Multiview Broadcasting into existing production and distribution facilities (Fig 4).
Another advantage is the multi-dimensional utilisation of user data. As the TV viewer interactively creates the content by selecting preferred camera angles, sounds and statistics, this data can be (anonymously) used to analyse viewing and listening behaviour and to improve the programme. Another possibility are commercials that can either be inserted into the streams the user actively watches or completely avoided if the stream is offered on a pay TV platform. Other interactive options, such as games, could be combined with it.
Possibilities in production
Although Hybrid Multiview Broadcast is easy and affordable to implement, production needs to adapt to a few new techniques and a slightly wider scope.
Ideally, the NativeWaves encoder should be in the OB truck or in the data center of a production studio, allowing for short SDI cable runs with uncompressed video. Technically speaking, distributing camera signals and sounds to the NativeWaves encoder can be handled by a routing matrix. But in practice, broadcasters always want to have control over signals, either for creative matters or for legal reasons. Stations might want to insert graphics or their logos into the single video streams, which requires a downstream keyer on the production side of things.
But what do you do when a camera stream shows the sudden horrible crash of a Formula 1 car, whose badly hurt driver can’t or shouldn’t be shown, for reasons of respect, ethics or personal rights? To deal with such contingencies, it is recommended that a “secondary director” keep tabs on all additional feeds, and is able to act in case of an emergency. “Intelligent” routing switchers or smaller video switchers with built-in keying might be an effective and affordable technical solution for such a scenario.
Sound needs to be considered as well. The loudness of the individual streams should be consistent to each other. Audio for TV should be levelled to either -23LUFS (according to EBU R 128) or -24LKFS (according to ATSC A/85), while internet streams mostly require a higher loudness of -16LUFS. A loudness processor, such as the Junger Audio TAP with 4 x 2 channels and ‘Level Magic’ can handle that precisely and automatically.
Additional audio streams allow for a number of special functions – mixes in different languages can be transmitted, as well as binaural mixes that enable immersive audio (3D audio) via headphones.
At the viewers’ home
Most viewers today own smartphones and tablets that are with them when they watch television. The app of the TV station with integrated NW-SDK that enables Hybrid Multiview Broadcast can be downloaded and installed in seconds. Using these devices as a ‘second screen’ to improve and individualise the TV viewing experience is an obvious choice.
Imagine the following scenario: A father and daughter are watching a football game via linear TV on the main screen. His tablet shows the in-goal camera of the home team with game statistics superimposed, and the audio from the TV is in English. His daughter’s smartphone has the trainer camera of the opposing football club; the audio, delivered via headphones, has French commentary and binaural, immersive sound.
Summary
The time has come for television to not only focus on more and better pixels (UHD, HDR) and new audio formats, but also to provide content that is more appealing, more interactive and more personalised – that is the only way to ensure the interest of current and future generations of TV viewers. Hybrid Multiview Broadcasting by NativeWaves is probably the best and most simple way to reach that goal. Additional video and audio streams and information are transmitted via the internet to mobile devices that act as perfectly-synced ‘second screens’. This enables viewers to enrich their TV main programme experience by selecting different camera angles, additional languages or audio mixes and augmented statistical data. Doing so enables television to adapt to the personal interest of the viewer and become an enjoyable experience… again.
Sources
- https://de.statista.com/statistik/daten/studie/183279/umfrage/durchschnittsalter-der-fernsehzuschauer-nach-sender/
- https://meedia.de/2018/01/03/tv-bilanz-2017-unter-50-jaehrige-schauen-immer-weniger-rtl-prosieben-und-sat-1-verlieren-massiv/