AI-powered automated subtitling system

Chatbots and user support

Estonia

Use case ID: 017

Author: Riigikogu (The Estonian Parliament)

Date: 5 June 2024 

Objective:

Produce AI-powered automated subtitles to make live broadcasts from the session hall accessible to hearing-impaired individuals by providing a textual representation of the spoken dialogue in an efficient way. 

Actors:

  • Members of parliament (speeches of members of parliament in the session hall)
  • Video recording and broadcasting system
  • Available real-time speech-to-text tool 

Prerequisites:

  • Well-trained real-time speech-to-text system of sufficient quality
  • Willingness to give part of the human work to the AI system

Scenario:

  1. The video recording and broadcasting system produces a video and audio stream for online viewers.
  2. One audio stream is sent through a real-time speech-to-text tool.
  3. The real-time speech-to-text tool creates subtitles from the audio stream. It can detect changes of speaker, tries to reconstruct unrecognised words (using a phoneme-to-grapheme tool), uses the compound-word recognizer to construct compound words, and inserts punctuation.
  4. The real-time speech-to-text tool sends out generated subtitles for Youtube Live captions.
  5. In the selected environment, the subtitle stream must be displayed together with the main stream. 

Alternate flows:

  • An alternative is to manually create online subtitles, which is a very labour-intensive task, especially for longer online broadcasts. 

Expected results:

  • The need for human labour is reduced. 

Potential challenges:

  • This system makes more mistakes when composing the text because there is less time to process the audio. 

Integrations with other systems:

  • There is no direct need for integration, although any kind of integration makes the entire system more automated and further reduces the need for human labour. 

Success metrics:

  • The speech-to-text error rate is less than 40%.

 

The Use cases for AI in parliaments collection is published by the IPU’s Centre for Innovation in Parliament as part of the Parliamentary Data Science Hub’s project to create guidelines for AI governance in parliaments.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International licence. It may be freely shared and reused with acknowledgement of the author and the IPU. 

A use case describes how a system should work. It is used to plan, develop and measure implementation. A use case is not the same as a case study, which is a descriptive text of an actual project’s implementation. Please note that this use case is provided “as is” and neither the IPU nor the author accepts any responsibility for its use.

For more information about the IPU’s work on artificial intelligence, please visit www.ipu.org/AI or contact [email protected].