Sending Video from a Public Stream to an Amazon Kinesis Video Stream Using GStreamer
Last updated: September 12, 2019
The Amazon Kinesis Video Streams service makes it easy to securely stream video from different devices to the AWS Cloud for real-time video processing, machine learning (ML), analytics, playback, and storage.
This blog post describes how to build a Docker image containing the GStreamer multimedia framework and the Amazon Kinesis Video GStreamer plugin (kvssink
). You launch a GStreamer pipeline in a running Docker container to send video from a public stream, for example, from a Periscope live stream to the Amazon Kinesis Video Streams service.
- To get further information about the Amazon Kinesis Video Streams service and learn how to stream video from a PC webcam or a media file to an Amazon Kinesis Video stream using the GStreamer multimedia framework, see Streaming Video from a PC Webcam to an Amazon Kinesis Video Stream Using GStreamer.
How to Send Video from a Public Stream to a Kinesis Video Stream Using GStreamer
The following diagram provides a general architectural overview of the solution that you implement using this procedure.
To send video from a public stream to an Amazon Kinesis Video stream using the GStreamer multimedia framework, perform the following steps:
- Build a Docker image containing GStreamer and the Amazon Kinesis Video GStreamer plugin. You can build and use this image on your PC or in the cloud, on an EC2 instance.
See Appendix A, How to Build a Docker Image Containing GStreamer and the Amazon Kinesis Video GStreamer Plugin. - Open the Kinesis Video Streams Console at https://console.aws.amazon.com/kinesisvideo and switch to the AWS region where you want to create a Kinesis Video stream. Then, create a new stream with the name
example-video-stream
and default settings. - If you want to run the Docker image on your PC, create a new IAM user.
If you want to run the Docker image on an EC2 instance, create a new IAM role and attach this role to your EC2 instance.
In both cases, use the following policy:{ "Version": "2012-10-17", "Statement": [ { "Action": [ "kinesisvideo:DescribeStream", "kinesisvideo:GetDataEndpoint", "kinesisvideo:PutMedia" ], "Resource": "arn:aws:kinesisvideo:*:*:stream/example-video-stream*", "Effect": "Allow" } ] }
- Run a new Docker container and get shell access inside this container.
docker run -it --rm --network='host' \ kinesis-video-producer-sdk-cpp-amazon-linux /bin/bash
- Do not close this shell prompt. In the following steps you use it for executing commands within the running container.
- Set up the credentials file that is used by the Kinesis Video Streams GStreamer plugin (
kvssink
).- If you run the Docker image on your PC, in the running Docker container create a new file
/root/credentials.txt
and put in this file the<AWS_ACCESS_KEY_ID>
and<AWS_SECRET_ACCESS_KEY>
of the IAM user that you created in step 3.
The file must contain only one following line:CREDENTIALS <AWS_ACCESS_KEY_ID> <AWS_SECRET_ACCESS_KEY>
- If you run the Docker image on an EC2 instance, in the running Docker container execute the following command to create the
credentials.txt
file containing the credentials of the IAM role of your EC2 instance.python << EOF > /root/credentials.txt from __future__ import print_function import urllib2 import json metadata_url = 'http://169.254.169.254/latest/meta-data/iam/security-credentials/' iam_role = urllib2.urlopen(metadata_url).read() credentials = json.load(urllib2.urlopen(metadata_url + iam_role)) print('CREDENTIALS', credentials['AccessKeyId'], credentials['Expiration'], credentials['SecretAccessKey'], credentials['Token']) EOF
- If you run the Docker image on your PC, in the running Docker container create a new file
- Send video from a public stream to your Amazon Kinesis Video stream.
In the following commands replace<AWS Region>
with the region name where you created your Amazon Kinesis Video stream (for example,us-east-1
).- If you want to send video from a HTTP (HLS) live stream, launch the following GStreamer pipeline in the running Docker container:
gst-launch-1.0 souphttpsrc location='<https://Stream URL>' ! \ hlsdemux ! decodebin ! \ x264enc bframes=0 key-int-max=15 bitrate=300 ! \ capsfilter caps='video/x-h264,stream-format=avc,alignment=au' ! \ kvssink stream-name='example-video-stream' \ storage-size=32 credential-path=/root/credentials.txt \ aws-region='<AWS Region>'
- Refer to Appendix B, How to Retrieve the URL of a Periscope Live Stream if you want to send video from a Periscope live stream.
- If you want to send video from a RTSP live stream which is already H.264 compressed, launch the following GStreamer pipeline:
gst-launch-1.0 rtspsrc location='<rtsp://Stream URL>' ! \ rtph264depay ! \ capsfilter caps='video/x-h264,stream-format=avc,alignment=au' ! \ kvssink stream-name='example-video-stream' \ storage-size=32 credential-path=/root/credentials.txt \ aws-region='<AWS Region>'
- If you want to send video from a HTTP (HLS) live stream, launch the following GStreamer pipeline in the running Docker container:
Appendix A, How to Build a Docker Image Containing GStreamer and the Amazon Kinesis Video GStreamer Plugin
To build a Docker image containing the GStreamer multimedia framework, the Amazon Kinesis Video GStreamer plugin (kvssink
), and the souphttpsrc
GStreamer element that is required to read media data from HTTP (HLS) video streams, perform the following steps:
- Clone the repository of the Kinesis Video Streams Producer Library for C++. Then, switch to the latest commit that existed at the time of the last update of this procedure.
git clone https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp; cd amazon-kinesis-video-streams-producer-sdk-cpp; git checkout $(git rev-list master -n 1 --first-parent --before='2019-09-12');
- Go to the directory that contains the
Dockerfile
with build instructions. Then, patch the originalDockerfile
to build additionally thesouphttpsrc
GStreamer element.cd docker_native_scripts/amazonlinux-docker
patch -p0 Dockerfile << EOF @@ -21,0 +22,2 @@ + yum install -y libsoup-devel && \\ + yum install -y gnutls-devel && \\ @@ -42,0 +45,2 @@ +RUN sed -i '/^export PKG_CONFIG_LIBDIR=/a export PKG_CONFIG_LIBDIR="\\\$PKG_CONFIG_LIBDIR:\/usr\/lib64\/pkgconfig"' ./install-script +RUN sed -i 's/^max_parallel=2/max_parallel=\\\$(grep -c "^processor" \/proc\/cpuinfo)/' ./install-script @@ -48,0 +53 @@ +ENV GIO_MODULE_DIR=/usr/lib64/gio/modules EOF
- Build a new Docker image.
docker build -t kinesis-video-producer-sdk-cpp-amazon-linux .
- Build time on a
c5.xlarge
EC2 instance that has 4 vCPUs is approximately15 minutes
.
- Build time on a
Appendix B, How to Retrieve the URL of a Periscope Live Stream
Each Periscope.tv
stream has a unique Stream ID. You can find it in the address bar of your browser when you open a stream.
For example, in the following URL, the Stream ID is 1P3KQLuedBixE
.
https://www.periscope.tv/w/1P3KQLuedBixE?channel=...
To retrieve the URL of a Periscope stream, you need to send a request containing the Stream ID to the Periscope API endpoint and extract the value of hls_url
. For example, for the stream 1P3KQLuedBixE
, execute the following command:
curl -s 'https://api.periscope.tv/api/v2/accessVideoPublic?broadcast_id=1P3KQLuedBixE' | \
python -c 'import sys, json; print json.load(sys.stdin)["hls_url"]'