Video#

Tip

This feature requires av dependency. You can install it via pip install "docarray[full]".

Load video data#

from docarray import Document

d = Document(uri='toy.mp4')
d.load_uri_to_video_tensor()

print(d.tensor.shape)
(250, 176, 320, 3)

For video data, .tensor is a 4-dim array, where the first dimension represents the frame id, or time. The last three dimensions represent the same thing as in image data. Here we got our d.tensor.shape=(250, 176, 320, 3), which means this video is sized in 176x320 and contains 250 frames. Based on the overall length of the video (10s), we can infer the framerate is around 250/10=25fps.

We can put each frame into a sub-Document in .chunks as use image sprite to visualize them.

for b in d.tensor:
    d.chunks.append(Document(tensor=b))

d.chunks.plot_image_sprites('mov.png')
../../_images/mov_bbb.png

Key frame extraction#

From the sprite image one can observe our example video is quite redundant. Let’s extract the key frames from this video and see:

from docarray import Document

d = Document(uri='toy.mp4')
d.load_uri_to_video_tensor(only_keyframes=True)
print(d.tensor.shape)
(2, 176, 320, 3)

Looks like we only have two key frames, let’s dump them into images and see what do they look like.

for idx, c in enumerate(d.tensor):
    Document(tensor=c).save_image_tensor_to_file(f'chunk-{idx}.png')
../../_images/chunk-0.png
../../_images/chunk-1.png

Makes sense, right?

Save as video file#

One can also save a Document .tensor as a video file. In this example, we load our .mp4 video and store it into a 60fps video.

from docarray import Document

d = (
    Document(uri='toy.mp4')
    .load_uri_to_video_tensor()
    .save_video_tensor_to_file('60fps.mp4', 60)
)

Create Document from webcam#

One can generate a stream of Documents from a webcam via generator_from_webcam():

from docarray import Document

for d in Document.generator_from_webcam():
    pass

This will create a generator that yields a Document for each frame. One can control the framerate via fps parameter. Note that the upper bound of the framerate is determined by the hardware of webcam, not the software. Press Esc to exit.