How does Auk turn a horizontal video into a vertical clip?

Updated

Auk crops each clip from your wide video to a vertical 9:16 frame and moves the crop to follow whoever is talking. It decides shot by shot: when a shot has nobody to follow, or several people share it, it uses a different layout instead of a single crop.

The editor on a two-person clip: the preview shows a Split layout with one speaker above the other, and the timeline's layout lane shows a Single segment and then a Split one

Footage: The B2B Marketing Strategy That Grew His Agency to $5M ARR by Success Clues Podcast, CC BY 4.0, cropped and captioned.

How does Auk know who is talking?

It finds the faces in each shot and matches them against the audio to tell who is speaking at each moment. The crop follows that person smoothly, and starts fresh at each cut instead of sliding across it.

What are the layouts?

Each clip is split into segments, and each segment has one layout, shown as a labelled chip on the editor timeline.

LayoutWhat it showsPicked by Auk today?
SingleA vertical crop that follows one personYes
FitThe whole wide frame, with the space above and below filledYes
SplitTwo people, one above the otherYes
ThreeOne wide box over two narrower onesYes
FourFour people in a 2x2 gridYes
FacecamOne person in a small band over the rest of the frameYes, for a webcam over a screen

A floating picture-in-picture box can sit over any of them. Auk adds one itself for a round webcam, or one too small for a band; you can add your own in the editor.

When does Auk use Fit instead of a crop?

When a shot has no face to follow, such as a slide, a screen recording or b-roll. A narrow crop would cut away what matters, so Auk shows the whole frame.

What does Auk do with a screen recording, slides or a game?

When a shot shows a screen with a webcam over it, as in a stream, a tutorial or a talk with slides, Auk uses Facecam: the person in a small band at the top, the screen below. The screen part follows where the picture changes, so the text or action being talked about stays in view. A round webcam, or one with a very small face, floats as a round picture-in-picture over the screen instead, kept away from the bottom right corner where TikTok, Reels and Shorts put their buttons.

A vertical video between two blurred bars, such as a phone game recorded in landscape, is cropped to the vertical part.

When does Auk use Split, Three or Four?

When two, three or four people of similar size share a shot for most of it. Each person gets a box; someone much smaller in the background does not count.

Two people sitting shoulder to shoulder share one box instead, because cropping them apart would cut one picture in two or leave each face filling its box. So three people with such a pair get a Split: the pair together in one band, the third person in the other, and whoever is speaking on top.

What if Auk frames the wrong person?

Change the layout, choose who fills each box, or drag the crop in the editor: see Adjusting the crop.

Related