AVAssetExportSession with instructions make black screen

Question

I'm trying to combine 2 audio files and 1 video file into 1 .mov file. I realize it with next code:

-(void)combineData{
AVMutableComposition *mixComposition = [[AVMutableComposition alloc] init];

AVAsset *audioAsset = [AVAsset assetWithURL:_songURL];
AVAsset* audioAsset2 = [AVAsset assetWithURL:[NSURL fileURLWithPath:[NSTemporaryDirectory() stringByAppendingPathComponent:[@"speechRecord" stringByAppendingPathExtension:@"caf"]]]];
AVURLAsset *videoAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:[NSTemporaryDirectory() stringByAppendingPathComponent:[@"movie" stringByAppendingPathExtension:@"mov"]]] options:nil];

AVMutableCompositionTrack *firstTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[firstTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];

AVMutableCompositionTrack *secondTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[secondTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[audioAsset2 tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];

AVMutableCompositionTrack *videoTrack = [mixComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[videoTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, videoAsset.duration) ofTrack:[[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0] atTime:kCMTimeZero error:nil];

AVMutableVideoCompositionLayerInstruction *layerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoTrack];
/*CGAffineTransform rotationTransform = CGAffineTransformMakeRotation(DEGREES_TO_RADIANS(90));
CGAffineTransform rotateTranslate = CGAffineTransformTranslate(rotationTransform,320,0);*/
[layerInstruction setTransform:videoTrack.preferredTransform atTime:kCMTimeZero];

AVMutableVideoCompositionInstruction * mainInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
mainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, videoAsset.duration);
[mainInstruction setLayerInstructions:[NSArray arrayWithObject:layerInstruction]];

mixComposition.naturalSize = videoTrack.naturalSize;

AVMutableVideoComposition *mainCompositionInst = [AVMutableVideoComposition videoComposition];
mainCompositionInst.renderScale = 1.0;
mainCompositionInst.renderSize = videoTrack.naturalSize;
mainCompositionInst.frameDuration = CMTimeMake(1, 30);
mainCompositionInst.instructions = [NSArray arrayWithObject:mainInstruction];
//mainCompositionInst.

NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *myPathDocs =  [documentsDirectory stringByAppendingPathComponent:@"mergeVideo.mov"];

if ([[NSFileManager defaultManager] fileExistsAtPath:myPathDocs]) {
    [[NSFileManager defaultManager] removeItemAtPath:myPathDocs error:nil];
    NSLog(@"Removing old mergeVideo");
}

NSURL *url = [NSURL fileURLWithPath:myPathDocs];

AVAssetExportSession *exporter = [[AVAssetExportSession alloc] initWithAsset:mixComposition presetName:AVAssetExportPresetHighestQuality];
exporter.outputURL=url;
exporter.outputFileType = AVFileTypeQuickTimeMovie; 
exporter.videoComposition = mainCompositionInst;
exporter.shouldOptimizeForNetworkUse = YES; 

[exporter exportAsynchronouslyWithCompletionHandler:^ 
 {
     dispatch_async(dispatch_get_main_queue(), ^{
         [self exportDidFinish:exporter];
     });
 }];

My first task is deal with 90 rotation of my new video. When I'm including mainCompositionInst into my code, my new video have all sounds, but it's have a black screen and still not rotated. If I don't using any instructions it works fine. Maybe I have mistakes in my code, or maybe not. What would you advise me?

See the answer here: https://stackoverflow.com/questions/46926795/black-video-outputted-via-avmutablevideocomposition-and-caanimation/51656801#51656801 — Maximum, Aug 03 '18 at 11:27

score 1 · Answer 1 · answered May 13 '14 at 11:19

I would try applying the rotation on top of the preferredTransform because it's possible that the preferredTransform translates your video so that it displays correctly on the screen.

I would try the following:

CGAffineTransform transform = videoTrack.preferredTransform;
transform = CGAffineTransformRotate(transform, DEGREES_TO_RADIANS(90));
//transform = CGAffineTransformTranslate(transform, 320, 0);
[layerInstruction setTransform:transform atTime:kCMTimeZero];

score 1 · Answer 2 · answered Nov 05 '16 at 23:40

Proper working sample can be found here: https://github.com/robovm/apple-ios-samples/blob/master/AVSimpleEditoriOS/AVSimpleEditor/AVSERotateCommand.m#L98

The things is that you need to translate the composition to compensate the movement caused by rotation (since rotation would cause it to move out of frame) as it is stated by comments in the Apple sample source code.

The correct translation to apply will be:

t1 = CGAffineTransformMakeTranslation(assetVideoTrack.naturalSize.height, 0.0);
// Rotate transformation
t2 = CGAffineTransformRotate(t1, degreesToRadians(90.0));

AVAssetExportSession with instructions make black screen

2 Answers2