0

I'm attempting to record video and audio from an ec2 instance (Ubuntu 16.04), using xvfb and pulse audio. Everything works fine, except that the output file contains "buzzy" speech (not sure if this is what "clipping" is?). This happens using either aac or mp3 audio codecs.

Sample audio file: http://s000.tinyupload.com/index.php?file_id=86237391396073526211

ffmpeg command:

ffmpeg \
-f x11grab -framerate 30 -video_size ${SCREEN_WIDTH}x${SCREEN_HEIGHT} -i :1.0 \
-f pulse -i default \
-c:v libx264 -preset veryfast -maxrate 2500k -bufsize 2500k -vf "format=yuv420p" -g 60 \
-c:a aac -strict -2 -channel_layout stereo  -ab 256k -ar 48000 -bufsize 512k -ac 2  \
-flags +global_header -f flv "test.mp4"

The output from ffmpeg when I run the command:

Input #0, x11grab, from ':1.0':
  Duration: N/A, start: 1497567394.762052, bitrate: N/A
    Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1360x764, 30 fps, 30 tbr, 1000k tbn, 30 tbc
Guessed Channel Layout for  Input Stream #1.0 : stereo
Input #1, pulse, from 'default':
  Duration: N/A, start: 1497567396.743667, bitrate: 1536 kb/s
    Stream #1:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
[libx264 @ 0x1977d20] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x1977d20] profile High, level 3.2
[libx264 @ 0x1977d20] 264 - core 148 r2643 5c65704 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=60 keyint_min=6 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=2500 vbv_bufsize=512 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'test.mp4':
  Metadata:
    encoder         : Lavf56.40.101
    Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 1360x764, q=-1--1, max. 2500 kb/s, 30 fps, 1k tbn, 30 tbc
    Metadata:
      encoder         : Lavc56.60.100 libx264
    Stream #0:1: Audio: aac ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 256 kb/s
    Metadata:
      encoder         : Lavc56.60.100 aac
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))

Based on various posts, I thought it might be something in the pulse config, particularly the "resample-method" setting, but that doesn't appear to do anything. Here is how the /etc/pulse/daemon.conf config file currently stands:

; daemonize = no
; fail = yes
; allow-module-loading = yes
; allow-exit = yes
; use-pid-file = yes
; system-instance = no
; local-server-type = user
; enable-shm = yes
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB
; lock-memory = no
; cpu-limit = no

; high-priority = yes
; nice-level = -11

; realtime-scheduling = yes
; realtime-priority = 5

; exit-idle-time = 20
; scache-idle-time = 20

; dl-search-path = (depends on architecture)

; load-default-script-file = yes
; default-script-file = /etc/pulse/default.pa

; log-target = auto
; log-level = notice
; log-meta = no
; log-time = no
; log-backtrace = 0

; resample-method = speex-float-1
; resample-method = speex-float-10
resample-method = speex-fixed-10
; enable-remixing = yes
enable-lfe-remixing = yes
; lfe-crossover-freq = 120

flat-volumes = no

; rlimit-fsize = -1
; rlimit-data = -1
; rlimit-stack = -1
; rlimit-core = -1
; rlimit-as = -1
; rlimit-rss = -1
; rlimit-nproc = -1
; rlimit-nofile = 256
; rlimit-memlock = -1
; rlimit-locks = -1
; rlimit-sigpending = -1
; rlimit-msgqueue = -1
; rlimit-nice = 31
; rlimit-rtprio = 9
; rlimit-rttime = 200000

; default-sample-format = s16le
default-sample-format = s32le
default-sample-rate = 44100
; default-sample-rate = 48000
; alternate-sample-rate = 48000
alternate-sample-rate = 44100
; default-sample-channels = 2
; default-channel-map = front-left,front-right

; default-fragments = 4
; default-fragment-size-msec = 25
; default-fragments = 4
; default-fragment-size-msec = 25


; enable-deferred-volume = yes
deferred-volume-safety-margin-usec = 1
; deferred-volume-extra-delay-usec = 0

Edit:

Ok, updated the ffmpeg command to capture only the audio, but still have the same issue:

ffmpeg \
-f pulse -i default \
-c:a aac -strict -2 -channel_layout stereo  -ab 256k -ar 48000 -bufsize 512k -ac 2  \
test.aac

I do get some interesting messages in the output about timestamps being off:

[aac @ 0x12a4900] Queue input is backward in timeits/s
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 175924
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 176948
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 177972
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 178996
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 180020
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 181044
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 182857 >= 182068
[aac @ 0x12a4900] Queue input is backward in time
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 183406
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 184430
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 185454
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 186478
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 187502
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 188526
[aac @ 0x12a4900] Queue input is backward in time
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 189550
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 190574
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 169622
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 170646
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 171670
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 172694
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 173718
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 174742
[aac @ 0x12a4900] Queue input is backward in time
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 175766
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 176790
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 157020
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 158083
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 159107
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 160131
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 161155
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 162179
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 163203
[aac @ 0x12a4900] Queue input is backward in time
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 164227
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 165251
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 154697
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 159400
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 160424
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 161448
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 162472
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 163496
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 164520
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 165544
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 166568
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 171795
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 172819
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 173843
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 174867
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 175891
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 176915
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 177939
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 178963
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 187279
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 189829
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 190853
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 191877
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 193354 >= 192901
[aac @ 0x12a4900] Queue input is backward in time
    Last message repeated 2 times
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 216609 >= 216172
[aac @ 0x12a4900] Queue input is backward in time
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 216609 >= 214950
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 216609 >= 214933
[adts @ 0x12a35a0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 216609 >= 214836
flashape
  • 567
  • 4
  • 10
  • Why is the extension `mp4` and how are you playing it? – Gyan Jun 16 '17 at 04:53
  • Oh, that was just for testing. I was playing it with VLC, however the actual output is streamed to youtube. The audio issue appears in both cases. – flashape Jun 16 '17 at 15:30
  • @flashape Does the audio still sound bad if you use a different tool to capture from pulse? Using `-f flv` and then outputting to MP4 is not recommended. Simplify your `ffmpeg` command to contain only the audio: does the issue still occur? – llogan Jun 16 '17 at 19:13
  • ok updated the post with the audio command and output...still the same issue, but with notable messages in the output. – flashape Jun 16 '17 at 21:09
  • Also, I was trying to use arecord to capture the audio, but when running "arecord -l" I get "arecord: device_list:268: no soundcards found..." – flashape Jun 16 '17 at 21:17
  • Ok recorded the audio using "arecord -d 10 test_arecord.wav". Same issue, sample file is here: http://s000.tinyupload.com/index.php?file_id=56279979395477575357 – flashape Jun 16 '17 at 21:34
  • Also, I feel like I should probably mention that the sound being recorded is coming from flash player in Firefox. – flashape Jun 16 '17 at 21:36
  • It is discouraged to crosspost [the same question](https://superuser.com/questions/1219992/ffmpeg-and-pulse-audio-creating-buzzy-speech) to multiple Stack Exchange sites. Offtopic here anyway since it's not a programming question. – llogan Jun 16 '17 at 22:08

2 Answers2

0

This is a truly bizarre problem... I've never heard sound like this.

I don't think it's a sample rate issue, at least not on its own. (Could be a broken resampler or something...) Speech is at the correct rate.

It almost sounds like there's something up with the channel count or bit depth. It's uniform, which means it's not a buffer under/overrun. It isn't apparent until there is sound, which means it's not an additional source of oscillation or noise.

default-sample-format = s32le

Is that your sample format actually in use? Because FFmpeg is telling you pcm_s16le. Try setting s16le in your Pulseaudio config and see what happens.

Brad
  • 159,648
  • 54
  • 349
  • 530
  • That's what it was originally. I just changed it back again, still no difference. – flashape Jun 17 '17 at 21:36
  • @flashape Well there definitely should be some difference. Your settings aren't being followed if nothing changes. – Brad Jun 17 '17 at 22:10
0

Apparently this was some issue with Firefox. Switched to Chrome and the problem went away.

flashape
  • 567
  • 4
  • 10