Xuggler-3.4/ffmpeg rtmp publishing fails

[lsanderson]

To reproduce:

1) Install xuggler-3.4 or later
2) Run red-5 0.9.0 with the oflaDemo app installed
3) Using the xuggler provided ffmpeg, execute the following:

ffmpeg -i $RED5_HOME/webapps/oflaDemo/streams/avatar.flv -re -acodec copy -vcodec copy -f flv rtmp://localhost/oflaDemo/test

Expected result: The video should be published live to the Red-5 server.

Actual result: The following stack trace

[ERROR] [NioProcessor-1] org.red5.server.net.rtmp.codec.RTMPProtocolDecoder - Error decoding buffer 
org.red5.server.net.protocol.ProtocolException: Error during decoding                               
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:203) [red5.jar:na]                                                                                        
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeBuffer(RTMPProtocolDecoder.java:122) [red5.jar:na]                                                                                  
        at org.red5.server.net.rtmp.codec.RTMPMinaProtocolDecoder.decode(RTMPMinaProtocolDecoder.java:61) [red5.jar:na]                                                                                 
        at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:225) [mina-core-2.0.0-RC1.jar:na]                                                                  
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1200(DefaultIoFilterChain.java:46) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:793) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:119) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:434) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:426) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:638) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:598) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:587) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$400(AbstractPollingIoProcessor.java:61) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:969) [mina-core-2.0.0-RC1.jar:na]
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core-2.0.0-RC1.jar:na]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_17]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_17]
        at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17]
Caused by: java.lang.IllegalArgumentException: null
        at java.nio.Buffer.limit(Buffer.java:249) [na:1.6.0_17]
        at org.red5.io.amf.Input.bufferToString(Input.java:275) [red5.jar:na]
        at org.red5.io.amf.Input.getString(Input.java:258) [red5.jar:na]
        at org.red5.io.amf.Input.readPropertyName(Input.java:565) [red5.jar:na]
        at org.red5.io.amf.Input.readKeyValues(Input.java:351) [red5.jar:na]
        at org.red5.io.amf.Input.readSimpleObject(Input.java:501) [red5.jar:na]
        at org.red5.io.amf.Input.readObject(Input.java:537) [red5.jar:na]
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeStreamMetadata(RTMPProtocolDecoder.java:978) [red5.jar:na]
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodeMessage(RTMPProtocolDecoder.java:564) [red5.jar:na]
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decodePacket(RTMPProtocolDecoder.java:385) [red5.jar:na]
        at org.red5.server.net.rtmp.codec.RTMPProtocolDecoder.decode(RTMPProtocolDecoder.java:190) [red5.jar:na]
        ... 18 common frames omitted

The problem is related to how ffmpeg passes parameters to the rtmp NOTIFY message type. It is using an ECMA-Array (CORE_MAP in Red-5 parlance) rather than an object.

The attached patch fixes it for me.

[lsanderson]

Patch

[aclarke]

Applied Larry's patch with formatting changes, ran unit and self tests and everything passes. Also confirmed this works with latest tip of tree Xuggler.

Revision r4038