Integration of MMA and Libstreaming

Topics: Question
Feb 2, 2015 at 3:54 PM
I am with a problem to receive the frames when my stream server (android based on libstreaming) sends them with a resolution higher than 320x240 and bitrate higher than 100 kbps. Could you, please, give me any suggestions that what might be happening in this case?
Coordinator
Feb 2, 2015 at 6:23 PM
Only if I can have some additional information.

Can you post a wireshark capture so I can take a look?


Can you also describe the symptoms of your issues?

Thanks
Feb 2, 2015 at 7:58 PM
Thank you for your quick response.

You can check the wireshark capture here: https://drive.google.com/open?id=0ByX7amYqzcbAU3VMQ2JuRzBtdVE&authuser=0

I built a streaming server based on libstreaming (https://github.com/fyhertz/libstreaming-examples#example-1). I can configure this server to stream in different configurations.
I can, for example, set the framerate, bitrate and resolution of the streaming.
However, My RtspClient (MMA) can only receive the frames when I send them from android with resolutions lower or equal to 320x240 and with a bitrate lower or equal to 100 kbps.
My complete solution needs to do video streaming with full HD quality (1080p).
So, when I set my server to transmit the frames in that resolution and a bitrate of 4000 kbps I can't get complete frames in the RTSPClient.


Capture of MMA log:
Client Requested :OPTIONS rtsp://192.168.1.152:1234/ RTSP/1.0
Content-Encoding: utf-8
User-Agent: ASTI RTP Client
Blocksize: 4096
Date: Mon, 02 Feb 2015 20:29:24 GMT
CSeq: 1
Timestamp: 0.0009745


Client Received RtspMessage :RTSP/1.0 200 OK
Server:  MajorKernelPanic RTSP Server
Cseq:  1
Content-Length:  0
Public:  DESCRIBE,SETUP,TEARDOWN,PLAY,PAUSE


Client Requested :DESCRIBE rtsp://192.168.1.152:1234/ RTSP/1.0
Content-Encoding: utf-8
Accept: application/sdp
User-Agent: ASTI RTP Client
Blocksize: 4096
Date: Mon, 02 Feb 2015 20:29:24 GMT
CSeq: 2
Timestamp: 0.0279749


Client Received RtspMessage :RTSP/1.0 200 OK
Server:  MajorKernelPanic RTSP Server
Cseq:  2
Content-Length:  337
Content-Base:  192.168.1.152:1234/
Content-Type:  application/sdp

v=0
o=- 0 0 IN IP4 192.168.1.152
s=Unnamed
i=N/A
c=IN IP4 192.168.1.129
t=0 0
a=recvonly
m=video 5006 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=428028;sprop-parameter-sets=Z0KAKNoB4AiflQ==,aM4G4g==
a=cliprect:0,0,1080,1920
a=framesize:96 1920-1080
a=framerate:24
a=control:trackID=1

Client Requested :SETUP rtsp://192.168.1.152:1234/trackID=1 RTSP/1.0
Content-Encoding: utf-8
Transport: RTP/AVP;unicast;client_port=10000-10001
User-Agent: ASTI RTP Client
Blocksize: 4096
Date: Mon, 02 Feb 2015 20:29:26 GMT
CSeq: 3
Timestamp: 2.0510008


Client Received RtspMessage :RTSP/1.0 200 OK
Server:  MajorKernelPanic RTSP Server
Cseq:  3
Content-Length:  0
Transport:  RTP/AVP/UDP;unicast;destination=192.168.1.129;client_port=10000-10001;server_port=41230-37722;ssrc=221b1087;mode=play
Session:  1185d20035702ca
Cache-Control:  no-cache


Client Requested :PLAY rtsp://192.168.1.152:1234/ RTSP/1.0
Content-Encoding: utf-8
Range: npt=0-
User-Agent: ASTI RTP Client
Blocksize: 4096
Date: Mon, 02 Feb 2015 20:29:27 GMT
Session: 1185d20035702ca
CSeq: 4
Timestamp: 3.0630127


Client Received RtspMessage :RTSP/1.0 200 OK
Server:  MajorKernelPanic RTSP Server
Cseq:  4
Content-Length:  0
RTP-Info:  url=rtsp://192.168.1.152:1234/trackID=1;seq=0
Session:  1185d20035702ca

  *****************Playing from Live Source
        *****************Local Id -1355953284   *****************Remote Id 572199047
        *****************
StartedListening to :rtsp://192.168.1.152:1234/
Waiting for connection. Press Q to exit
Press K to send KeepAlive
Press D to DisconnectSocket
Press A to Attach events for packets
Press E to Detach packet events
Press I to Attach Interleaved events
Press U to Detach Interleaved events
Client Playing for :00:00:00.0039999
        Got a RTPFrame(96) PacketCount = 14 Complete = True HighestSequenceNumber = 45
        Got a RTPFrame(96) PacketCount = 16 Complete = True HighestSequenceNumber = 61
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 78
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 95
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 112
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 129
        Got a RTPFrame(96) PacketCount = 16 Complete = True HighestSequenceNumber = 145
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 162
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 179
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 196
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 213
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 230
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 247
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 264
        *******Got a RTPFrame With Missing Packets PacketCount = 16 Complete = False HighestSequenceNumber = 281
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 298
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 315
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 332
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 349
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 366
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 383
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 400
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 417
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 434
        Got a RTPFrame(96) PacketCount = 1 Complete = True HighestSequenceNumber = 435
        Got a RTPFrame(96) PacketCount = 1 Complete = True HighestSequenceNumber = 435
        *******Got a RTPFrame With Missing Packets PacketCount = 20 Complete = False HighestSequenceNumber = 459
        Got a RTPFrame(96) PacketCount = 9 Complete = True HighestSequenceNumber = 468
        Got a RTPFrame(96) PacketCount = 16 Complete = True HighestSequenceNumber = 484
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 501
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 518
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 535
        *******Got a RTPFrame With Missing Packets PacketCount = 14 Complete = False HighestSequenceNumber = 552
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 569
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 586
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 603
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 620
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 637
        Got a RTPFrame(96) PacketCount = 18 Complete = True HighestSequenceNumber = 655
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 672
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 689
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 706
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 723
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 740
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 757
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 774
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 791
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 808
        Got a RTPFrame(96) PacketCount = 18 Complete = True HighestSequenceNumber = 826
        Got a RTPFrame(96) PacketCount = 17 Complete = True HighestSequenceNumber = 843
        *******Got a RTPFrame With Missing Packets PacketCount = 16 Complete = False HighestSequenceNumber = 860
        Got a RTPFrame(96) PacketCount = 1 Complete = True HighestSequenceNumber = 861
        Got a RTPFrame(96) PacketCount = 1 Complete = True HighestSequenceNumber = 861
        *******Got a RTPFrame With Missing Packets PacketCount = 29 Complete = False HighestSequenceNumber = 891
        Got a RTPFrame(96) PacketCount = 11 Complete = True HighestSequenceNumber = 902
As you can see, I always get incomplete frames on MMA RtspClient (*******Got a RTPFrame With Missing Packets PacketCount = 16 Complete = False HighestSequenceNumber = 281).
Therefore, I can never save a valid h264 video.

I hope I have been more clear with my problem.

Thanks.
Feb 2, 2015 at 8:07 PM
One detail more: I use VLC player as client to play this same stream and everything works fine.
Coordinator
Feb 2, 2015 at 11:19 PM
I will take a look at this as soon as possible, probably not before Wednesday the 4th.

In the mean time can you indicate what IP addresses I am looking at if possible as you didn't filter any of the traffic :)

I believe you that VLC player works fine but can you also include a capture of VLC working where my library does not so I can compare the results. (Please filter the results if possible)

Thanks!
Feb 3, 2015 at 12:33 PM
Sorry my friend... I didn't realize I had not filtered the wireshark capture.

In order to do the comparative between VLC and MMA you need of the wireshark capture, right?
Here are them:

MMA RTSPClient - https://drive.google.com/file/d/0ByX7amYqzcbAQnltbGNFWGtnTGs/view?usp=sharing
VLC Player - https://drive.google.com/file/d/0ByX7amYqzcbAMTJHX19vemdBaVU/view?usp=sharing

The Server IP: 192.168.1.152.
The Client IP: 192.168.1.129.

I really need your help to close a alternative solution to VLC (client side) and I am available for any clarification.

Thanks again!
Feb 3, 2015 at 8:16 PM
I realized that the frequency of MMA receives the incompletes frames is related to the Key_I_Frame_Interval set on libstreaming.
For instance, if this interval was set to 5 seconds, for each 5 seconds the MMA receives a incomplete frame.

Perhaps this information can help you to understand better my problem :)).
Coordinator
Feb 4, 2015 at 6:00 PM
Thank you for the additional information and captures.

I will be checking this out at some point today and I will update you!
Coordinator
Feb 6, 2015 at 12:10 AM
Hey, you are probably just experiencing UDP re-ordering.

No matter what configuration is applied to the socket in code it is at the mercy of the physical world.

If the data from packet with seq = 67 arrives before the packet with seq = 66 or seq = 65 then the RtpClient handles this with a FrameChanged event.

It is up to the consumer to determine if the frame is Complete or will be complete or if a NACK may be required etc.

I think you are just confused by the console output, please check at the end of the tests the results and you will see

'
RTP Info▓▓▓▓▓▓▓▓▓▓▓▓
Rtp Packets Recieved: X
Encountered Frames with missing packets: 0
Encountered Empty Frames: 0
Total Frames: 0
Frames still missing packets: 0
'

If no the frames which have missing packets will be in the 'missing' HashSet.

Take a look and let me know if you have any further questions!
Marked as answer by juliusfriedman on 2/9/2015 at 9:16 AM
Feb 9, 2015 at 3:47 PM
Ok guy...

I will verify your suggestions and try to solve my problem.
I will let you know if i have any other questions.

Thanks.
Marked as answer by juliusfriedman on 2/9/2015 at 9:16 AM