Developing Xamarin RTP client for an existing gateway streaming video and audio

Topics: Question
Feb 1, 2016 at 4:15 PM
Hello,

the managed media Aggregation Solution Looks very interesting and promising, thanks for that!

I have my xamarin Client which has already the sip Connection to the customers Gateway working (using sip). And I already have a SessionDescription for an incoming call.

Now I want to receive data using RTP for Streaming Video and Audio.
I am new to this Topic.

Is there a sample for a Client app using RTP from a sessionDescription?
Or some hint where do I have to start? Some help would be great :-)

Thanks a lot and have a nice day!

Eric
Feb 2, 2016 at 11:33 AM
Got the Connection working, and I am working on the (media) Options now, thanks a lot!
Feb 2, 2016 at 1:39 PM
Hello,

I got my SIP Connection working.

Then I initialize my Client like this (I am not sure if this is right):
                Media.Rtsp.RtspClient client = new Media.Rtsp.RtspClient("rtsp://" + session.Connection.Address + ":" + RTPPort, Media.Rtsp.RtspClient.ClientProtocolType.Udp);

                client.SessionId = session.Owner.SessionID.ToString();

                client.OnConnect += (sender, args) => { Logger.Instance.Log("Connected to :" + client.CurrentLocation); };
                client.OnRequest += (sender, request) => { Logger.Instance.Log("Client Requested :" + request.Location + " " + request.MethodString); };
                client.OnResponse += (sender, request, response) => { Logger.Instance.Log("Client got response :" + response.StatusCode + ", for request: " + request.Location + " " + request.MethodString); };
                client.OnPlay += (sender, args) =>
                {
                    //Indicate if LivePlay
                    if (client.LivePlay)
                    {
                        Logger.Instance.Log("Playing from Live Source");
                    }

                    //Indicate if StartTime is found
                    if (client.StartTime.HasValue)
                    {
                        Logger.Instance.Log("Media Start Time:" + client.StartTime);

                    }                

                    //Indicate if EndTime is found
                    if (client.EndTime.HasValue)
                    {
                        Logger.Instance.Log("Media End Time:" + client.EndTime);
                    }
                };
                client.OnDisconnect += (sender, args) => { Logger.Instance.Log("Disconnected from :" + client.CurrentLocation); };

                try
                {
                    client.Connect();
                }
                catch (Exception ex)
                {
                    Logger.Instance.Log("Problem occured during connection RTP: " + ex.ToString());
                }

                //If We are connected
                if (client.IsConnected && client.Client != null)
                {
                    //Try to StartListening
When I now

client.StartPlaying();

it takes a Long time, then I get an exception that tells me

{Media.Common.TaggedException1[Media.Rtsp.RtspClient]: Unable to get options, See InnerException. ---> Media.Common.TaggedException1[Media.Rtsp.RtspMessage]: See Tag for Response.
--- End of inner exception stack trace ---
at Media.Common.Extensions.Exception.ExceptionExtensions.Raise[T] (Media.Common.TaggedException1 exception) [0x00003] in C:\Projekte\net7mma-111867\Common\Extensions\ExceptionExtensions.cs:70
at Media.Common.Extensions.Exception.ExceptionExtensions.RaiseTaggedException[T] (Media.Common.Extensions.Exception.T tag, System.String message, System.Exception innerException) [0x00000] in C:\Projekte\net7mma-111867\Common\Extensions\ExceptionExtensions.cs:120
at Media.Rtsp.RtspClient.SendOptions (Boolean useStar, System.String sessionId) [0x0016a] in C:\Projekte\net7mma-111867\Rtsp\RtspClient.cs:3182
at SKSSipClient.ViewModels.IncomingViewModel.InitSession (Independentsoft.Sip.Sdp.SessionDescription session) [0x00239] in C:\Projekte\SKS-Sip-Client-PJSIP-OwnMasterDLL-withSip.NetAndMMA\Kinkel-ClientUITest\Kinkel_ClientUITest\ViewModels\IncomingViewModel.cs:195 }

InnerException says:
"Media.Common.TaggedException
1[Media.Rtsp.RtspMessage]"

When I use SendOptions instead I get the same exception.

Any help please? What am I doing wrong?

I did not apply any media Information yet, could that be the Problem?

Thanks a lot!

Eric
Feb 2, 2016 at 2:28 PM
This might be a Problem the existing Gateway supports RTP but not RTSP?
If that is the problem, then I am back to my question how to initialize a new RTP Client from my existing sip Connection.
Feb 2, 2016 at 4:25 PM
I can at least activate a RTP Client now from my sip session it seems.
                Media.Sdp.SessionDescription sessionDescription = new Media.Sdp.SessionDescription(session.ToString());
                sessionDescription.SessionName = session.Name;
                sessionDescription.SessionId = session.Owner.SessionID.ToString();

                RtpClient rtpClient = Media.Rtp.RtpClient.FromSessionDescription(sessionDescription);

                rtpClient.RtpPacketReceieved += (sender, rtpPacket, tc) => { Logger.Instance.Log("Got a RTP packet, SequenceNo = " + rtpPacket.SequenceNumber + " PayloadType = " + rtpPacket.PayloadType + " Length = " + rtpPacket.Length); };
                rtpClient.RtpFrameChanged += (sender, sender2, rtpFrame, tc) => { Logger.Instance.Log("Got a RTPFrame PacketCount = " + rtpFrame.CurrentFrame.Count + " Complete = " + rtpFrame.CurrentFrame.IsComplete); };
                rtpClient.RtcpPacketReceieved += (sender, rtcpPacket, tc) => { Logger.Instance.Log("Got a RTCP packet " + " Type=" + rtcpPacket.PayloadType + " Length=" + rtcpPacket.Length + " Bytes = " + BitConverter.ToString(rtcpPacket.Payload.Array)); };
                rtpClient.RtcpPacketReceieved += (sender, rtcpPacket, tc) => { Logger.Instance.Log("Sent a RTCP packet " + " Type=" + rtcpPacket.PayloadType + " Length=" + rtcpPacket.Length + " Bytes = " + BitConverter.ToString(rtcpPacket.Payload.Array)); };

                rtpClient.Activate();
This is not crashing. I am trying to get Video (H264) and Audio packages now. Do they need to be declared in the sip part or can I add media descriptions here?
Coordinator
Feb 4, 2016 at 10:57 PM
Hey, sorry for the late reply.

Glad to see everything is working.

There's an example in the unit tests for RFC6184 which is used for H.264, take a look at that and let me know what questions you have.
Marked as answer by juliusfriedman on 2/4/2016 at 2:57 PM
Feb 5, 2016 at 2:25 PM
Hello Julius,

on my Xamarin App it is still not working (that's the not implemented bug. On my PC I can activate the RTPClient, but it does not receive RTP Packets.
Using Wireshark I found out the RTP Client seems to send a "Goodbye" when activated. I do not have one in my code.

Do you have any idea what could cause this goodbye?

Thanks a lot,

Eric
Coordinator
Feb 5, 2016 at 7:14 PM
Goodbye is sent as a result of not receiving data in the required amount of time, which defaults to 5 seconds.

Do you see RTP packets arriving from the network and your just not receiving them?

Can you also please post a copy of the SDP so I can take a look please and also a small Wireshark capture so I can see the resulting traffic?

I will take a look and let you know what I find.
Marked as answer by juliusfriedman on 2/5/2016 at 11:14 AM
Feb 8, 2016 at 2:45 PM
Hello Julius,

thanks a lot for your help again.

After adding a small "Hack" to your library, I got it working on my android device (hack is described here):
https://net7mma.codeplex.com/discussions/650864

My SIP Sessiondescription is now this when I activate the rtp client:

{v=0
o=sdm 2890844526 2890844526 IN IP4 127.0.0.1
s=SIP Call
c=IN IP4 192.168.1.114
t=0 0
m=audio 7078 RTP/AVP 8 0
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
m=video 9078 RTP/AVP 96 97 98
a=rtpmap:96 H264/90000
a=rtpmap:97 H264/90000
a=rtpmap:98 H264/90000
}

I tried with and without Setting different ports in the sip media description.

Activate the RTP Client now works. First nothing happened, then I found this in your RTPclient:
            //Send the initial senders report
            SendSendersReports();

            //Send the initial receivers report
            SendReceiversReports();
They were disabled by comment, I enabled them again.

Now I get packages after RTPClient.Activate, but I always get this error message:

502 47.225575 192.168.1.114 192.168.1.118 ICMP 130 Destination unreachable (Port unreachable)

Any idea what am I missing here? I am trying with my android attached via USB to my PC with Visual Studio 2015 and Xamarin 4.0 still.
The Nexus 7 tablet is successfully connected via WLAN and after SIP Initialisation to the Gateway which send RTP packets now.

Thanks a lot!

Frame 467: 439 bytes on wire (3512 bits), 439 bytes captured (3512 bits) on interface 0
Interface id: 0 (\Device\NPF_{F0D43F54-ABF9-423D-9BF8-443A02BA5AB0})
Encapsulation type: Ethernet (1)
Arrival Time: Feb  8, 2016 15:28:53.977689000 Mitteleurop�ische Zeit
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1454941733.977689000 seconds
[Time delta from previous captured frame: 0.039867000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 31.516130000 seconds]
Frame Number: 467
Frame Length: 439 bytes (3512 bits)
Capture Length: 439 bytes (3512 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:sip]
[Coloring Rule Name: UDP]
[Coloring Rule String: udp]
Ethernet II, Src: AsustekC_38:41:bf (08:60:6e:38:41:bf), Dst: Camerone_fe:02:a4 (d4:79:c3:fe:02:a4)
Destination: Camerone_fe:02:a4 (d4:79:c3:fe:02:a4)
Source: AsustekC_38:41:bf (08:60:6e:38:41:bf)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 192.168.1.118, Dst: 192.168.1.114
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
Total Length: 425
Identification: 0x3b86 (15238)
Flags: 0x00
Fragment offset: 0
Time to live: 64
Protocol: UDP (17)
Header checksum: 0xb985 [validation disabled]
Source: 192.168.1.118
Destination: 192.168.1.114
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 44975 (44975), Dst Port: 5060 (5060)
Source Port: 44975
Destination Port: 5060
Length: 405
Checksum: 0x6876 [validation disabled]
[Stream index: 3]
Session Initiation Protocol (REGISTER)
Feb 8, 2016 at 3:16 PM
Here is the port unreachable Error Log from wireshark.

Since SocketExtension FindOpenPort did not work for me, I tried to return Port 30000 and 30001 there.


Frame 1917: 130 bytes on wire (1040 bits), 130 bytes captured (1040 bits) on interface 0
Interface id: 0 (\Device\NPF_{F0D43F54-ABF9-423D-9BF8-443A02BA5AB0})
Encapsulation type: Ethernet (1)
Arrival Time: Feb  8, 2016 16:08:30.897812000 Mitteleurop�ische Zeit
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1454944110.897812000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 581.378490000 seconds]
Frame Number: 1917
Frame Length: 130 bytes (1040 bits)
Capture Length: 130 bytes (1040 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:icmp:ip:udp:rtcp]
[Coloring Rule Name: ICMP errors]
[Coloring Rule String: icmp.type eq 3 || icmp.type eq 4 || icmp.type eq 5 || icmp.type eq 11 || icmpv6.type eq 1 || icmpv6.type eq 2 || icmpv6.type eq 3 || icmpv6.type eq 4]
Ethernet II, Src: Camerone_fe:02:a4 (d4:79:c3:fe:02:a4), Dst: AsustekC_38:41:bf (08:60:6e:38:41:bf)
Destination: AsustekC_38:41:bf (08:60:6e:38:41:bf)
Source: Camerone_fe:02:a4 (d4:79:c3:fe:02:a4)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 192.168.1.114, Dst: 192.168.1.118
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes
Differentiated Services Field: 0xce (DSCP: Unknown, ECN: ECT(0))
Total Length: 116
Identification: 0x5e75 (24181)
Flags: 0x00
Fragment offset: 0
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x970d [validation disabled]
Source: 192.168.1.114
Destination: 192.168.1.118
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Internet Control Message Protocol
Type: 3 (Destination unreachable)
Code: 3 (Port unreachable)
Checksum: 0x818b [correct]
Unused: 00000000
Internet Protocol Version 4, Src: 192.168.1.118, Dst: 192.168.1.114
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes
    Differentiated Services Field: 0x2f (DSCP: Unknown, ECN: CE)
    Total Length: 88
    Identification: 0xf711 (63249)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 255
    Protocol: UDP (17)
    Header checksum: 0x001b [validation disabled]
    Source: 192.168.1.118
    Destination: 192.168.1.114
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 30002 (30002), Dst Port: 1 (1)
    Source Port: 30002
    Destination Port: 1
    Length: 68
    Checksum: 0x034d [validation disabled]
    [Stream index: 110]
Real-time Transport Control Protocol (Sender Report)
    10.. .... = Version: RFC 1889 Version (2)
    ..0. .... = Padding: False
    ...0 0000 = Reception report count: 0
    Packet type: Sender Report (200)
    Length: 6 (28 bytes)
    Sender SSRC: 0x8cfda3e9 (2365432809)
    Timestamp, MSW: 3663932907 (0xda632deb)
    Timestamp, LSW: 4294967295 (0xffffffff)
    [MSW and LSW as NTP timestamp: Feb  8, 2016 15:08:27.999999000 UTC]
    RTP timestamp: 4294901760
    Sender's packet count: 0
    Sender's octet count: 0
Real-time Transport Control Protocol (Source description)
    10.. .... = Version: RFC 1889 Version (2)
    ..0. .... = Padding: False
    ...0 0001 = Source count: 1
    Packet type: Source description (202)
    Length: 7 (32 bytes)
    Chunk 1, SSRC/CSRC 0x8CFDA3E9
        Identifier: 0x8cfda3e9 (2365432809)
        SDES items
[RTCP frame length check: OK - 60 bytes]
Feb 11, 2016 at 12:32 PM
Hello,

I found this on the net and added it to Media.RTPClient:
    public static System.Net.IPAddress MapToIPv6(this System.Net.IPAddress addr)
    {
        if (addr.AddressFamily != System.Net.Sockets.AddressFamily.InterNetwork)
            System.Console.WriteLine("Must pass an IPv4 address to MapToIPv6");

        string ipv4str = addr.ToString();

        return System.Net.IPAddress.Parse("::ffff:" + ipv4str);
    }
I used it to get rid of one of my not implemented exceptions. Perhaps this could be added to your Project as a Workaround when the original method is missing on Xamarin?

I now run into a IP Protection Level Problem on my Xamarin client it seems.
Coordinator
Feb 17, 2016 at 5:24 PM
Edited Feb 19, 2016 at 12:03 PM
I addressed most of this in the Bug discussion, let me know if you still need me to look at this.
Marked as answer by juliusfriedman on 2/17/2016 at 9:24 AM