RtpSource from SDP: multicast

Topics: Question
Feb 17, 2015 at 1:54 PM
Hi,

After getting the latest sources and adding multicast subscription to the rtpclient (uploaded as a patch), I am looking for a way to get the multicast stream to be forwarded.

I can see the UDP packets in wireshark and VLC tries two times to open the stream from the rtpserver but then gives up after (what I suppose is a timeout). I see entries related to transport context missing HandleIncomingRtcpPacket - No Context for packet 265914378@201

I'm not sure if I understand the way it should work, but even an RtspStreamArchiver instance on which I call prepare and start for the defined stream doesn't write anything in the folder it creates.

If needed, I can send the wireshark capture for the session.

Kind regards,

Bart
Server Started @ 17/02/2015 14:39:47
Starting Stream: test Id=c425d009-6170-49ec-9fda-4b7c4fecef04
Listening on: 0.0.0.0:554

Accepted Client: 2a6dc7b0-12ca-442c-b285-776987bc5f32 @ 17/02/2015 14:39:54
Request=> OPTIONS rtsp://10.16.0.10/live/test RTSP/1.0
CSeq:  2
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Adding Client: 2a6dc7b0-12ca-442c-b285-776987bc5f32
Response=> RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:39:54 GMT

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Request=> DESCRIBE rtsp://10.16.0.10/live/test RTSP/1.0
CSeq:  3
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Accept:  application/sdp

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Response=> RTSP/1.0 200 OK
CSeq: 3
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/
Content-Length: 393
Content-Encoding: utf-8
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:39:54 GMT

v=0
o=ASTI-Media-Server 15604358331546532788 -2842385742163018826 IN IP4 10.16.0.10
s=ASTI-Streaming-Session-test
i=H264 session of stream1-low
u=http://10.16.1.22
t=0 0
m=video 0/1 RTP/AVP 102
i=Video stream
a=fmtp:102 width=1280;height=720;depth=0;framerate=30000;fieldrate=30000;
a=framerate:30
a=rtpmap:102 H264/90000
a=control:/live/c425d009-6170-49ec-9fda-4b7c4fecef04/video
 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Request=> SETUP rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04//liv
e/c425d009-6170-49ec-9fda-4b7c4fecef04/video RTSP/1.0
CSeq:  4
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Transport:  RTP/AVP;unicast;client_port=60546-60547

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Response=> RTSP/1.0 200 OK
CSeq: 4
Transport: RTP/AVP;source=10.16.0.10;unicast;client_port=60546-60547;server_port
=30002-30003;ssrc=2F691701
Session: 882082602;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:39:54 GMT

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Request=> PLAY rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/ RTSP
/1.0
CSeq:  5
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Session:  882082602
Range:  npt=0.000-

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Response=> RTSP/1.0 200 OK
CSeq: 5
Session:  882082602;timeout=30
Range: npt=now-
RTP-Info: url=rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/video;
ssrc=2F691701
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:39:54 GMT

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

Media.Rtp.RtpClient-08a59265-fb3b-4dee-93e9-13b3ba251995@SendRecieve - Begin
Media.Rtp.RtpClient-08a59265-fb3b-4dee-93e9-13b3ba251995@ParseAndCompleteData -
Remaining= 4
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 544371827@250
Media.Rtp.RtpClient-08a59265-fb3b-4dee-93e9-13b3ba251995@ParseAndCompleteData -
Remaining= 4
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 544371827@250
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@201
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@202
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@201
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@202
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@201
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@202
RestartFaultedStreams
DisconnectAndRemoveInactiveSessions
Request=> TEARDOWN rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/
RTSP/1.0
CSeq:  6
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Session:  882082602

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@201
Accepted Client: 86ad9910-f0c8-490b-ae5a-1719f20da440 @ 17/02/2015 14:40:05
08a59265-fb3b-4dee-93e9-13b3ba251995HandleIncomingRtcpPacket - No Context for pa
cket 207696049@203
Response=> RTSP/1.0 200 OK
CSeq: 6
Session:  882082602;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:05 GMT

 Session=> 2a6dc7b0-12ca-442c-b285-776987bc5f32

An existing connection was forcibly closed by the remote host
    at System.Net.Sockets.Socket.EndSendTo(IAsyncResult asyncResult)
   at Media.Rtsp.RtspServer.ProcessSendComplete(IAsyncResult ar) in c:\Projects\
Research\net7mma\RtspServer\RtspServer.cs:line 1551


Marking Client:2a6dc7b0-12ca-442c-b285-776987bc5f32 Disconnected
Request=> OPTIONS rtsp://10.16.0.10/live/test RTSP/1.0
CSeq:  2
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Adding Client: 86ad9910-f0c8-490b-ae5a-1719f20da440
Response=> RTSP/1.0 200 OK
CSeq: 2
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:05 GMT

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Request=> DESCRIBE rtsp://10.16.0.10/live/test RTSP/1.0
CSeq:  3
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Accept:  application/sdp

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Response=> RTSP/1.0 200 OK
CSeq: 3
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/
Content-Length: 393
Content-Encoding: utf-8
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:05 GMT

v=0
o=ASTI-Media-Server 15604358378791279095 -2842385694918272519 IN IP4 10.16.0.10
s=ASTI-Streaming-Session-test
i=H264 session of stream1-low
u=http://10.16.1.22
t=0 0
m=video 0/1 RTP/AVP 102
i=Video stream
a=fmtp:102 width=1280;height=720;depth=0;framerate=30000;fieldrate=30000;
a=framerate:30
a=rtpmap:102 H264/90000
a=control:/live/c425d009-6170-49ec-9fda-4b7c4fecef04/video
 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Request=> SETUP rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04//liv
e/c425d009-6170-49ec-9fda-4b7c4fecef04/video RTSP/1.0
CSeq:  4
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Transport:  RTP/AVP/TCP;unicast;interleaved=0-1

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Response=> RTSP/1.0 200 OK
CSeq: 4
Transport: RTP/AVP/TCP;source=10.16.0.10;interleaved=0-1;ssrc=FC7D2ABE
Session: -216715872;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:05 GMT

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Request=> PLAY rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/ RTSP
/1.0
CSeq:  5
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Session:  -216715872
Range:  npt=0.000-

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Response=> RTSP/1.0 200 OK
CSeq: 5
Session:  -216715872;timeout=30
Range: npt=now-
RTP-Info: url=rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/video;
ssrc=FC7D2ABE
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:05 GMT

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Media.Rtp.RtpClient-aed7dac9-f610-433b-b46d-93607d9048c7@SendRecieve - Begin
aed7dac9-f610-433b-b46d-93607d9048c7HandleIncomingRtcpPacket - No Context for pa
cket 1265914378@201
aed7dac9-f610-433b-b46d-93607d9048c7HandleIncomingRtcpPacket - No Context for pa
cket 1265914378@202
aed7dac9-f610-433b-b46d-93607d9048c7HandleIncomingRtcpPacket - No Context for pa
cket 1265914378@201
aed7dac9-f610-433b-b46d-93607d9048c7HandleIncomingRtcpPacket - No Context for pa
cket 1265914378@202
Request=> TEARDOWN rtsp://10.16.0.10/live/c425d009-6170-49ec-9fda-4b7c4fecef04/
RTSP/1.0
CSeq:  6
User-Agent:  LibVLC/2.1.3 (LIVE555 Streaming Media v2014.01.21)
Session:  -216715872

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

Response=> RTSP/1.0 200 OK
CSeq: 6
Session:  -216715872;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Tue, 17 Feb 2015 14:40:15 GMT

 Session=> 86ad9910-f0c8-490b-ae5a-1719f20da440

An existing connection was forcibly closed by the remote host
    at System.Net.Sockets.Socket.EndSendTo(IAsyncResult asyncResult)
   at Media.Rtsp.RtspServer.ProcessSendComplete(IAsyncResult ar) in c:\Projects\
Research\net7mma\RtspServer\RtspServer.cs:line 1551


Marking Client:86ad9910-f0c8-490b-ae5a-1719f20da440 Disconnected
Server Streamed : 340
Rtsp Sent : 2706
Rtsp Recieved : 1592
Stopping Server
Connected Clients:2
Server Stopped @ 17/02/2015 14:40:17
Uptime:00:00:29.9619959
Sent:2706
Receieved:1592
Disposing Client: 2a6dc7b0-12ca-442c-b285-776987bc5f32 @ 17/02/2015 14:40:17
Disposing Client: 86ad9910-f0c8-490b-ae5a-1719f20da440 @ 17/02/2015 14:40:17
Coordinator
Feb 17, 2015 at 4:47 PM
I didn't review your patch in detail yet but I will tonight or tomorrow.

You also are using your own socket sending and receiving code.

Whatever I can do to help I will but technically your doing it wrong.

Your patch is probably helpful but useless as it shouldnt be needed as one can use the existing sockets overload of the initialization method of the transport context right now.

hopefully that makes senses.

E.g configure your socket and then give it to the context.

if you need anything further please let me know.

I will eventually support automatically creating the multicast socket but more thought then what you have put into the patch needs to be there.

And definitely should use ISocketReference...

Thanks and let me know how I can help further.

P.s. the same can be said for month time and date parsing in relation to your riff patch. (Needs a utility method).

Thanks again.
Marked as answer by juliusfriedman on 2/17/2015 at 9:47 AM
Feb 19, 2015 at 10:33 AM
Hi,

I have reworked my example a bit after your reply; mostly because the multicast wasn't working yet.

I've also been thinking about passing in an existing socket to the RtpClient constructor, but saw the same socket would then be reused for all media descriptions in the SDP. In the case the SDP contains a video and an audio stream, each on a different port, multiple sockets would need to be passed to the constructor and some filtering should be done to match the socket for the current media description.

I agree with you that more thought has to go into creating the sockets, my intention was just to get it to work with my sdp etc. For now I will create classes inheriting rtpsource and rtpclient for my use case. If you want to I can send you the code.

Thanks for the great work you've already put into the library.

Kind regards,

Bart
Coordinator
Feb 19, 2015 at 5:59 PM
Well my first question would be how can you submit a patch which you didn't test?

Next, I would ask what do you mean that each passing in a socket forces it to be used for all media descriptions?

NO, It does not. Not even with your patch; With all due respect the SSRC and Payload Type of the packet do that and if there is a Media Description also on that context it can be related as such but not only to or from because the payload types may overload for audio video and other wise in the same session, thus why you need a different SSRC unless you are using a specific profile.

You should post your code only so that I can review your use case and determine if that is also a supported paradigm and outline how you can achieve the same thing without deriving the class because that is not my intention, I don't want to force people to do that otherwise I would have marked it abstract.

E.g. why you have to derive it JUST to achieve multicast sending or receiving is BEYOND me at the current time, however I hope you are aware there is also 'Rtp.Conference' which is more likely where you should focus your energy and more likely where you efforts would be fruitful especially since I haven't done really anything there yet; you would be free to make it suit your application and style of programming and then have me correct and or adopt it and support it for inclusion in the library in a way such it would benefit everyone including yourself.

Also why would you want to use BeginSend or Being Receive especially for multicast? I hope you know that ends up being slower than using Socket.Read and Socket.Write by a factor of about at least 2 fold especially when you consider the context switching involved. With a normal read of write there is still a switch but only for the call, not the call to the thread pool then the call to the logic then the call back which all must be synchronized manually or your are in big trouble and also causes you to have to lock which puts you in danger of dead lock.

If you would like to create a new issue for Rtp.Conference go ahead, this way you can also post your updated code and patches there and I can check them out as required and comment with you and we can work together to get your application working first and then I can adopt the code for use in my library where required.

Hopefully that made sense.

No problem at all and if I can do anything to improve the library or my answer further please let me know!

Thanks!
Marked as answer by juliusfriedman on 2/19/2015 at 10:59 AM
Feb 20, 2015 at 1:35 PM
Regarding the patch: my bad. I forgot to include IPv6 support (which you apparently noticed) and did not subscribe to the correct port.

For the comment regarding the existing socket and multiple mediadescriptions:

Using this SDP my understanding is video comes on port 51372 and audio on port 49170.
v=0
o=jdoe 2890844526 2890842807 IN IP4 10.47.16.5
s=SDP Seminar
i=A Seminar on the session description protocol
u=http://www.example.com/seminars/sdp.pdf
e=j.doe@example.com (Jane Doe)
c=IN IP4 224.2.17.12/127
t=2873397496 2873404696
a=recvonly
m=audio 49170 RTP/AVP 0
m=video 51372 RTP/AVP 99
a=rtpmap:99 h263-1998/90000
If I create an instance of RtpClient using this SDP and a socket instance, I asume the foreach over media descriptions in the session description will call TransportContext.Initialize with the same Socket.
foreach over MediaDescriptions in SessionDescription

...
if (false == hasSocket && string.Compare(md.MediaProtocol, Media.Rtp.RtpClient.RtpAvpProfileIdentifier, true) == 0)
{
    --> new socket for this media description
    int localPort = Utility.FindOpenPort(ProtocolType.Udp);
    tc.Initialize(localIp, remoteIp, localPort++, localPort++, rtpPort ?? md.MediaPort, rtcpPort ?? md.MediaPort + 1);
}
else if (hasSocket)//If had a socket use it
{
    tc.Initialize(existingSocket); --> initialize 1st, 2nd, ... transport context with same socket
}
else
{
    tc.Initialize(localIp, remoteIp, rtpPort ?? md.MediaPort);
}
...
Coordinator
Feb 20, 2015 at 2:35 PM
Edited Feb 21, 2015 at 3:39 PM
Ain't a thing.

Your question is asking me about calling Initialize but what's the question? about the media description? What about it?

Is the static method TransportContext.FromMediaDescription not suited for you?

I would also probably reserve the port once its verified as open.

There's also a RtpClient.FromSessionDescription.

All you did was post my code with some additional comments but honestly I have no idea what your question is.

The media description can be used to find an associated context for a packet.

So can the socket or ssrc.

if your context is not being found I would check the version of the context, the remote synchronization source identifier and associated media description payload types.

Why you can't configure the socket(s) with ISocketReferencence is also a question, it gets you the sockets used by any context of the RtpClient.

You can then enable broadcasting if required.

How are you filtering non rtp traffic on the multicast address is another thing to think about; another reason I don't automatically setup broadcasting.

The port can be defined but so what, the port doesn't enforce that I can't send non rtp traffic does it?

Are you awake how multicast communication works?

Usually, this is controlled at the network level.

Since you are doing this at the application level shouldn't you then be calling ParseAndComplete data only when you expect rtp or rtcp?

Are you attempting to use the RtpClinet to filter Non Rtp traffic ?

As previously stated if you post your code I will review and try to help.

The other option is to not use code and write a list of questions and I will attempt to answer them.

If you have questions about my code cite the line number and revision, don't post my code adding comments or modify my code without deriving the class because its confusing.

If you need anything else please let me know.
Marked as answer by juliusfriedman on 2/20/2015 at 9:21 AM
Coordinator
Feb 21, 2015 at 3:44 PM
You can also post a Wireshark capture (filtered) which will show me your traffic and better understand the requirements of your application.

From there I will probably be better equipped to respond on how you can accommodate your applications requirements and I could also possibly get some ideas for more general use of such concept in the library.

The sooner the better because once I start on something else I won't have as much time to spare.

Thanks.
Marked as answer by juliusfriedman on 2/21/2015 at 8:44 AM
Coordinator
Mar 9, 2015 at 2:44 AM
Hello,

I haven't heard from you since the last time I responded.

I have added some code which changes how events are raised and packets are processed.

There still are changes required which I will be working on which MUST be made if the RtpClient is filtering data from the Multicast Address, e.g. data which is not Rtp may still be desired to be propagated.

I will be using the OnInterleavedData function for this purpose and I will probably change it's name to OnDataReceived.

Hopefully that gives you enough information to indicate to me if your paradigm would be supported.

Let me know so I can ensure your use case is supported.

I have an update planned for sometime this week and if you can get back to me soon enough I will see about including it in that update.

-Julius
Marked as answer by juliusfriedman on 3/8/2015 at 7:44 PM
Mar 9, 2015 at 9:38 AM
Hi,

After your reply, I spoke to my product owner and looked at what I need to accomplish. I have to show an H264 stream on the network in a browser using FlowPlayer. Another requirement is to save an H264 stream so it can be re-streamed on demand later.

At first I thought I would use the RtpDump class for this together with the RtpSink and use a pure .net solution so I would be able to modify it to my needs. After discussing again, it turns out FFMpeg does exactly what's needed.

Again: your project is good at what it does and the description on CodeProject is interesting, but unfortunately it's no fit for my needs.

Kind regards,

Bart
Coordinator
Mar 9, 2015 at 2:26 PM
Not a problem and thanks for the update.

You never really stated what your needs were previously so I was just trying to articulate from a technical perspective.

It's good that FF MPEG can meet your needs I am sure that other libraries do also, I would also remind you that you can definitely use the RtpDump format to save any Rtp stream to disk and then play it back when desired.

It seems like you NEED decoding and unfortunately isn't really implemented in this project for any codec yet, but once decoders are implemented it sounds would you will then have what you need, correct?

Have you seen csCodec ? Are you aware that Silverlight has it's own decoder?

You could also use Media Foundation or still use FFMPEG for just the decoding portion if you needed as the RtspServer is really just a facility to aggregate the stream to many viewers without degrading the connection.

One thing you will notice is that FFMPEG has quite a few memory leaks which only become apparent when using it as the solution, if you can work around them that's great but it may require a more in-dept encapsulation than you currently have especially if you are inter-operating between c#.

Also be cautious when choosing the forward facing server, especially if media is allowed to be pushed to the server.

https://lists.libav.org/pipermail/libav-devel/2015-January/066305.html

My server is not nearly as susceptible to such attacks and with that being said I wish you the best of luck and this library will continue to be here for you in the future if and when you decide you do need it.
Marked as answer by juliusfriedman on 3/9/2015 at 7:26 AM