Consuming RTSP streams

Jun 13, 2014 at 7:03 AM

I'm trying to consume a broadcasted stream directed at the server from OBS (Open Broadcaster Software) which I would like to perform some processing on before pushing out to a media streaming site such as or youtube. The processing is obviously external to this project but I'm having trouble accessing the stream being received. Do I need to create a new type of RtspStream based on the client session? Is this a scenario this library can handle without major work? Any response would be greatly appreciated!

Jun 13, 2014 at 4:23 PM
I don't see why not....

Just make a RtspClient to consume your stream, then take each frame and perform whatever processing you need.

Subsequently when you are done processing do whatever it is with the processed data you like.

You can also make the RtspServer do this e.g you can have a AnalyzedRtspStream which derives from RtspStream then sets ready only after certain processing has been performed, then you can push only the processed frames to the client.

Currently I am working on some .h264 stuff and some MPEG stuff so as soon as I am done with that I will be uploading those changes but they should have nothing to do with what your trying to accomplish unless I am missing something!

Let me know if you need any further information!
Jun 13, 2014 at 8:55 PM
I just updated the code.

The changes should show you the various ways to create a stream either from the disk or from an existing stream.

I am also going to be working on a CachingStream/ArchivedStream which takes the data and puts it to rtpdump format on the disk for playback later.

Something like that might be useful for your scenario because it would allow you to process a Queue and then after you have what you want you can then just Enque it into the m_Frames member from where it will automatically be sent at the correct time.
Jun 13, 2014 at 11:10 PM
Thank you for the quick updates! The issue isn't with processing the stream and doing the work I need to do on it, I'll just publish the stream out to ffmpeg or another library that can process the media, I primarily want to use this as a routing engine to handle multiple streams being pushed to the server from either OBS or nVidias ShadowPlay. This means the server needs to listen for streams, they aren't being requested from a URL they are being pushed to one by a client and although I can get the session up on the server I only seem to be able to push streams to them that have been registered on the server when I need to be able to do something such as the following ->
  1. OBS (Client 1) Starts Screen Sharing
  2. Stream is pushed to RTSP Server rtsp://serverURL:555/Publish/Client1
  3. Server starts archiving stream
  4. We decide to "go live" with the stream and start publishing it.
  5. Process the stream to add in house artifacts such as logo and tickers.
  6. Establish connection to, Youtube, etc and start streaming.
There are a lot of other things we want to do with this solution but this is the most straight forward use case. I'm getting stuck accessing the stream that is being pushed to the rtspServer.
Jun 13, 2014 at 11:14 PM
Pushing to the server would require an implementation of the Announce method.

Why don't you derive the RtspServer and implement a handler for Announce and then you can have clients Announce the stream...

Once the Announcement happens you can then do whatever it is you need to do.

The other way to do this is to have another listener listen for a certain message / packet and when it gets that do some inter-process communication and then tell the server to add a stream with the given url.

Simple as that.

Let me know if you need anything else!
Jun 14, 2014 at 12:45 AM
Aha that's what I got started on last night before I posted the question, I'll get back onto it and implement announce in the project.

If I get it done I'll see what I can do about submitting a patch.
Jun 14, 2014 at 12:55 AM
Cool thanks!

If announce gets implemented it will probably be specific to the implementation though...

The only reason why it's not implemented now is that it serves no purpose at the current moment although I could implement pushing the announce from a source back to the server so any streams which get changed also get changed when the source changes and subsequently also push that change to all clients.

It would be fairly easy to implement such a thing, all that I would have to do is listen to then RtspResponse event from the RtspSource and then watch for Response.Method == Announce and then each Client which has that source would have their SDP updated and pushed out.

If you do implement this it would also make sense to have some type of password protection so the outside world cannot add streams to the server.... either that or check that only IP's on the same LAN can Announce or something like that.

The other way would be as I said to just do inter-process communication and leave the classes as they are which I think makes the most sense in this case.

Just take the data from FFMPEG output and wrap it up in RtpPacket e.g. Packetize it and Queue it so another thread can send it as per the RFC2435 class which does this for JPEG, those latest classes I just added to this and should provide you with a starting point..
Jun 14, 2014 at 12:58 AM
Aaand here's where I got stuck... When I start streaming from OBS to the server the first packet gets rejected by the RtspMessage class, the m_RequestLine isn't anything recognisable - "�\0\0\0\0)#���l֮R�I���볦�<�\f>�$^" is the content of m_RequestLine my first guess is there's an encoding issue but frankly I'm well outside of my usual problem domain at this point so I'm not sure if I'm barking up the wrong tree or not.

I'll go further up the stack and see if I've headed down the wrong rabbit hole.
Jun 14, 2014 at 1:16 AM
Edited Jun 14, 2014 at 1:19 AM
Do you have a server I can test against? That usually indicates some type of framing error... ^ is not a valid channel unless you have a lot of streams :P usually its either /0 or 01 , '02' or '03'

Where in the call stack does that occur from? is the RtpClient in OnInterleavedData or is the RtspClient in SendRtspRequest?

And lastly that shouldn't be problem since there is logic to handle this when it occurs, it usually means the server started sending Rtsp ALF RtpPackets before the response to a request was received.

The logic goes to a label PlayingNoResponse in the SendPlay method on the RtspClient.
Jun 14, 2014 at 1:27 AM
It's not a server, it's the free OBS (Open Broadcast Software) which is commonly used to stream to or, in the broadcaster settings I use rtsp://localhost:554/publish/client1 (or whatever URL) you don't even need to add a source to the application you can just broadcast a black screen to test with, and then click start steam outside to initiate a connection to the server and start sending data.

This occurrs in the ProcessRecieve method of the RtspServer...

Idiot mode is fully enabled here, am I back to needing a new RtspClient implementation? they all seem to generate requests which isn't what I want to do here.
Jun 14, 2014 at 1:31 AM
Edited Jun 14, 2014 at 1:35 AM
Something tells me you are sending the wrong kind of data to the RtspServer...

You would start the OBS software...

Then in the RtspServer class add a source stream to 'rtsp://localhost:554/publish/client1' you can verify this works by making a RtspClient to test the stream which is all the server does anyway (use a RtspClient) to consume the stream and then aggregate packets on demand....

I am sure that it will do what you want... how are you sending the data currently?

I have tested with many servers and with VLC and there is no problem as you describe.. even with all the different dialects I have encountered... just a few labels were needed for corner cases...

Also I am not aware of any other RtspClient implementation which is going to do this for you, hence why I wrote this library...

It seems you just need to derive a class from the RtspSourceStream and then when packets arrive instead of pushing them out process them and then push them out...
Jun 14, 2014 at 1:55 AM
Edited Jun 14, 2014 at 2:30 AM
That sounds about right... The solution in net7mma as it stands is for connecting to a known streaming server and initiating, receiving and re-publishing the stream to multiple clients. I want to do something slightly different. I want to listen for incoming streams that I don't know about at compile time.

When I add a source stream with that URL the server attempts to create an outbound connection to that URL, which isn't the intention on my end. There is only one server here, and that is the RtspServer object, for the purposes of my current exploratory work it doesn't need to connect out to anything it just needs to sit and listen and when a stream arrives I want to be able to connect another client and start receiving that stream. Once I know this works then I'll start work on getting the stream pushed out to and youtube live.

I'm sending the data from OBS, the solution I'm trying to create is a stream collection point for streams from 25 gaming PC's in a gaming center. The server should be sat listening on a port, and when a new stream arrives I want to be able to apply logic so I can route a given stream out for processing and back in for further distribution to various streaming services such as and youtube live.

I don't want to connect to a server and recieve a stream from it, I want the rtsp server to be open to accept a stream from any of the internal machines and from that point we can choose which streams to publish out and which to send for additional processing such as compositing the streams into arrangements so we can show the viewpoints of four players in a team game as one example. The processing elements and compositing we can do, and I'm 70% certain I could do this with the rtmp module I've got compiled for nginx but I'm more familiar with .Net and this library looks to have all of the implementation I should need to start doing this kind of routing.

Thank you for your help so far though, I'm sure your library has 99% of what I need regarding RTSP/RTMP implementation and it's very nicely written, either I'm completely missing something in the project or I'm trying to do something you didn't intend.


The above image is the initial architecture for my solution, the clients are gaming PC's in an internet cafe (there are 25 currently) and they will be directing a stream to the server whenever they are playing a game they wish to stream out to one of the available streaming services. If a team is playing a game then we want to compile their streams into a grid or another arrangement before pushing the stream out to their streaming site of choice. I know this isn't the intended use for your library but I feel it is the closest starting point for the routing engine in my solution.

I don't have the option of providing an end point on each of the clients for the server to connect to as none of the game streaming software available provides this feature, they all "push" streams to a listening RTSP server.
Jun 14, 2014 at 2:59 AM
I think I understand the library a bit better now, I believe I'll need something like an RtspListenClient type that will accept a socket to listen on instead of attempting to connect out.
Jun 14, 2014 at 3:05 AM
Edited Jun 14, 2014 at 3:07 AM
I am not even sure why you need the RtspServer in that case..

You can just use RtspClient and then for each stream you wanna add make a new one and do something with the packets... or the frames

e.g. archive them or do processing.

Then if you want to re-stream them to twitch or YouTube you can do that in the same encapsulation where you are doing the processing etc.

Having a RtspServer doesn't provide you with much here because no one will be consuming your stream with Rtsp?

Also this library only supports Rtp right now, sending Rtmp is not supported although it could happen because the RtpPacket doesn't really verify the data therein...
Jun 14, 2014 at 3:19 AM
In this case the server in its current form doesn't really fit, however there will be local clients connecting to request copies of the stream for display around the store and the server here will fit without much modification.

I'm assuming there will need to be a server of some sort to listen for inbound streams and create a new RtspClient to handle the incoming stream data.
Jun 14, 2014 at 3:41 AM
Edited Jun 14, 2014 at 3:45 AM
Yes that will be the "Server" it just doesn't need to be a RtspServer.

You could probably also achieve that with a timer which checks for new streams and then adds them when necessary.

Then to re-stream the sources... you can use the RtspServer, you can make a VirtualRtspSource derived from RtspSource and then have that virtual source either check a queue or listen for packets somewhere else and then send them out to anyone who connects to the stream via the RtspServer.

Just call OnRtpPacketReceieved on the underlying RtpClient on the VirtualRtspSource.

As far as publishing you can then use the RtspClient to send an Announce to Youtube or whoever and tell them to start consuming the VirtualRtspSource... if it works like that...

Announce can be sent by the RtspClient however currently the RtspServer just doesn't do anything when it receives one.

Hope that helps!
Jun 14, 2014 at 3:41 PM
And looking at this again you really don't even need a VirtualRtspSource, you basically just need to setup the stream when your OBS is ready, the rest is already done and will occur for you when you need it to.

Your code only needs to start and stop the stream in the RtspServer for re-streaming when you are ready for it to do so.

Additionally I just updated the code again with some methods and classes which might help you, e.g. I clearly defined Sink and Source and added methods to send raw data, albeit at the current moment it just wraps the data in a RtpPacket instance, I will be changing this in the future to have a TransportManager class etc which should allow the implementation of Rtmp etc..

If you need anything else let me know!
Marked as answer by juliusfriedman on 9/8/2014 at 7:35 AM