RtspServer issue with Azure

Topics: Bug Archive
Apr 27, 2015 at 7:37 AM
Hi Julius,

First of all I would like to thank you for the great library. I've been playing with this library since 111192 version till 111212. What my team wants to do is to create media server which can receive video feed from android then view it on the desktop. I cannot use mono since we decided to use native development for many reasons. So my setup would be:
  • Android send video (mjpeg or h264 format) thru whatever transport we have (tcp, http, etc) to a media receiver. We have time constraint here and got no time to implement the RFC's on our own.
  • The media receiver will then communicate using RTSP protocol to the media server. In this part we'll need to packetize the media (correct me if I'm wrong).
  • The media server needs to be able to feed the live packets as well as stored packets when required. I also need to be able to add stream source on the fly. I'm not sure whether I need another application server to manage the live streams and stored stream.
  • The client. nothing to talk about this one. I've test your RtspClient and most of the time It serves its purpose well even when VLC doesn't.
So basically your library provide most of our needs. Any suggestion on the implementation is welcomed.

My team members are geographically separated. So we spawn an azure instance to test our application servers (we build client/server applications prior to this media service). I created an Rtsp server to test since we wanted to start with the working one first. In the first few runs of the app, exception occurs:
  • Starting Stream: Alpha Id=517f9a94-558e-42c4-9590-bffc267e1f17
    Thread is dead; priority cannot be accessed.
    at System.Threading.Thread.GetPriorityNative()
    at Media.Rtsp.RtspServer.StartStreams() in \net7mma-111212\RtspServer\RtspServer.cs:line 1106
    It only happen in the azure instance. The workaround currently is by setting the priority to below normal in the StartStream method and MaintainStream method because it happens there also. I got the BelowNormal value while debugging the code.
    Let me know if you have a correct solution since it doesn't happen when I only have bandit as stream source. Fix this and you will make your library cloud friendly for people like me :)
Another issue I found is on the Media.Rtsp.Server.MediaTypes.RFC2435Media but this one is nothing to do with the azure. Since the issue is rather minor so I post it together here. If I only have bandit as the source, the SendPackets method will not run correctly since it runs before the server starts due to idle cores. It executes before the stream starts. So the workaround is in the Start method to move up the
base.Start();
before the
            //Make the thread
            m_RtpClient.m_WorkerThread = new System.Threading.Thread(SendPackets);
so the whole code for Start method is:
        //SourceStream Implementation
        public override void Start()
        {
            if (m_RtpClient != null) return;

            //Create a RtpClient so events can be sourced from the Server to many clients without this Client knowing about all participants
            //If this class was used to send directly to one person it would be setup with the recievers address
            m_RtpClient = new Rtp.RtpClient();

            SessionDescription = new Sdp.SessionDescription(0, "v√ƒ", Name);
            SessionDescription.Add(new Sdp.Lines.SessionConnectionLine()
            {
                ConnectionNetworkType = Sdp.Lines.SessionConnectionLine.InConnectionToken,
                ConnectionAddressType = Sdp.SessionDescription.WildcardString,
                ConnectionAddress = System.Net.IPAddress.Any.ToString()
            });

            //Add a MediaDescription to our Sdp on any available port for RTP/AVP Transport using the RtpJpegPayloadType            
            SessionDescription.Add(new Sdp.MediaDescription(Sdp.MediaType.video, 0, Rtp.RtpClient.RtpAvpProfileIdentifier, RFC2435Media.RFC2435Frame.RtpJpegPayloadType));

            //Indicate control to each media description contained
            SessionDescription.Add(new Sdp.SessionDescriptionLine("a=control:*"));

            //Ensure the session members know they can only receive
            SessionDescription.Add(new Sdp.SessionDescriptionLine("a=sendonly")); //recvonly?
            
            //that this a broadcast.
            SessionDescription.Add(new Sdp.SessionDescriptionLine("a=type:broadcast"));
            

            //Add a Interleave (We are not sending Rtcp Packets becaues the Server is doing that) We would use that if we wanted to use this ImageSteam without the server.            
            //See the notes about having a Dictionary to support various tracks
            m_RtpClient.TryAddContext(new Rtp.RtpClient.TransportContext(0, 1, sourceId, SessionDescription.MediaDescriptions.First(), false, 0));

            //Add the control line
            SessionDescription.MediaDescriptions.First().Add(new Sdp.SessionDescriptionLine("a=control:trackID=1"));

            //Add the line with the clock rate in ms, obtained by TimeSpan.TicksPerMillisecond * clockRate            

            base.Start();

            //Make the thread
            m_RtpClient.m_WorkerThread = new System.Threading.Thread(SendPackets);
            m_RtpClient.m_WorkerThread.TrySetApartmentState(System.Threading.ApartmentState.MTA);
            //m_RtpClient.m_WorkerThread.IsBackground = true;
            //m_RtpClient.m_WorkerThread.Priority = System.Threading.ThreadPriority.BelowNormal;
            m_RtpClient.m_WorkerThread.Name = "SourceStream-" + Id;

            //If we are watching and there are already files in the directory then add them to the Queue
            if (m_Watcher != null && !string.IsNullOrWhiteSpace(base.Source.LocalPath) && System.IO.Directory.Exists(base.Source.LocalPath))
            {
                foreach (string file in System.IO.Directory.GetFiles(base.Source.LocalPath))
                {

                    if (false == SupportedImageFormats.Any(ext => file.EndsWith(ext, StringComparison.OrdinalIgnoreCase))) continue;

                    try
                    {
#if DEBUG
                        System.Diagnostics.Debug.WriteLine("ImageStream" + Id + " Encoding: " + file);
#endif
                        //Packetize the Image adding the resulting Frame to the Queue (Encoded implicitly with operator)
                        using (var image = System.Drawing.Image.FromFile(file)) Packetize(image);
#if DEBUG
                        System.Diagnostics.Debug.WriteLine("ImageStream" + Id + " Done Encoding: " + file);
#endif
                    }
                    catch (Exception ex)
                    {
                        throw ex;
                    }
                }

                //If we have not been stopped already
                if (/*State != StreamState.Started && */ m_RtpClient.m_WorkerThread != null)
                {
                    //Only ready after all pictures are in the queue
                    Ready = true;
                    m_RtpClient.m_WorkerThread.Start();                    
                }
#if DEBUG
                System.Diagnostics.Debug.WriteLine("ImageStream" + Id + " Started");
#endif
            }
            else
            {
                //We are ready
                Ready = true;
                m_RtpClient.m_WorkerThread.Start();
            }
        }
Look forward to your response.

Regards,

Fatah
Coordinator
Apr 30, 2015 at 12:44 AM
Not really sure what your question is.

Thanks for the interest in the library.
Marked as answer by juliusfriedman on 4/29/2015 at 5:44 PM
Apr 30, 2015 at 1:23 AM
In order to get around with the issue on the azure, in the RtspServer.StartStreams and MaintainServer I change the code from:
new Thread(stream.Start)
                    {
                        Priority = m_ServerThread.Priority
                    }.Start();
to
new Thread(stream.Start)
                    {
                        Priority = ThreadPriority.BelowNormal // or ThreadPriority.AboveNormal
                    }.Start();
is that ok? Would it change the behaviour of the server? From debugging the code I actually get ThreadPriority.BelowNormal for both method.

and for the RFC2435Media, I only suggest a fix. I don't know whether you have a better solution for the problem.

Regards,
Coordinator
Apr 30, 2015 at 1:40 AM
okay?

...

As in yes I understand, a general agreement about something?

What exactly?

Changing the priority of a thread?

Sorry but I honestly am not sure how I can help you.

Thanks for your interest in the library.
Marked as answer by juliusfriedman on 4/29/2015 at 6:40 PM
Apr 30, 2015 at 2:07 AM
yes. Changing the thread priority with value without querying the thread priority.

Calling
Priority = m_ServerThread.Priority
from my understanding will query the thread priority of m_ServerThread (by calling System.Threading.Thread.GetPriorityNative() method) which will cause the exception in the azure but I'm not sure why because it doesn't happen on my local pc.
If I change the priority with a direct value such as ThreadPriority.BelowNormal the exception never occur.

The fix I do is just a matter of workaround solution. I don't know if it will affect your server. From experience I learn that we have to design threads correctly, avoid racing conditions etc. etc. So I'm not sure whether it will ruin your design. Basically what I want to have is an rtsp server that can run on normal pc and the cloud. So I do hope you can kindly evaluate and do something with the issue.
Coordinator
Apr 30, 2015 at 2:22 AM
Thanks for the information. Glad the software was able to help you.
....

Why do you ask questions to which you apparently already know the answer?

With that being said I sincerely appreciate your interest in the library.

Please don't forget to adhere to the Apache License.

If you have any tangible issues or questions which are related to my code or design please let me know.

Thanks
Marked as answer by juliusfriedman on 4/29/2015 at 7:22 PM
Apr 30, 2015 at 4:26 AM
Thanks for the prompt response. I asked it because there's still more related to threading. Should I post it here or make another thread because it happens also on the local pc?
Coordinator
Apr 30, 2015 at 4:30 AM
I'm sorry but I can't really provide free consulting or training for your application.

I hope you understand and thanks for your interest.
Marked as answer by juliusfriedman on 4/29/2015 at 9:30 PM
Apr 30, 2015 at 4:47 AM
Edited Apr 30, 2015 at 4:48 AM
I understand :) but I'm reporting problem in your library here and not consulting my application.

I see this project to be potential. That's why I'd like to help when I could.
My interest is actually on the client/server communication currently. I've test the client locally and remotely to wowza and it works as expected. The server however have issue on the handshake part.

I evaluate the handshake using your client. These are the issues I found on the server:
  1. The server sometimes doesn't respond to the requests.
  2. When it's good, the handshake can only get to the SETUP part. Here's the handshake steps:
    Client ---------------------------- Server
    *. request OPTIONS --------------->
    *. <-------------- respond OPTIONS
    *. request DESCRIBE --------------->
    *. <-------------- respond DESCRIBE
    *. request SETUP ------------------->
    *. <------------------ respond SETUP
    *. request PLAY --------------------> //sent to server but cannot make it to RtspServer.ProcessRtspRequest. So no response here and no RTP data sent.
    *. request GET_PARAMETER ------> //the keep alive sent correctly in timely manner
    *. <----- respond GET_PARAMETER
I can proceed to the PLAY by disconnecting the client by calling:
            rtspClient.StopPlaying();
            rtspClient.Disconnect();
then reconnect to the server.
Since the m_ReceivedMessages != 0 it doesn't send the OPTIONS. And since the SessionDescription is not null it doesn't send the DESCRIBE either. So the handshake start from setup:
Client ---------------------------- Server
  • request SETUP ------------------->
  • <------------------ respond SETUP
  • request PLAY --------------------> //this time can make it to RtspServer.ProcessRtspRequest.
  • <------------------- respond PLAY //and RTP packet sent correctly.
VLC will restart the sequence from OPTIONS but maybe you purposely design the client differently.

For issue no.1 I'm quite sure it has something to do with the threading.
For issue no. 2 maybe a mix with the threading and something else.

I hope you can spot them and fix them in the next release. I understand fixing threading problem can takes time.

The tests were done locally. I created a server (console application) with only bandit as a stream source and a WPF client to view the images sent by the server.

Thanks
Apr 30, 2015 at 5:01 AM
oops sorry. I tot we're in the bug section. my bad. you can ignore the post.
Coordinator
Apr 30, 2015 at 5:21 AM
In the latest stable release what?

Furthermore
Please make a unit test which replicates your bug otherwise I doubt I will be able to address further.
Marked as answer by juliusfriedman on 4/29/2015 at 10:21 PM
Apr 30, 2015 at 6:21 AM
I use the 111212 version.

Here's a console app as unit test. Once again I use bandit as stream source:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Media.Rtsp;
using Media.Rtsp.Server;

namespace RtspClientUnitTest
{
    class Program
    {
        static string executingAssemblyLocation = System.Reflection.Assembly.GetExecutingAssembly().Location;
        static void Main(string[] args)
        {
            RtspServer rtspServer = new RtspServer(System.Net.IPAddress.Any, 1554);
            rtspServer.Logger = new RtspServerConsoleLogger();
            string localPath = System.IO.Path.GetDirectoryName(executingAssemblyLocation);
            Media.Rtsp.Server.MediaTypes.RFC2435Media sampleStream = null;
            rtspServer.TryAddMedia(sampleStream = new Media.Rtsp.Server.MediaTypes.RFC2435Media("Bandit", localPath + "\\Media\\Bandit\\") { Loop = true });
            rtspServer.Start();
            //Wait for the server to start.
            while (false == rtspServer.IsRunning) System.Threading.Thread.Sleep(0);

            RtspClient rtspClient = new RtspClient("rtsp://127.0.0.1:1554/live/bandit", RtspClient.ClientProtocolType.Tcp, 1500);     //worked
            rtspClient.Logger = new Media.Common.Loggers.ConsoleLogger();
            rtspClient.OnResponse += rtspClient_OnResponse;
            rtspClient.StartPlaying();
            rtspClient.StopPlaying();

            Console.WriteLine();
            Console.WriteLine("Press any key to close the console.");
            Console.ReadKey();
            Environment.Exit(0);
        }

        static void rtspClient_OnResponse(RtspClient sender, RtspMessage request, RtspMessage response)
        {
            Console.WriteLine(response.ToString());
        }
    }
}
Here's the output:

Server Started @ 4/30/2015 6:18:43 AM
Starting Stream: Bandit Id=f9fc1a22-3908-4537-bc69-ed941e8b9ee9
Adding Client: d48368c4-fb1b-48f1-91b3-b7ad445e0dab
Accepted Client: d48368c4-fb1b-48f1-91b3-b7ad445e0dab @ 4/30/2015 6:18:43 AM Add
ed =True
Request=> OPTIONS rtsp://127.0.0.1:1554/live/bandit RTSP/1.0
CSeq: 1

Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

Response=> RTSP/1.0 200 OK
CSeq: 1
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT

Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

RTSP/1.0 200 OK
CSeq: 1
Public: OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE, TEARDOWN, GET_PARAMETER
Allow: ANNOUNCE, RECORD, SET_PARAMETER
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT


Request=> DESCRIBE rtsp://127.0.0.1:1554/live/bandit RTSP/1.0
Accept: application/sdp
CSeq: 2

Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

Response=> RTSP/1.0 200 OK
CSeq: 2
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://127.0.0.1/live/f9fc1a22-3908-4537-bc69-ed941e8b9ee9/
Content-Length: 230
Content-Encoding: utf-8
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT

v=0
o=ASTI-Media-Server 15630947311917684198 -2815796761791867414 IN IP4 127.0.0.1
s=ASTI-Streaming-Session-Bandit
a=sendonly
a=type:broadcast
m=video 0 RTP/AVP 26
a=control:/live/f9fc1a22-3908-4537-bc69-ed941e8b9ee9/video
Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

RTSP/1.0 200 OK
CSeq: 2
Content-Type: application/sdp
Cache-Control: no-cache
Content-Base: rtsp://127.0.0.1/live/f9fc1a22-3908-4537-bc69-ed941e8b9ee9/
Content-Length: 230
Content-Encoding: utf-8
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT

v=0
o=ASTI-Media-Server 15630947311917684198 -2815796761791867414 IN IP4 127.0.0.1
s=ASTI-Streaming-Session-Bandit
a=sendonly
a=type:broadcast
m=video 0 RTP/AVP 26
a=control:/live/f9fc1a22-3908-4537-bc69-ed941e8b9ee9/video

Request=> SETUP rtsp://127.0.0.1:1554/live/bandit//live/f9fc1a22-3908-4537-bc69-
ed941e8b9ee9/video RTSP/1.0
Transport: RTP/AVP/TCP;unicast;interleaved=0-1
CSeq: 3

Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

Response=> RTSP/1.0 200 OK
CSeq: 3
Transport: RTP/AVP/TCP;source=127.0.0.1;interleaved=0-1;ssrc=A4D936C1
Session: 962273462;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT

Session=> d48368c4-fb1b-48f1-91b3-b7ad445e0dab

RTSP/1.0 200 OK
CSeq: 3
Transport: RTP/AVP/TCP;source=127.0.0.1;interleaved=0-1;ssrc=A4D936C1
Session: 962273462;timeout=30
Server: ASTI Media Server RTSP 1.0
Date: Thu, 30 Apr 2015 06:18:43 GMT


21c5fb39-47bc-4614-929f-0fea04b6d152 HandleIncomingRtcpPacket - No Context for p
acket -286614676@200

21c5fb39-47bc-4614-929f-0fea04b6d152 HandleIncomingRtcpPacket - No Context for p
acket -286614676@202 Media.Rtsp.RtspClient-2a38890a-d67e-43e4-a4f3-07a25ffddc82@SendRtspMessage: The
operation completed successfully
Media.Rtsp.RtspClient-2a38890a-d67e-43e4-a4f3-07a25ffddc82@SendRtspMessage: Cann
ot access a disposed object.
Object name: 'System.Net.Sockets.Socket'.
Connected Clients:1
Server Stopped @ 4/30/2015 6:18:56 AM
Uptime:00:00:12.4100173
Sent:924
Receieved:366
Disposing Client: d48368c4-fb1b-48f1-91b3-b7ad445e0dab @ 4/30/2015 6:18:56 AM

Press any key to close the console.
Coordinator
Apr 30, 2015 at 6:24 AM
Dude, what's your question?

Where's my car?

Do your own homework.

Goodnight
Marked as answer by juliusfriedman on 4/29/2015 at 11:24 PM
Apr 30, 2015 at 10:26 AM
juliusfriedman wrote:
Dude, what's your question?

Where's my car?

Do your own homework.

Goodnight
Sorry if I offended did you in any way. I think we fail to communicate here. I just trying to help by reporting problems I found.
Let me sum it up:
  1. I thank you for the library.
  2. I found a problem in the RtspServer StartStream method and MonitorServer method. Suggest the fix by setting the thread priority immediately without querying it.
  3. I found a problem in your RFC2435Media Start method and propose the fix.
  4. You're glad for my interest in the library.
  5. I report a problem in RtspServer's handshake whereby it only reach to SETUP part.
  6. You ask for the version and enquire me to make a unit test which replicate the bug.
    juliusfriedman wrote:
In the latest stable release what?

Furthermore
Please make a unit test which replicates your bug otherwise I doubt I will be able to address further.
  1. I answer by saying I use version 111212 of your library and code a simple unit test to prove it, together with the output result I get.
so
Dude, what's your question?
none. I'm reporting here.
Do your own homework.
I think I did yours.
Goodnight
well... I live in another side of the world where it's day time currently and English is not my native language or even the second language. So you can respond me in your day time. No stress here.

one more thing:
As I wrote the unit test I realize that RtspServer.Stop method has logic error.
 if (IsRunning) return;
while it should be
 if (!IsRunning) return;
my two cents.

As a fellow developer I just want to help. I cannot commit to your project but I can report problems and perhaps propose a fix when I could by posting them in your codeplex project.
If you don't like it then you can close this thread. I won't bug you anymore nor will I post another thread.

Regards.
Coordinator
Apr 30, 2015 at 1:05 PM
Edited Apr 30, 2015 at 1:09 PM
... I think you have bigger problems than communication but I definitely don't have time to argue about it.

This definitely isn't homework to me and for someone who wants help I surely disagree the way your going about it.

Thanks Again for your interest.
Marked as answer by juliusfriedman on 4/30/2015 at 6:05 AM
Apr 30, 2015 at 2:05 PM
juliusfriedman wrote:
... I think you have bigger problems than communication but I definitely don't have time to argue about it.

This definitely isn't homework to me and for someone who wants help I surely disagree the way your going about it.

Thanks Again for your interest.
Well... I taught you're different than Ross :) or maybe I did it wrongly so you misunderstood of the intention.

Thanks for your time. Close the thread already.

Good night.
Marked as answer by juliusfriedman on 4/30/2015 at 12:40 PM
Coordinator
May 5, 2015 at 11:57 PM
I dunno what you taught, nor what you thought.

Almost anything is possible in any circumstance(in general) and nothing is misunderstood on my part. (I do believe)

Thanks again for your interest, regardless of the intention.
Marked as answer by juliusfriedman on 5/5/2015 at 4:57 PM
Coordinator
Jun 2, 2016 at 11:39 AM
The issues you cited were on shutdown of the server in certain conditions.

This was resolved by removing priorty checking from the server thread to stop the stream.

If there was anything else which you needed please do let me know.