Incremental memory consumption by RTSPClient

Topics: Question
Aug 22, 2016 at 3:22 AM
Edited Aug 22, 2016 at 3:24 AM
Hi,

I'm using RTSPClient to read RTSP stream and store it into disk, however, the client seems to allocate memory exponentially. I used profiler to see where it was allocating so much memory and it seems RTPFrameChanged event handler is the caveat. I tried to forcefully call GC which results in bad output stream but not much improvement in consumption.

Where do you think it goes wrong?


Please find the event handler below, also I have a queue which is handled by another thread to store output on disk.
Dim mediaType As Media.Sdp.MediaType = If(context Is Nothing, Media.Sdp.MediaType.unknown, context.MediaDescription.MediaType)

        Dim mediaTypeString As String = mediaType.ToString()

        If Media.Common.IDisposedExtensions.IsNullOrDisposed(rtpFrame) OrElse mediaType <> Media.Sdp.MediaType.video Then
            Return
        End If

        If rtpFrame.IsEmpty Then
            'Got a EMTPTY RTP FRAME
        ElseIf rtpFrame.IsMissingPackets Then
            'Got frame with missing packets
        End If

        Dim count As Integer = DataQueue.Count
        If count > 5 Then
            'Do not store more than  packets in the queue
            Return
        End If

        If final AndAlso mediaType = Media.Sdp.MediaType.video Then 'Depacketize and write to corresponding out stream
            Using profileFrame As New Media.Rtsp.Server.MediaTypes.RFC6184Media.RFC6184Frame(rtpFrame)
                profileFrame.Depacketize()
                If Not profileFrame.HasDepacketized Then Return

                Dim buffer() As Byte = profileFrame.Buffer.ToArray(),
                    addedHeader = False

                If False = _initializedStream Then
                    Dim fmtp As New Media.Sdp.Lines.FormatTypeLine(context.MediaDescription.FmtpLine)

                    Dim sps As Byte() = Nothing, pps As Byte() = Nothing

                    'If there was a fmtp line then iterate the parts contained.
                    For Each p As String In fmtp.Parts
                        'Determine where in the string the desired token in.
                        Dim token As String = Media.Common.Extensions.[String].StringExtensions.Substring(p, "sprop-parameter-sets=")

                        'If present extract it.
                        If False = String.IsNullOrWhiteSpace(token) Then
                            'Get the strings which corresponds to the data without the datum split by ','
                            Dim data As String() = token.Split(","c)

                            'If there is any data then assign it

                            If data.Length > 0 Then
                                sps = System.Convert.FromBase64String(data(0))
                            End If

                            If data.Length > 1 Then
                                pps = System.Convert.FromBase64String(data(1))
                            End If

                            'Done
                            Exit For
                        End If
                    Next

                    Dim header As New List(Of Byte)

                    'Prepend the SPS if it was found
                    If sps IsNot Nothing Then
                        'Emulation prevention, present for SPS or PPS
                        header.Add(0)
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(0))
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(1))
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(2))
                        For Each spsb As Byte In sps
                            header.Add(spsb)
                        Next
                    Else
                        Throw New System.Exception("SequenceParameterSet not found")
                    End If

                    'Prepend the PPS if it was found.
                    If pps IsNot Nothing Then
                        'Emulation prevention, present for SPS or PPS
                        header.Add(0)
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(0))
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(1))
                        header.Add(Media.Codecs.Video.H264.NalUnitType.StartCodePrefix(2))
                        For Each ppsb As Byte In pps
                            header.Add(ppsb)
                        Next
                    Else
                        Throw New System.Exception("PicutureParameterSet not found")
                    End If

                    _initializedStream = True
                    _header = header.ToArray()
                    addedHeader = True
                End If


                If False = profileFrame.ContainsSequenceParameterSet OrElse False = profileFrame.ContainsPictureParameterSet Then
                    Dim mergerBuffer(_header.Length + buffer.Length - 1) As Byte
                    Array.Copy(_header, mergerBuffer, _header.Length)
                    Array.Copy(buffer, 0, mergerBuffer, _header.Length, buffer.Length)
                    DataQueue.Enqueue(mergerBuffer)
                    GC.AddMemoryPressure(mergerBuffer.Length + buffer.Length)
                Else
                    DataQueue.Enqueue(buffer)
                    GC.AddMemoryPressure(buffer.Length)
                End If

                buffer = Nothing
                profileFrame.Buffer.Close()
                profileFrame.Buffer.Dispose()
                profileFrame.Clear()
                profileFrame.Dispose()
            End Using

            rtpFrame.Buffer.Close()
            rtpFrame.Buffer.Dispose()
            rtpFrame.Clear()
            rtpFrame.Dispose()
        End If
Coordinator
Aug 22, 2016 at 3:26 AM
Thanks for your interest in the project.

Please post the complete example code including the main method such that I can replicate your issue and either fix the problem or point out something your doing wrong.
Marked as answer by juliusfriedman on 8/21/2016 at 7:26 PM
Aug 22, 2016 at 3:41 AM
Edited Aug 22, 2016 at 3:48 AM
Please find the source code @ https://drive.google.com/open?id=0BzNAuBW5ZIvHMXNPNGxmbjNRVFE Thanks for the quick reply.
Coordinator
Aug 22, 2016 at 11:38 AM
I will take a look when time permits, I'm fairly sure this is something from your code and not mine and I will report back on such as indicated.
Marked as answer by juliusfriedman on 8/22/2016 at 3:38 AM
Aug 22, 2016 at 2:05 PM
Sure, thank you for the help. However, even commenting my code and just running RTSPClient with

RtpFrameChanged = True

has the same effect.
Coordinator
Aug 22, 2016 at 2:14 PM
Sorry, I can't replicate that.

Can you make your camera accessible via the Internet? Send me the address and credentials in an email and I will try again on that specific device.
Marked as answer by juliusfriedman on 8/22/2016 at 6:14 AM
Aug 22, 2016 at 2:51 PM
Please find the stream address in the message I sent you recently.
Coordinator
Aug 22, 2016 at 7:10 PM
Sorry, I cannot replicate your issue with my code alone.

I will attempt to debug your project when time permits.

If I can do anything else for you just let me know.

Sincerely,
Julius
Marked as answer by juliusfriedman on 8/22/2016 at 11:13 AM
Aug 22, 2016 at 7:32 PM
Hi Julius,

Thank you for the reply. Please let me know what you find out.

Regards,

Dipen
Aug 22, 2016 at 8:15 PM
Can you suggest me where can I look for to find out possible problem as profiling suggests problem lies in "OnRTPFrameChanged" event handler specifically in methods:

Media.Rtp.RtpClient.HandleIncomingRtpPacket(object,class Media.Rtp.RtpPacket,class Media.Rtp.RtpClient/TransportContext)
  • Media.Rtp.RtpPacket.Clone(bool,bool,bool,bool,bool,bool)
  • Media.Rtp.RtpClient.OnRtpFrameChanged(class Media.Rtp.RtpFrame,class Media.Rtp.RtpClient/TransportContext,bool)
and

Media.Rtp.RtpClient.ParseAndHandleData(class Media.Common.MemorySegment,bool&,bool&,int32&,int32&)

Thanks,

Dipen
Coordinator
Aug 22, 2016 at 8:26 PM
Yea, here's your advice...

You handle that event, e.g. you provide the code which does something when that event is fired and as a result of your logic which handles that event there is apparently a memory leak.

There's nothing I can see right now in the code which would cause a memory leak in the methods you are referencing except your own code.
Marked as answer by juliusfriedman on 8/22/2016 at 12:26 PM
Aug 22, 2016 at 9:45 PM
Well even if I comment the code and just return from the event handler without doing anything it grows in memory.
Coordinator
Aug 22, 2016 at 9:59 PM
I emailed you back because I can't replicate that and I wanted to give you the chance to show me how...

Anyway if I have time tomorrow maybe I can help further.
Marked as answer by juliusfriedman on 8/22/2016 at 1:59 PM
Coordinator
Aug 23, 2016 at 2:09 PM
Edited Aug 23, 2016 at 2:10 PM
I believe I have replicated your issue; Are you running in DEBUG mode?

If so this is where the incremental memory consumption comes into play....

This is intentional for the time being; I will probably revise such proleptically but as of the current time this is used for debugging and profiling purposes.

Thanks for your interest in the project and if you need anything else please let me know.
Marked as answer by juliusfriedman on 8/23/2016 at 6:09 AM
Aug 23, 2016 at 2:41 PM
Hi Julius,

Thanks for getting back, and I really appreciate your help.
The issue indeed solved, and thank you very much for creating such a great library.

Regards,

Dipen Shah