RFC2435Frame - "This Frame not Complete"

Topics: Bug Archive
Mar 4, 2016 at 12:36 PM
Hello,

I just checked latest version, IsMissingPackets is false and IsComplate is true but when try to save image code exits with "This Frame not Complete" exception.

This code works at old versions.
if (!frame.IsMissingPackets && frame.IsComplete)
            {
                i1++;

                var str = new StringBuilder(30);
                str.AppendFormat("{0} OK: {1}", DateTime.Now.ToString("dd.MM.yyyy hh:mm:ss"), i1);

                Console.WriteLine(str);

                // SAVE
                try
                {
                    RFC2435Media.RFC2435Frame t = new RFC2435Media.RFC2435Frame(frame);

                    using (System.Drawing.Image jpeg = t)
                    {
                        jpeg.Save(@"C:\imgs\result" + i1 + ".jpg", System.Drawing.Imaging.ImageFormat.Jpeg);
                      

                    }
                }
                catch (Exception ex)
                {

                    Console.WriteLine(ex.Message);
                }

            }
            else
            {
                ii1++;

                Console.WriteLine(DateTime.Now.ToString("dd.MM.yyyy hh:mm:ss") + " ERR: " + ii1);
            }
            
       
        }
Coordinator
Mar 4, 2016 at 2:51 PM
Edited Mar 4, 2016 at 2:56 PM
Thanks for reporting this, I am having a hard time replicating this.

If 'IsComplete' is true then you would have all packets including the Marker packet...
    public virtual bool IsComplete {
        get { return false == IsDisposed && false == IsEmpty && HasMarker && false == IsMissingPackets; }
    }
The RFC2435Frame class would not throw and error there unless there was RFC2035 packets found by examining the Type field of the packet.
                if (false == allowLegacyPackets && 
                    Type >= 2 && Type <= 5)
                {
                    //Should allow for 2035 decoding seperately
                    throw new ArgumentException("Type numbers 2-5 are reserved and SHOULD NOT be used.  Applications based on RFC 2035 should be updated to indicate the presence of restart markers with type 64 or 65 and the Restart Marker header.");
                }

If you would simply like to use incomplete frames you can force the decoding with
  frame.PrepareBuffer(false, true);
If you want to always use the RFC Tables and allow incomplete frames you would use
  frame.PrepareBuffer(false, true, true);
And finally if you wanted to allow legacy packets, incomplete frames and to use the standard tables you would use
  frame.PrepareBuffer(true, true, true);
I am at a loss as to how you can get that exception when you said the IsComplete is true....

I thought about changing the logic of IsMissingPackets to
                    //If the packet sequence number is equal to highest sequence number, the result could depend on the marker.
                    if (pSeq == m_HighestSequenceNumber) return true; // a.Marker;
But this is not correct, the packet having a marker has nothing to do with missing packets except if you think of future packets...

The name of the property may be confusing because the frame is technically missing the marker packet, but the summary is clear
     /// <summary>
    /// Indicates if all contained packets are sequential up the Highest Sequence Number contained in the RtpFrame.
    /// (Must have 1 packet with marker or more than 1 packet)
    /// </summary>

That's why HasMarker is a separate property.

Could you please put together a small unit test which shows exactly what this bug is or is related to?

Thanks!
Marked as answer by juliusfriedman on 3/4/2016 at 6:51 AM
Coordinator
Mar 4, 2016 at 5:48 PM
Edited Mar 4, 2016 at 6:00 PM
I don't have many tests for Add and I think that may be the culprit but it's hard to say without a test or a capture.

I made a unit test which I think shows a possible error in the Add logic.
if (diff > count || seq > m_HighestSequenceNumber)
Is true when m_HighestSequenceNumber is 0.

This occurs when the first packet in the frame has a sequence number of 0 AND is the marker packet...

I modified the unit test to illustrate this by modifying TestRemovingPackets to also test Adding packets and renamed it appropriately.

'
    public void TestAddingAndRemovingPackets()
    {
        //Create a frame
        using (Media.Rtp.RtpFrame frame = new Media.Rtp.RtpFrame(0))
        {

            //Add marker packet with seq = 0
            frame.Add(new Media.Rtp.RtpPacket(2, false, false, Media.Common.MemorySegment.EmptyBytes)
               {
                   SequenceNumber = ushort.MinValue,
                   Marker = true
               });


            //Add a lower order packet which MAY belong to the frame, 65535 could be a previous packet or a large jump, it would depend on the timestamp.
            frame.Add(new Media.Rtp.RtpPacket(2, false, false, Media.Common.MemorySegment.EmptyBytes)
            {
                SequenceNumber = ushort.MaxValue
            });

            //Add a higher order packet which does not belong to the frame because the marker packet was set on packet 0, the timestamp should also be different.
            try
            {
                frame.Add(new Media.Rtp.RtpPacket(2, false, false, Media.Common.MemorySegment.EmptyBytes)
                {
                    SequenceNumber = 1
                });

                throw new Exception("Should not be allowed to add packet");
            }
            catch(InvalidOperationException)
            {
                //Expected
            }

            if (frame.HighestSequenceNumber != ushort.MinValue) throw new Exception("Unexpected HighestSequenceNumber");

            if (frame.LowestSequenceNumber != ushort.MaxValue) throw new Exception("Unexpected HighestSequenceNumber");

            //Remove a non existing packet
            using (Media.Rtp.RtpPacket packet = frame.Remove(1))
            {
                if (packet != null) throw new Exception("Packet is not null");
            }

            //Remove two existing packets
            using (Media.Rtp.RtpPacket packet = frame.Remove(ushort.MaxValue))
            {
                if(packet == null) throw new Exception("Packet is null");
            }

            using (Media.Rtp.RtpPacket packet = frame.Remove(ushort.MinValue))
            {
                if (packet == null) throw new Exception("Packet is null");
            }

            if (false == frame.IsEmpty) throw new Exception("Frame is not empty");
        }
    }
'

I have fixed this in my latest code and will update shortly.

111923 should fix this, Please let me know if this is not what your expecting and thanks for bringing this up!
Marked as answer by juliusfriedman on 3/4/2016 at 9:48 AM
Coordinator
Mar 4, 2016 at 9:01 PM
Edited Mar 4, 2016 at 9:28 PM
Hey, I am getting ready to call it a weekend soon and I just wanted to see if we are on the same page.

After some thinking... I made some additional changes in 111925 in Add of the RtpFrame which makes the logic there more consistent with how it has been but I have modified the unit test for the scenario above.

This time there are 2 packets added after the marker and you will quickly see that this can continue in perpetuity and get very ugly when there is discontinuity, e.g. 65535, 65533, 65534, 65531, 65530, 65532...

Especially if there happens to be another marker encountered...

I would like to see what you have to say first about the current changes before proceeding further, e.g. is this what you are trying to describe above?

How would you like to see it be handled and can you provide a supporting use case?

E.g I can see how I may handle something like this without messing with the logic in the RtpFrame class at all, if the RtpPacket was provided via an event from the RtpClient then the SequenceNumber has already been verified to be within a permissible gap it could probably just be added without looking for the Marker at all. e.g. calling RtpFrame.Add(packet, false) from the RtpClient when the event is raised.

It could also be handled with a few more checks in the code, but I have to understand what exactly we are trying to achieve here... or what the Bug is.



Thanks!
Marked as answer by juliusfriedman on 3/4/2016 at 1:02 PM
Mar 7, 2016 at 9:24 AM
Hello,

First of all thx. for your kind work and explanations.

Unfortunately version 111925 gives me same results.

My aim is to save all the streams from rtsp source to disk or ram for further image processing. For example if cam streams at 15 fps, should be able to save about15 complete frames.

I am using cams from Samsung and Sony, try with both.

Your explanation is little complex for me but try to debug further and with different combination of frame.PrepareBuffer(false, true) functions.

I see that most of frames returns with IsComplate false. Try with UDP and TCP, both returns same results.
1. TCP RTSP START
------------------------------
07.03.2016 10:31:59 ERR: 1 frame.IsMissingPackets :True frame.IsComplete : False

07.03.2016 10:31:59 ERR: 2 frame.IsMissingPackets :True frame.IsComplete : False

07.03.2016 10:31:59 ERR: 3 frame.IsMissingPackets :False frame.IsComplete : Fals
e
07.03.2016 10:31:59 ERR: 4 frame.IsMissingPackets :True frame.IsComplete : False

07.03.2016 10:31:59 ERR: 5 frame.IsMissingPackets :True frame.IsComplete : False

07.03.2016 10:31:59 ERR: 6 frame.IsMissingPackets :True frame.IsComplete : False

07.03.2016 10:31:59 ERR: 7 frame.IsMissingPackets :False frame.IsComplete : Fals
e
07.03.2016 10:31:59 ERR: 8 frame.IsMissingPackets :False frame.IsComplete : Fals
e
07.03.2016 10:31:59 ERR: 9 frame.IsMissingPackets :False frame.IsComplete : Fals
e
07.03.2016 10:31:59 ERR: 10 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:31:59 ERR: 11 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:31:59 OK: 1 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:03 ERR: 12 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:03 ERR: 13 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:03 ERR: 14 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:03 ERR: 15 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:03 ERR: 16 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:03 ERR: 17 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:03 ERR: 18 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:03 OK: 2 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:13 OK: 3 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:14 ERR: 19 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:14 OK: 4 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:15 OK: 5 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:21 OK: 6 frame.IsMissingPackets : False frame.IsComplete : True

07.03.2016 10:32:21 ERR: 20 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:22 OK: 7 frame.IsMissingPackets : False frame.IsComplete : True

This Frame not Complete
07.03.2016 10:32:23 ERR: 21 frame.IsMissingPackets :True frame.IsComplete : Fals
e
This Frame not Complete
07.03.2016 10:32:23 ERR: 22 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 24 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 23 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 26 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 25 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 27 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 28 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 29 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 30 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 32 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 31 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 34 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 35 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 36 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 33 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 37 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 38 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 39 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 40 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 41 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 42 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:23 ERR: 43 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:24 ERR: 44 frame.IsMissingPackets :True frame.IsComplete : Fals
e
07.03.2016 10:32:24 ERR: 45 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:24 ERR: 46 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:24 ERR: 47 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:24 ERR: 48 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:24 ERR: 49 frame.IsMissingPackets :False frame.IsComplete : Fal
se
07.03.2016 10:32:24 OK: 8 frame.IsMissingPackets : False frame.IsComplete : True

Debug further i see that IsComplete is false at PrepareBuffer function but at Client_RtpFrameChanged event IsCompelete is true, so code throws "This Frame not Complate" ex.
 public virtual void  PrepareBuffer(bool allowLegacyPackets = false, bool allowIncomplete = false, bool useRfcQuantizer = false)
            {

                if (IsEmpty) throw new ArgumentException("This Frame IsEmpty. (Contains no packets)");
                else if (false == allowIncomplete && false == IsComplete) throw new ArgumentException("This Frame not Complete");
            
Finally same stack trace that may help
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.PrepareBuffer(Boolean allowLegacyPackets, Boolean allowIncomplete, Boolean useRfcQuantizer) in c:\Projects3\MEDIA\net7mma-111925\RtspServer\MediaTypes\RFC2435Media.cs:line 1318
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.ToImage() in c:\Projects3\MEDIA\net7mma-111925\RtspServer\MediaTypes\RFC2435Media.cs:line 1634
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.op_Implicit(RFC2435Frame f) in c:\Projects3\MEDIA\net7mma-111925\RtspServer\MediaTypes\RFC2435Media.cs:line 1680
   at testRtsp3.Program.Client_RtpFrameChanged(Object sender, RtpFrame frame, TransportContext tc, Boolean final) in c:\Projects3\MEDIA\testRtsp3\Program.cs:line 151
Coordinator
Mar 7, 2016 at 5:10 PM
Edited Mar 7, 2016 at 5:27 PM
Can you please include the Length, Timestamp, Sequence and Marker in the packets your logging as well as provide a Wireshark Capture?

'07.03.2016 10:31:59 OK: 1 frame.IsMissingPackets : False frame.IsComplete : True'

Seems like a repeat packet, this occurs for seven other packets as well according to your output.

What is hard to tell is if the RTP Timestamp is the same and if the byte lengths are the same of the packets and why your getting them especially under TCP.

Regardless of this though I do see problems where Add will not determine the correct index when performing Insert and can potentially incorrectly set the m_LowestSequenceNumber when discontinuity is encountered.

I will include a UnitTest and fix in the next release as soon as possible and hopefully that will fix your issue and if not we can troubleshoot further.
Marked as answer by juliusfriedman on 3/7/2016 at 9:10 AM
Coordinator
Mar 8, 2016 at 12:06 AM
This should be addressed in 11927.

I will eventually add more unit tests to confirm.

Please let me know if your still having trouble.

Thanks again for bringing this up!
Marked as answer by juliusfriedman on 3/7/2016 at 4:06 PM
Mar 8, 2016 at 10:08 AM
Hi,

Sorry for that but 111927 do not works too.

h**p://85.25.239.145/wireshark_dump11.rar

Same logs

08.03.2016 11:48:40 ERR: 884 IsMissingPackets :False IsComplete : False Timesta
mp : -2139416966 LowestSequenceNumber : 28224 HighestSequenceNumber : 28228 HasM
arker : False
08.03.2016 11:48:40 ERR: 885 IsMissingPackets :False IsComplete : False Timesta
mp : -2139416966 LowestSequenceNumber : 28224 HighestSequenceNumber : 28229 HasM
arker : False
08.03.2016 11:48:40 OK: 293 IsMissingPackets :False IsComplete : True Timestamp
: -2139416966 LowestSequenceNumber : 28224 HighestSequenceNumber : 28230 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 294 IsMissingPackets :False IsComplete : True Timestamp
: -2139422996 LowestSequenceNumber : 28217 HighestSequenceNumber : 28223 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 886 IsMissingPackets :True IsComplete : False Timestam
p : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28231 HasMa
rker : False
08.03.2016 11:48:40 ERR: 887 IsMissingPackets :False IsComplete : False Timesta
mp : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28232 HasM
arker : False
08.03.2016 11:48:40 ERR: 888 IsMissingPackets :False IsComplete : False Timesta
mp : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28233 HasM
arker : False
08.03.2016 11:48:40 ERR: 889 IsMissingPackets :False IsComplete : False Timesta
mp : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28234 HasM
arker : False
08.03.2016 11:48:40 ERR: 890 IsMissingPackets :False IsComplete : False Timesta
mp : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28235 HasM
arker : False
08.03.2016 11:48:40 ERR: 891 IsMissingPackets :False IsComplete : False Timesta
mp : -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28236 HasM
arker : False
08.03.2016 11:48:40 OK: 295 IsMissingPackets :False IsComplete : True Timestamp
: -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28237 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 296 IsMissingPackets :False IsComplete : True Timestamp
: -2139416966 LowestSequenceNumber : 28224 HighestSequenceNumber : 28230 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 892 IsMissingPackets :True IsComplete : False Timestam
p : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28238 HasMa
rker : False
08.03.2016 11:48:40 ERR: 893 IsMissingPackets :False IsComplete : False Timesta
mp : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28239 HasM
arker : False
08.03.2016 11:48:40 ERR: 894 IsMissingPackets :False IsComplete : False Timesta
mp : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28240 HasM
arker : False
08.03.2016 11:48:40 ERR: 895 IsMissingPackets :False IsComplete : False Timesta
mp : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28241 HasM
arker : False
08.03.2016 11:48:40 ERR: 896 IsMissingPackets :False IsComplete : False Timesta
mp : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28242 HasM
arker : False
08.03.2016 11:48:40 ERR: 897 IsMissingPackets :False IsComplete : False Timesta
mp : -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28243 HasM
arker : False
08.03.2016 11:48:40 OK: 297 IsMissingPackets :False IsComplete : True Timestamp
: -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28244 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 298 IsMissingPackets :False IsComplete : True Timestamp
: -2139411026 LowestSequenceNumber : 28231 HighestSequenceNumber : 28237 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 898 IsMissingPackets :True IsComplete : False Timestam
p : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28245 HasMa
rker : False
08.03.2016 11:48:40 ERR: 899 IsMissingPackets :False IsComplete : False Timesta
mp : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28246 HasM
arker : False
08.03.2016 11:48:40 ERR: 900 IsMissingPackets :False IsComplete : False Timesta
mp : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28247 HasM
arker : False
08.03.2016 11:48:40 ERR: 901 IsMissingPackets :False IsComplete : False Timesta
mp : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28248 HasM
arker : False
08.03.2016 11:48:40 ERR: 902 IsMissingPackets :False IsComplete : False Timesta
mp : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28249 HasM
arker : False
08.03.2016 11:48:40 ERR: 903 IsMissingPackets :False IsComplete : False Timesta
mp : -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28250 HasM
arker : False
08.03.2016 11:48:40 OK: 299 IsMissingPackets :False IsComplete : True Timestamp
: -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28251 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 300 IsMissingPackets :False IsComplete : True Timestamp
: -2139404996 LowestSequenceNumber : 28238 HighestSequenceNumber : 28244 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 904 IsMissingPackets :True IsComplete : False Timestam
p : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28252 HasMa
rker : False
08.03.2016 11:48:40 ERR: 905 IsMissingPackets :False IsComplete : False Timesta
mp : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28253 HasM
arker : False
08.03.2016 11:48:40 ERR: 906 IsMissingPackets :False IsComplete : False Timesta
mp : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28254 HasM
arker : False
08.03.2016 11:48:40 ERR: 907 IsMissingPackets :False IsComplete : False Timesta
mp : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28255 HasM
arker : False
08.03.2016 11:48:40 ERR: 908 IsMissingPackets :False IsComplete : False Timesta
mp : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28256 HasM
arker : False
08.03.2016 11:48:40 ERR: 909 IsMissingPackets :False IsComplete : False Timesta
mp : -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28257 HasM
arker : False
08.03.2016 11:48:40 OK: 301 IsMissingPackets :False IsComplete : True Timestamp
: -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28258 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 302 IsMissingPackets :False IsComplete : True Timestamp
: -2139399056 LowestSequenceNumber : 28245 HighestSequenceNumber : 28251 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 910 IsMissingPackets :True IsComplete : False Timestam
p : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28259 HasMa
rker : False
08.03.2016 11:48:40 ERR: 911 IsMissingPackets :False IsComplete : False Timesta
mp : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28260 HasM
arker : False
08.03.2016 11:48:40 ERR: 912 IsMissingPackets :False IsComplete : False Timesta
mp : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28261 HasM
arker : False
08.03.2016 11:48:40 ERR: 913 IsMissingPackets :False IsComplete : False Timesta
mp : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28262 HasM
arker : False
08.03.2016 11:48:40 ERR: 914 IsMissingPackets :False IsComplete : False Timesta
mp : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28263 HasM
arker : False
08.03.2016 11:48:40 ERR: 915 IsMissingPackets :False IsComplete : False Timesta
mp : -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28264 HasM
arker : False
08.03.2016 11:48:40 OK: 303 IsMissingPackets :False IsComplete : True Timestamp
: -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28265 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 304 IsMissingPackets :False IsComplete : True Timestamp
: -2139393026 LowestSequenceNumber : 28252 HighestSequenceNumber : 28258 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 916 IsMissingPackets :True IsComplete : False Timestam
p : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28266 HasMa
rker : False
08.03.2016 11:48:40 ERR: 917 IsMissingPackets :False IsComplete : False Timesta
mp : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28267 HasM
arker : False
08.03.2016 11:48:40 ERR: 918 IsMissingPackets :False IsComplete : False Timesta
mp : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28268 HasM
arker : False
08.03.2016 11:48:40 ERR: 919 IsMissingPackets :False IsComplete : False Timesta
mp : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28269 HasM
arker : False
08.03.2016 11:48:40 ERR: 920 IsMissingPackets :False IsComplete : False Timesta
mp : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28270 HasM
arker : False
08.03.2016 11:48:40 ERR: 921 IsMissingPackets :False IsComplete : False Timesta
mp : -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28271 HasM
arker : False
08.03.2016 11:48:40 OK: 305 IsMissingPackets :False IsComplete : True Timestamp
: -2139381056 LowestSequenceNumber : 28266 HighestSequenceNumber : 28272 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 OK: 306 IsMissingPackets :False IsComplete : True Timestamp
: -2139386996 LowestSequenceNumber : 28259 HighestSequenceNumber : 28265 HasMar
ker : True
This Frame not Complete
08.03.2016 11:48:40 ERR: 922 IsMissingPackets :True IsComplete : False Timestam
p : -2139375026 LowestSequenceNumber : 28273 HighestSequenceNumber : 28273 HasMa
rker : False
08.03.2016 11:48:40 ERR: 923 IsMissingPackets :False IsComplete : False Timesta
mp : -2139375026 LowestSequenceNumber : 28273 HighestSequen
Coordinator
Mar 8, 2016 at 8:03 PM
Edited Mar 8, 2016 at 8:06 PM
I'm gonna have to look into this in a bit more detail and let you know what I find.

Please also take a look at the latest code and make sure that the issue persists although there haven't been many changes which should effect this beyond what was updated.

There may also be a problem with some of the properties of the rtpframe which are causing this issue due to recent changes for the default values or otherwise but it seems unrelated.

Sorry for any inconvenience this may have caused so far and thanks for providing additional information.

I'll look at the Wireshark capture shortly and let you know.
Marked as answer by juliusfriedman on 3/8/2016 at 12:03 PM
Coordinator
Mar 8, 2016 at 9:35 PM
Edited Mar 8, 2016 at 9:36 PM
Weirdly enough you are not experiencing any re-ordering or duplicate packets (at least based on the capture)

What exception are you getting now and what is the stack trace? Frame Is Not Complete

I think what is happening is that now you are calling ToImage at the incorrect time, can you provide an example of how your using it? (Especially when this occurs)

I this most of these problems can be avoided simply by checking for 'IsComplete' before calling ToImage, or the final parameter on the event depending on what you are trying to do, e.g. to decode a frame partially before it's received or otherwise you may need to use the overload of PrepareBuffer to allow the incomplete frame data to be written to the stream if possible, if you just want to wait until the very last possible time before handling the frame then consider check 'final' to determine if the event will be fired again.

Typically something like this but it doesn't handle certain odd cases as I will explain below.
'if (false == final && false == frame.IsComplete) return;'

If needed one can implement a mechanism to retain frames for longer by capturing the packets and storing them for use elsewhere as long as they are still considered valid for your application, this is usually called a JitterBuffer and is useful at the application level to basically decide when a sequence of packets would still be valid, BUT this is typically not required and I have made provisions that allow for this to be more easily dealt with at the 'Frame' level since there can never be a legitimate change in the Timestamp field in a series of packets, it is completely possible that my modifications are not 100% correct for short series of packets where there is bad network congestion / re-ordering (such as series of packets 2 - 16 in length).

Consider the following which may help to explain this:

Timestamp = 0, 3 packets (0, 1, 2)

Timestamp = 1, 2 packets (3, 4)

Timestamp = 2, 1 packets (5)

Timestamp = 3, 1 packet (6)

Timestamp = 4, 2 packets (7, 8)

Timestamp = 5, 2 packets (9, 10)

Receiver

NewFrame A
Timestamp = 0, (seq 2) [IsComplete, but Missing 0, 1] (should have Marker packet and may will return true for IsComplete because no other packets are contained in the frame)

NewFrame B
Timestamp = 1 (seq 3) [Missing 4]

This is a problem and would cause the 'final' parameter to be true and NewFrame A to be switched to NewFrame B and NewFrame A to be allocated as NewFrame C

NewFrame C
Timestamp = 2 (seq 5) [IsComplete]

New Frame B
Timestamp = 0 (seq 1) [IsComplete, Missing 0 but unknown because it's re-ordered]

New Frame B
Timestamp = 0 (seq 0) [IsComplete]

... Depending on the order the example can be better or worse but that should illustrate the possible issue.

To implement something like a JitterBuffer more memory would have to be used to keep packets around longer and doesn't serve a real purpose in aggregating the packets simply because packets should be sent as quickly as possible to the remote party.

The RtspServer may be able to help eliminate re-ordering but based on your capture I can't see any re-ordering anyway so I still need to clarify what issue your having any why to accurately help further.

You can also attempt to diagnose and improve your network environment so that there is not as much reordering / loss but again your capture doesn't really show any signs of that.

Another interpretation of the capture however is that your are spending a lot of time in logging and or other application logic or simply at a breakpoint which is causing the underlying issues in the application itself which would mean it is not coming from the network at all...

You may have luck providing a SocketConfiguration to 'ConfigureRtspSocket' through the RtspClient, you can try to increase the ReceiveBufferSize at first to see if this helps and if it does then at that point you should possibly try 'EnableTcpCongestionAlgorithm' to see if it is more a network issue or move of a application issue.

Let me know how I can help further!

It would also be helpful to include a sample of how your creating the RtspClient and using the events from the RtpClient so I can see if theres something else I am missing here...
Marked as answer by juliusfriedman on 3/8/2016 at 1:35 PM
Mar 9, 2016 at 1:43 PM
Hi,

Txh. for your kind explanations.

Stack trace is below;
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.PrepareBuffer(Boolean allowLegacyPackets, Boolean allowIncomplete, Boolean useRfcQuantizer) in c:\Projects3\MEDIA\net7mma-111928\RtspServer\MediaTypes\RFC2435Media.cs:line 1318
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.ToImage() in c:\Projects3\MEDIA\net7mma-111928\RtspServer\MediaTypes\RFC2435Media.cs:line 1634
   at Media.Rtsp.Server.MediaTypes.RFC2435Media.RFC2435Frame.op_Implicit(RFC2435Frame f) in c:\Projects3\MEDIA\net7mma-111928\RtspServer\MediaTypes\RFC2435Media.cs:line 1680
   at testRtsp3.Program.Client_RtpFrameChanged(Object sender, RtpFrame frame, TransportContext tc, Boolean final) in c:\Projects3\MEDIA\testRtsp3\Program.cs:line 123
You may find my test program at h**p://85.25.239.145/rtsp_client1.rar

My Cams and Development PC are connected to same dedicated switch.
Coordinator
Mar 9, 2016 at 2:37 PM
I will let you know when I have downloaded the code.

I am seriously considering adding a simple frame cache which can act as a jitter buffer which would also help to eliminate reordering at the cost of additional latency.

I envision a property or two on the RtpClient to support this, 'bool BufferFrameEvents' and 'int MaxFramesInBuffer'.

Therefor when a frame moves from last and will be disposed normally, when BufferFrame events is true I would be able to reduce the point at which final is true until the Frame was removed from the buffer.

This however presents the potential for different events such as frames dropped from the buffer and also presents a problem that if the stream has a lot of loss that you potentially will still have the same problem.

A slightly different approach is to revise the frame handling logic into a RtpFramer which can do as the application required.

Yet another way would be to add an additional Rtpframe to the logic which gives more than 2 frames to reference for framing.

You can probably see though that when final is true in the current implementation this can already be handled by taking the frame which cause the exception and storing it so when RtpPackets arrive for it you can add them to the stored fame or remove the stored frame as you require.

In your case it would be easy as rfc2435 has the fragment offset which can be used to tell if you have the actual frame start.

In short it seems that if you call IsComplete before calling ToImage you will not have that problem as the class for Rfc2435 uses the information in the profile header to determine if it has the first packet, all of the other logic should work appropriately.

I'll take a look at your code and let you know what I find soon.
Marked as answer by juliusfriedman on 3/9/2016 at 6:37 AM
Coordinator
Mar 9, 2016 at 4:28 PM
Edited Mar 9, 2016 at 6:25 PM
Just downloaded the code, I will keep you updated.

Update

Looking at the code the only possible thing I see is the SAVE code
                // SAVE
                try
                {
                    using (RFC2435Media.RFC2435Frame t = new RFC2435Media.RFC2435Frame(frame))
                    {
                        //Sometimes a profile defines a field which can determine if the first packet was received...
                        bool frameComplete = t.IsComplete;

                         //Here the first packet may not have the FragmentOffset == 0, this needs to be handled.
                         if(false == frameComplete) return; 

                        //Optionally allow Incomplete packets, but not legacy packets and use the alternate tables
                        t.PrepareBuffer(false, true, false);

                        //You don't have to Dispose the Image here as the frame will do that for you when the frame is Disposed
                       //If you want the image to persist afterward you would have to copy the frame or the image at this point.
                        Image jpeg = t; //using (Image jpeg = t)
                        {
                            jpeg.Save(@"C:\imgs\result" + i1 + ".jpg", ImageFormat.Jpeg);
                        }
                    }
                }
                catch (Exception ex)
                {

                    Console.WriteLine(ex.Message);
                }
Based on the capture though you are not having the first packet missing problem but the example still applies for completeness and in your case because you are missing packets...

The other interesting thing I am trying to see is why IsComplete would return true when IsMissingPackets should be false according to the capture.

I am going to make a unit test to try to account for this but you are definitely missing at least one packet in the sequences where the exception occurred according to the capture. (28253)

So that sequence should look like this:
Timestamp = 2155574270
28252, IsMissingPackets = False
-28253 (Missing, but would result is no change to IsMissingPackets)
28254, IsMissingPackets = True
28255, ...
28256, ...
28257, ...
28258, Marker packet, IsComplete = false, IsMissingPackets = true, HasMarker = true

Timestamp = 2155580300
28259, Frame switches places to last frame or final event occurs. IsMissingPackets = False
28260, IsMissingPackets = False
28261,..
28262,..
28263,..
28264,.. IsMissingPackets = False, IsComplete = false (No marker)
-282265 (Missing, would contain marker packet)

Timestamp = 2155586240
28266, Frame switches places to last frame or final event occurs. IsMissingPackets = False
->
28273, (IsComplete = true, IsMissingPackets = false)


None the less, I think that you can fix your problem by allowing a larger receive buffer size on the RtspSocket when using TCP, I specifically disable buffering which may not be desirable in all situations.
internal static void ConfigureRtspSocket(Socket socket)
        {
            if (socket == null) throw new ArgumentNullException("Socket");

            Media.Common.Extensions.Socket.SocketExtensions.EnableAddressReuse(socket);
            //socket.ExclusiveAddressUse = false;
            //socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);

            //Don't buffer send.
            socket.SendBufferSize = 0;

            //Don't buffer receive.
            //socket.ReceiveBufferSize = 0;

            //Dont fragment
            if (socket.AddressFamily == AddressFamily.InterNetwork) socket.DontFragment = true;

            //Rtsp over Tcp
            if (socket.ProtocolType == ProtocolType.Tcp)
            {
                //
                //Media.Common.Extensions.Socket.SocketExtensions.EnableTcpNoSynRetries(socket);

                //
                //Media.Common.Extensions.Socket.SocketExtensions.EnableTcpTimestamp(socket);

                //
                //Media.Common.Extensions.Socket.SocketExtensions.SetTcpOffloadPreference(socket);

                //
                //Media.Common.Extensions.Socket.SocketExtensions.EnableTcpCongestionAlgorithm(socket);

                // Set option that allows socket to close gracefully without lingering.
                Media.Common.Extensions.Socket.SocketExtensions.DisableLinger(socket);

                //Retransmit for 0 sec
                if (Common.Extensions.OperatingSystemExtensions.IsWindows) Media.Common.Extensions.Socket.SocketExtensions.DisableTcpRetransmissions(socket);

                //If both send and receieve buffer size are 0 then there is no coalescing when nagle's algorithm is disabled
                Media.Common.Extensions.Socket.SocketExtensions.DisableTcpNagelAlgorithm(socket);
                //socket.NoDelay = true;

                //Allow more than one byte of urgent data
                //Media.Common.Extensions.Socket.SocketExtensions.EnableTcpExpedited(socket);
                //socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.Expedited, true);

                //Receive any urgent data in the normal data stream
                //Media.Common.Extensions.Socket.SocketExtensions.EnableTcpOutOfBandDataInLine(socket);
            }
        }
Then tell the client to use the configuration with 'client.ConfigureSocket = ConfigureSocket;' before 'StartPlaying' is called.

Hopefully that will help with the missing packets.

I will see about adding some type of JitterBuffer but honestly with loss it can't help you, the decoder has to cope with it which is another reason why I have not added something to allow for this in a general level like an ExpiringList.

You can attempt to cope with certain types of loss by using information from previous packets or frames such as quality, height and width or quantization tables but this is outside of the scope of RFC2435.

Why I don't make instance fields accessible such as Quality, Height and Width is because they are included with every packet, at first it might make sense to make a property out of the QuantizationTables though so that other frames can borrow the tables when the first packet is missing in later frames but in such cases you would also be missing all or portion of the first block in the scan data which occurred after the quantization table header in the payload of the first packet... (Which means you would have some artifacts in the decoded result)

If increasing the receive buffer size doesn't help you can try reducing the Quality or Framerate at the Camera level itself and see if that helps you also.

Please do let me know if your still having trouble after that and I will take another look.
Marked as answer by juliusfriedman on 3/9/2016 at 10:24 AM
Coordinator
Mar 10, 2016 at 6:56 PM
111930 may give you better results.

I have also added some methods which will help obtaining the quantization tables if you need them.

Let me know if you are still having trouble!
Marked as answer by juliusfriedman on 3/10/2016 at 10:56 AM
Mar 11, 2016 at 8:26 AM
Hello,

Sorry but 111931 does not works too.

Debug further and try codes above but nothing looks to be changed.

To be sure that everything works also test with old version 110806, that works very well.

Found that;
  1. t.IsComplete is always false, so code always return, but frame .IsCompale is true.
 using (RFC2435Media.RFC2435Frame t = new RFC2435Media.RFC2435Frame(frame))
                    {
                        //Sometimes a profile defines a field which can determine if the first packet was received...
                        bool frameComplete = t.IsComplete;

                        //Here the first packet may not have the FragmentOffset == 0, this needs to be handled.
                        if (false == frameComplete) return;
  1. Very rarely base.IsComplete is true, but code below always return false.
 public override bool IsComplete
            {
                get
                {
                    //First use the existing IsComplete logic.
                    if (false == base.IsComplete) return false;

                    var packet = m_Packets.First();

                    //The FragmentOffset must be 0 at the start of a new frame.
                    return Common.Binary.ReadU24(packet.Payload.Array, packet.HeaderOctets + 1, BitConverter.IsLittleEndian) == 0;
                }
            } 
Coordinator
Mar 11, 2016 at 12:26 PM
Edited Mar 11, 2016 at 12:30 PM
I will check 110806 and let you know.

The reason why this was logic changed was because it's the correct way to determine if a frame is aligned from start to end.

Without the logic it's possible that the image could still be decoded without the tables and first block but data will be missing from the result.

FragmentOffset must be zero for the first packet in a frame.

I do want to double check and make sure I haven't introduced any bugs so I'll compare with the older version and let you know.

Depending on what I find I'll either fix it or add a property which can be used maintain that logic if possible.

In your case IsComplete should be false on the plain Rtpframe unless you missed the first packet, at which point IsComplete would be true if you recieved the marker packet. When using RFC2435Frame even if you have the marker and no missing packets you might have missed the first packet of the frame and that's where it's logic steps in to confirm this.

I will take another look and sorry for any inconvenience.

Julius
Marked as answer by juliusfriedman on 3/11/2016 at 4:26 AM
Coordinator
Mar 11, 2016 at 2:53 PM
Edited Mar 11, 2016 at 5:39 PM
111932 should fix any difference in the code, the small difference was using Payload and not calculating the offset appropriately.

I am not sure the fixes help with the loss you experience but maybe when run in release the code may have slightly better performance, definitely let me know if we are on the same page now.

Let me know how it goes and thanks for bringing this up.

-Julius
Marked as answer by juliusfriedman on 3/11/2016 at 6:53 AM