Managed Media Aggregation Stability & Performance on Xamarin Devices (Android and iOS)

Topics: Bug Archive
May 17, 2016 at 7:42 AM
Edited May 17, 2016 at 7:45 AM
Hi Julius,

thanks again for your help and this library :-)

We now upgraded from Version 112027 to the new 51.

Since we did, the Picture only Shows some blocky parts, we do not get a complete Picture anymore. Only some blocks who Change fast but only cover small pieces of the Picture.

The Audio is more broken with klicking noise.

Could you please have a look, it would be nice to have the actual Version working on Android, where I go back to Version 112027 now and on iOS I am still using older 111988 (because I Need to test and Change a lot of stuff to get it working on iOS).

If I have it working on Android again, I would then work on the iOS Problems and let you know, which Problems occur on iOS (if you are interested).

We are still working with a copy of RFC6184 (becuase System.Drawing does not work on Xamarin) and using the RTPClient this way:
        internal void OnSourceFrameChanged(object sender, RtpFrame frame = null, RtpClient.TransportContext tc = null, bool final = false)
        {
            if (frame != null)
            {
                if ((frame.PayloadType == Globals.H264PlayloadType) && final)
                {
                    if (!isVideoOn)
                    {
                        isVideoOn = true;
                        if (RefreshUI != null)
                            RefreshUI("Video is starting", Xamarin.Forms.Color.Green);
                    }
                    Media.Common.BaseDisposable.SetShouldDispose(frame, false);
                    _videoQueue.Add(frame);

                    if (Globals.DoDetailedLog && frame.IsMissingPackets)
                    {
                        ClientLogger.Instance.Log(string.Format("Attention! Data loss in RTPClient OnSourceFrameChanged at Time: {0}:{1} ", DateTime.Now.ToString(), DateTime.Now.Millisecond));
                    }
                }
                else if (tc.MediaDescription.MediaType == MediaType.audio)
                {
                    Media.Common.BaseDisposable.SetShouldDispose(frame, false);
                    _audioQueue.Add(frame);

            }
        }
        else
        {
            ClientLogger.Instance.Log("NULL - RTP frame received");
        }
        }
Like you suggested, we are using Queues now which work their data in their own thread. Works good this way. Only Thing I now did was changing to the new MMA Version, which causes the mentioned Problems above...

The Video Queue is working this way using our own RFC6184 class copied from yours:

        private void _videoQueue_DataAvailable(object sender, ThreadingQueueDataEventArgs<RtpFrame> e)
        {
            using (RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(e.Item))
            {
                profileFrame.Depacketize(); //false true
                if (profileFrame.HasDepacketized)
                {
                    if (VideoDataReceived != null)
                        VideoDataReceived(this, new DataEventArgs() { Data = profileFrame.Buffer.ToArray() });

                    Media.Common.BaseDisposable.SetShouldDispose(e.Item, true);
                }
            }
        }
thanks a lot!
Coordinator
May 17, 2016 at 2:00 PM
Edited May 17, 2016 at 2:09 PM
I am interested in the issues on IOS but I think that they are resolved with the ResumeOnError.

Media.Common.BaseDisposable.SetShouldDispose has an optional parameter at the end which is calling dispose.

I will see about making that API more fluent so that it's not confused but you should call it with Media.Common.BaseDisposable.SetShouldDispose(e.Item, true, false); if you don't want to dispose right away.

There is really not that much to change between versions, especially if you don't use the ThreadEvents or IListSockets, especially nothing that should change the logic of existing programs..
Marked as answer by juliusfriedman on 5/17/2016 at 7:03 AM
May 18, 2016 at 8:51 AM
Hi Julius,

when I use Version 112027 everything works fine.

When I use 112051 or just tried the latest 112057, I only see very few small blocks instead of a Picture and Sound is more broken / clicking.
Without changing anything in my Code, just exchanged the assemblies.

I have 4 times used the Media.Common.BaseDisposable.SetShouldDispose. When I try with the last optional Parameter (I before did not set it, so Default wass false) set to true, the app crashes sometimes as soon as I start the rtpclient. in other cases it does not Crash but I get no Picture again, just some few single small blocks.

When I return to the 112027 assemblies, everything is fine again.

Can you please adive me how to get the actual Version running or how to find the Problem or help you find the Problem?

We would like to have a new Version running, so we can freeze it in Android for a while, get it running on iOS then so we have a working base on both platforms, then I will start with a Windows 10 phone next. :-)


Thanks and have a nice day!
Coordinator
May 18, 2016 at 11:08 AM
Edited May 18, 2016 at 11:09 AM
I have a few more updates to get out today or tommorow, hopefully they might resolve your issue.

I will see what's the big difference in 112027 vs the current with some testing.

The best way I can advise to help is by comparing known good output and logs and profiling information from release to release, debugging also helps.

Can you try enabling ThreadEvents in the RtpClient and see if you get better or worse results.
Marked as answer by juliusfriedman on 5/18/2016 at 4:08 AM
Coordinator
May 18, 2016 at 4:39 PM
I have made some more updates @ 112059.

I have really good results with this version....

Lets see how it works for you.
Marked as answer by juliusfriedman on 5/18/2016 at 9:39 AM
Coordinator
May 18, 2016 at 6:46 PM
112060 is released. More bug fixes :)
Marked as answer by juliusfriedman on 5/18/2016 at 11:46 AM
Coordinator
May 18, 2016 at 10:59 PM
112062 more bug fixes :)
Marked as answer by juliusfriedman on 5/18/2016 at 3:59 PM
May 19, 2016 at 6:29 AM
Edited May 19, 2016 at 7:52 AM
Hi Julius,

I tried 112062 with the same Problem again (just see some few Little blocks instead of a Picture).

What are ThreadEvents? I didn't find something like this. Where should I activate?
Please advice me what to set or what to test if that helps you helping me ;-)

I have a presentation soon. If I do not get the latest RTPClient running I will freeze on 112027 for a while and make that Version running on iOS but I would prefer a newer Version :-)

Thanks a lot!
May 20, 2016 at 6:46 AM
Edited May 20, 2016 at 7:28 AM
Hi Julius,

trying 112064 brings up the same Problem, especially the Video I get is totally broken, I never get a complete Picture, only some very small blocks, which are moving fast.

Any idea what I can examine to find the Problem?

I Need to freeze the RTPClient now. Of Course I will check updates every week for the next months, but not daily anymore.
And I have a presentation begin of next week.

How do I get it working? How to find the error? Or using 112027?

Thanks a lot :-)

PS: I logged now. Here my used Code for Framechanged:
        internal void OnSourceFrameChanged(object sender, RtpFrame frame = null, RtpClient.TransportContext tc = null, bool final = false)
        {
            if (frame != null)
            {
                if ((frame.PayloadType == Globals.H264PlayloadType) && final)
                {
                    if (Globals.DoDetailedLog)
                        ClientLogger.Instance.Log(string.Format("Got a RTP Frame PacketCount {0}  IsComplete {1}  IsFinal {2} Timestamp {3}  Time: {4}:{5}", frame.Count, frame.IsComplete, final, frame.Timestamp, DateTime.Now.ToString(), DateTime.Now.Millisecond));

                    Media.Common.BaseDisposable.SetShouldDispose(frame, false);
                    _videoQueue.Add(frame);



        private void _videoQueue_DataAvailable(object sender, ThreadingQueueDataEventArgs<RtpFrame> e)
        {
            if (VideoDataReceived != null)
                VideoDataReceived(this, new DataEventArgs() { Data = DepacketizeH264(e.Item) });
            Media.Common.BaseDisposable.SetShouldDispose(e.Item, true);
        }


        private byte[] DepacketizeH264(RtpFrame frame)
        {
            byte[] result = new byte[0];

            using (RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(frame))
            {
                profileFrame.Depacketize();
                if (profileFrame.HasDepacketized)
                {
                    result = profileFrame.Buffer.ToArray();
                }
            }
            return result;
        }
Smetimes now it runs into that Decoder IllegalStateException, sometimes not. But I never get a Picture anymore, just a few blocky Fragments. Using 112027 without other changes works fine...


05-20 09:12:55.109 D/SurfaceUtils(26222): set up nativeWindow 0xb9988b58 for 720x576, color 0x7fa30c04, rotation 0, usage 0x42002900
Thread started: #15
inputBufferIndex 2 Time: 20.05.2016 09:12:55:135
05-20 09:12:55.135 I/mono-stdout(26222): inputBufferIndex 2 Time: 20.05.2016 09:12:55:135
Got a RTP Frame PacketCount 16 IsComplete True IsFinal True Timestamp -907233281 Time: 20.05.2016 09:12:55:143
05-20 09:12:55.143 I/mono-stdout(26222): Got a RTP Frame PacketCount 16 IsComplete True IsFinal True Timestamp -907233281 Time: 20.05.2016 09:12:55:143
Got a RTP Frame PacketCount 11 IsComplete True IsFinal True Timestamp -907227231 Time: 20.05.2016 09:12:55:183
05-20 09:12:55.183 I/mono-stdout(26222): Got a RTP Frame PacketCount 11 IsComplete True IsFinal True Timestamp -907227231 Time: 20.05.2016 09:12:55:183
inputBufferIndex 3 Time: 20.05.2016 09:12:55:186
05-20 09:12:55.186 I/mono-stdout(26222): inputBufferIndex 3 Time: 20.05.2016 09:12:55:186
Got a RTP Frame PacketCount 8 IsComplete True IsFinal True Timestamp -907224301 Time: 20.05.2016 09:12:55:218
05-20 09:12:55.218 I/mono-stdout(26222): Got a RTP Frame PacketCount 8 IsComplete True IsFinal True Timestamp -907224301 Time: 20.05.2016 09:12:55:218
Got a RTP Frame PacketCount 7 IsComplete True IsFinal True Timestamp -907220701 Time: 20.05.2016 09:12:55:280
05-20 09:12:55.280 I/mono-stdout(26222): Got a RTP Frame PacketCount 7 IsComplete True IsFinal True Timestamp -907220701 Time: 20.05.2016 09:12:55:280
inputBufferIndex 0 Time: 20.05.2016 09:12:55:282
05-20 09:12:55.283 I/mono-stdout(26222): inputBufferIndex 0 Time: 20.05.2016 09:12:55:282
Got a RTP Frame PacketCount 12 IsComplete True IsFinal True Timestamp -907217100 Time: 20.05.2016 09:12:55:328
05-20 09:12:55.329 I/mono-stdout(26222): Got a RTP Frame PacketCount 12 IsComplete True IsFinal True Timestamp -907217100 Time: 20.05.2016 09:12:55:328
inputBufferIndex 1 Time: 20.05.2016 09:12:55:331
05-20 09:12:55.331 I/mono-stdout(26222): inputBufferIndex 1 Time: 20.05.2016 09:12:55:331
Got a RTP Frame PacketCount 9 IsComplete True IsFinal True Timestamp -907211680 Time: 20.05.2016 09:12:55:357
05-20 09:12:55.358 I/mono-stdout(26222): Got a RTP Frame PacketCount 9 IsComplete True IsFinal True Timestamp -907211680 Time: 20.05.2016 09:12:55:357
inputBufferIndex 2 Time: 20.05.2016 09:12:55:359
05-20 09:12:55.359 I/mono-stdout(26222): inputBufferIndex 2 Time: 20.05.2016 09:12:55:359
Got a RTP Frame PacketCount 5 IsComplete True IsFinal True Timestamp -907208080 Time: 20.05.2016 09:12:55:379
05-20 09:12:55.379 I/mono-stdout(26222): Got a RTP Frame PacketCount 5 IsComplete True IsFinal True Timestamp -907208080 Time: 20.05.2016 09:12:55:379
inputBufferIndex 3 Time: 20.05.2016 09:12:55:381
05-20 09:12:55.381 I/mono-stdout(26222): inputBufferIndex 3 Time: 20.05.2016 09:12:55:381
Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -907204480 Time: 20.05.2016 09:12:55:400
05-20 09:12:55.400 I/mono-stdout(26222): Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -907204480 Time: 20.05.2016 09:12:55:400
inputBufferIndex 0 Time: 20.05.2016 09:12:55:402
05-20 09:12:55.402 I/mono-stdout(26222): inputBufferIndex 0 Time: 20.05.2016 09:12:55:402
Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -907200880 Time: 20.05.2016 09:12:55:414
05-20 09:12:55.414 I/mono-stdout(26222): Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -907200880 Time: 20.05.2016 09:12:55:414
inputBufferIndex 1 Time: 20.05.2016 09:12:55:416
05-20 09:12:55.416 I/mono-stdout(26222): inputBufferIndex 1 Time: 20.05.2016 09:12:55:416
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -907197280 Time: 20.05.2016 09:12:55:430
05-20 09:12:55.430 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -907197280 Time: 20.05.2016 09:12:55:430
Exception occured in H264Decoder ProcessDecoderOutput: Java.Lang.IllegalStateException: Exception of type 'Java.Lang.IllegalStateException' was thrown.
05-20 09:12:55.439 I/mono-stdout(26222): Exception occured in H264Decoder ProcessDecoderOutput: Java.Lang.IllegalStateException: Exception of type 'Java.Lang.IllegalStateException' was thrown.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in /Users/builder/data/lanes/3236/ee215fc9/source/mono/external/referencesource/mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:143
at Android.Runtime.JNIEnv.CallVoidMethod (IntPtr jobject, IntPtr jmethod, Android.Runtime.JValue* parms) [0x00063] in /Users/builder/data/lanes/3236/ee215fc9/source/monodroid/src/Mono.Android/src/Runtime/JNIEnv.g.cs:571
05-20 09:12:55.440 I/mono-stdout(26222): at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in /Users/builder/data/lanes/3236/ee215fc9/source/mono/external/referencesource/mscorlib/system/runtime/exceptionservices/exceptionservicescommon.cs:143
05-20 09:12:55.440 I/mono-stdout(26222): at Android.Runtime.JNIEnv.CallVoidMethod (IntPtr jobject, IntPtr jmethod, Android.Runtime.JValue* parms) [0x00063] in /Users/builder/data/lanes/3236/ee215fc9/source/monodroid/src/Mono.Android/src/Runtime/JNIEnv.g.cs:571
at Android.Media.MediaCodec.Configure (Android.Media.MediaFormat format, Android.Views.Surface surface, Android.Media.MediaCrypto crypto, MediaCodecConfigFlags flags) [0x00082] in /Users/builder/data/lanes/3236/ee215fc9/source/monodroid/src/Mono.Android/platforms/android-23/src/generated/Android.Media.MediaCodec.cs:1068
at sks_Client.Droid.H264Decoder.ProcessDecoderOuput () [0x00064] in C:\Projekte\sks-Kinkel\Repos\Apps Gateway 2.0\sks-Client\sks-Client\sks_Client.Droid\H264Decoder.cs:405
--- End of managed exception stack trace ---
May 20, 2016 at 7:29 AM
Edited May 20, 2016 at 8:07 AM
Later it is working again without Illegal State Exceptions but I still only get few small block Fragments, never a Picture:

05-20 09:13:12.887 I/mono-stdout(26222): inputBufferIndex 2 Time: 20.05.2016 09:13:12:887
05-20 09:13:12.894 I/mono-stdout(26222): after render outputBufferIndex 15 Time: 20.05.2016 09:13:12:893
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905606052 Time: 20.05.2016 09:13:12:925
05-20 09:13:12.926 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905606052 Time: 20.05.2016 09:13:12:925
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905602454 Time: 20.05.2016 09:13:12:962
05-20 09:13:12.962 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905602454 Time: 20.05.2016 09:13:12:962
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905598854 Time: 20.05.2016 09:13:13:5
05-20 09:13:13.005 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905598854 Time: 20.05.2016 09:13:13:5
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905595254 Time: 20.05.2016 09:13:13:42
05-20 09:13:13.042 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905595254 Time: 20.05.2016 09:13:13:42
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905591654 Time: 20.05.2016 09:13:13:85
05-20 09:13:13.085 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905591654 Time: 20.05.2016 09:13:13:85
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905588054 Time: 20.05.2016 09:13:13:125
05-20 09:13:13.126 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905588054 Time: 20.05.2016 09:13:13:125
05-20 09:13:13.177 I/art (26222): Starting a blocking GC Explicit
05-20 09:13:13.197 I/art (26222): Explicit concurrent mark sweep GC freed 231(90KB) AllocSpace objects, 0(0B) LOS objects, 40% free, 16MB/27MB, paused 320us total 19.963ms
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905584453 Time: 20.05.2016 09:13:13:198
05-20 09:13:13.198 D/Mono (26222): GC_OLD_BRIDGE num-objects 36 num_hash_entries 36 sccs size 36 init 0.00ms df1 0.68ms sort 0.01ms dfs2 0.95ms setup-cb 0.05ms free-data 0.05ms links 0/0/0/0 dfs passes 72/36
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905580854 Time: 20.05.2016 09:13:13:203
05-20 09:13:13.198 D/Mono (26222): GC_MINOR: (Nursery full) pause 13.19ms, total 13.48ms, bridge 0.00ms promoted 6240K major 6240K los 873K
05-20 09:13:13.198 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905584453 Time: 20.05.2016 09:13:13:198
05-20 09:13:13.203 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905580854 Time: 20.05.2016 09:13:13:203
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905577253 Time: 20.05.2016 09:13:13:242
05-20 09:13:13.242 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905577253 Time: 20.05.2016 09:13:13:242
inputBufferIndex 3 Time: 20.05.2016 09:13:13:243
05-20 09:13:13.244 I/mono-stdout(26222): inputBufferIndex 3 Time: 20.05.2016 09:13:13:243
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905573653 Time: 20.05.2016 09:13:13:282
05-20 09:13:13.282 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905573653 Time: 20.05.2016 09:13:13:282
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905570051 Time: 20.05.2016 09:13:13:322
05-20 09:13:13.322 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905570051 Time: 20.05.2016 09:13:13:322
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905566454 Time: 20.05.2016 09:13:13:362
05-20 09:13:13.362 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905566454 Time: 20.05.2016 09:13:13:362
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905562853 Time: 20.05.2016 09:13:13:402
05-20 09:13:13.402 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905562853 Time: 20.05.2016 09:13:13:402
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905559253 Time: 20.05.2016 09:13:13:442
05-20 09:13:13.442 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905559253 Time: 20.05.2016 09:13:13:442
inputBufferIndex 0 Time: 20.05.2016 09:13:13:444
05-20 09:13:13.444 I/mono-stdout(26222): inputBufferIndex 0 Time: 20.05.2016 09:13:13:444
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905555653 Time: 20.05.2016 09:13:13:485
05-20 09:13:13.485 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905555653 Time: 20.05.2016 09:13:13:485
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905552053 Time: 20.05.2016 09:13:13:522
05-20 09:13:13.523 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905552053 Time: 20.05.2016 09:13:13:522
inputBufferIndex 1 Time: 20.05.2016 09:13:13:524
05-20 09:13:13.524 I/mono-stdout(26222): inputBufferIndex 1 Time: 20.05.2016 09:13:13:524
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905548453 Time: 20.05.2016 09:13:13:565
05-20 09:13:13.565 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905548453 Time: 20.05.2016 09:13:13:565
inputBufferIndex 2 Time: 20.05.2016 09:13:13:567
05-20 09:13:13.568 I/mono-stdout(26222): inputBufferIndex 2 Time: 20.05.2016 09:13:13:567
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905544853 Time: 20.05.2016 09:13:13:606
05-20 09:13:13.606 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905544853 Time: 20.05.2016 09:13:13:606
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905541252 Time: 20.05.2016 09:13:13:661
05-20 09:13:13.661 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905541252 Time: 20.05.2016 09:13:13:661
inputBufferIndex 3 Time: 20.05.2016 09:13:13:664
after render outputBufferIndex 4 Time: 20.05.2016 09:13:13:669
05-20 09:13:13.664 I/mono-stdout(26222): inputBufferIndex 3 Time: 20.05.2016 09:13:13:664
05-20 09:13:13.669 I/mono-stdout(26222): after render outputBufferIndex 4 Time: 20.05.2016 09:13:13:669
Got a RTP Frame PacketCount 12 IsComplete True IsFinal True Timestamp -905537653 Time: 20.05.2016 09:13:13:687
05-20 09:13:13.688 I/mono-stdout(26222): Got a RTP Frame PacketCount 12 IsComplete True IsFinal True Timestamp -905537653 Time: 20.05.2016 09:13:13:687
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905534053 Time: 20.05.2016 09:13:13:725
05-20 09:13:13.725 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905534053 Time: 20.05.2016 09:13:13:725
inputBufferIndex 0 Time: 20.05.2016 09:13:13:726
05-20 09:13:13.726 I/mono-stdout(26222): inputBufferIndex 0 Time: 20.05.2016 09:13:13:726
after render outputBufferIndex 5 Time: 20.05.2016 09:13:13:729
05-20 09:13:13.729 I/mono-stdout(26222): after render outputBufferIndex 5 Time: 20.05.2016 09:13:13:729
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905530453 Time: 20.05.2016 09:13:13:765
05-20 09:13:13.766 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905530453 Time: 20.05.2016 09:13:13:765
inputBufferIndex 1 Time: 20.05.2016 09:13:13:767
05-20 09:13:13.767 I/mono-stdout(26222): inputBufferIndex 1 Time: 20.05.2016 09:13:13:767
Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905526852 Time: 20.05.2016 09:13:13:805
05-20 09:13:13.805 I/mono-stdout(26222): Got a RTP Frame PacketCount 2 IsComplete True IsFinal True Timestamp -905526852 Time: 20.05.2016 09:13:13:805
inputBufferIndex 2 Time: 20.05.2016 09:13:13:807
05-20 09:13:13.807 I/mono-stdout(26222): inputBufferIndex 2 Time: 20.05.2016 09:13:13:807
Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905523253 Time: 20.05.2016 09:13:13:848
05-20 09:13:13.849 I/mono-stdout(26222): Got a RTP Frame PacketCount 3 IsComplete True IsFinal True Timestamp -905523253 Time: 20.05.2016 09:13:13:848
Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -905519653 Time: 20.05.2016 09:13:13:886
05-20 09:13:13.887 I/mono-stdout(26222): Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -905519653 Time: 20.05.2016 09:13:13:886
inputBufferIndex 3 Time: 20.05.2016 09:13:13:889
05-20 09:13:13.889 I/mono-stdout(26222): inputBufferIndex 3 Time: 20.05.2016 09:13:13:889
Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -905516053 Time: 20.05.2016 09:13:13:925
05-20 09:13:13.925 I/mono-stdout(26222): Got a RTP Frame PacketCount 4 IsComplete True IsFinal True Timestamp -905516053 Time: 20.05.2016 09:13:13:925
inputBufferIndex 0 Time: 20.05.2016 09:13:13:927
05-20 09:13:13.927 I/mono-stdout(26222): inputBufferIndex 0 Time: 20.05.2016 09:13:13:927
after render outputBufferIndex 3 Time: 20.05.2016 09:13:13:931

I also checked NALUnits and Sequencenumbers, both looked okay for me (like before).
May 20, 2016 at 2:14 PM
Hi Julius,

65 is working perfect on Android.!

Whatever you did, it is was great :-)
I will try to get it working for IOS on monday and inform you what changes I succeseeded or failed with...

Thanks a lot and have a nice Weekend!
Marked as answer by juliusfriedman on 5/23/2016 at 1:47 PM
May 23, 2016 at 9:14 AM
Working great on Android.
On iOS it gets me an exception somewhere in Depacketize. I'll debug and let you know.
May 23, 2016 at 9:34 AM
Edited May 23, 2016 at 11:30 AM
It seems iOS is having a strange Problem. I added some console logs for finding the Problem:
        /// <summary>
        /// Depacketizes a single packet
        /// </summary>
        /// <param name="packet"></param>
        public virtual void Depacketize(RtpPacket packet)
        {
            Console.WriteLine("RTP frame packet Depacketize starts with sequence: {0} ", packet.SequenceNumber);
            if (Common.IDisposedExtensions.IsNullOrDisposed(packet)) return;

            int index = (short)packet.SequenceNumber;

            if (Depacketized.ContainsKey(index)) return;

            Console.WriteLine("RTP frame packet adding starts with PayloadDataSegment: {0} ", packet.PayloadDataSegment);
            Depacketized.Add(index, packet.PayloadDataSegment);
        }
I nearly always get 0 packets, sometimes 1:

2016-05-23 11:32:38.294 sks_ClientiOS[23670:4530971] RTP frame Depacketize starts with packets: 0
2016-05-23 11:32:38.298 sks_ClientiOS[23670:4530971] 11:32:38.297 ERROR: [0x16f0c7000] 49: vm_map failed: 0x4 ((os/kern) invalid argument)
2016-05-23 11:32:38.308 sks_ClientiOS[23670:4530971] System.NullReferenceException: Object reference not set to an instance of an object
at System.Runtime.InteropServices.Marshal.ReadInt64 (IntPtr ptr, Int32 ofs) [0x00012] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Runtime.InteropServices/Marshal.cs:873
at System.Runtime.InteropServices.Marshal.ReadIntPtr (IntPtr ptr, Int32 ofs) [0x00018] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Runtime.InteropServices/Marshal.cs:903
at AudioToolbox.AudioQueue.FillAudioData (IntPtr audioQueueBuffer, Int32 offset, IntPtr source, Int32 sourceOffset, nint size) [0x00000] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/src/AudioToolbox/AudioQueue.cs:497
at sks_Client.iOS.AudioDecoder.Decode (System.Byte[] data) [0x00032] in C:\Projekte\sks-Kin
kel\Repos\Apps Gateway 2.0\sks-Client\sks-Client\sks_Client.iOS\AudioDecoder.cs:133
2016-05-23 11:32:38.318 sks_ClientiOS[23670:4530971] RTP frame Depacketize starts with packets: 1
2016-05-23 11:32:38.319 sks_ClientiOS[23670:4530971] 11:32:38.319 ERROR: [0x16f0c7000] 49: vm_map failed: 0x4 ((os/kern) invalid argument)
2016-05-23 11:32:38.320 sks_ClientiOS[23670:4530971] System.NullReferenceException: Object reference not set to an instance of an object
at System.Runtime.InteropServices.Marshal.ReadInt64 (IntPtr ptr, Int32 ofs) [0x00012] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Runtime.InteropServices/Marshal.cs:873
at System.Runtime.InteropServices.Marshal.ReadIntPtr (IntPtr ptr, Int32 ofs) [0x00018] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/corlib/System.Runtime.InteropServices/Marshal.cs:903
at AudioToolbox.AudioQueue.FillAudioData (IntPtr audioQueueBuffer, Int32 offset, IntPtr source, Int32 sourceOffset, nint size) [0x00000] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/src/AudioToolbox/AudioQueue.cs:497
at sks_Client.iOS.AudioDecoder.Decode (System.Byte[] data) [0x00032] in C:\Projekte\sks-Kin
kel\Repos\Apps Gateway 2.0\sks-Client\sks-Client\sks_Client.iOS\AudioDecoder.cs:133

Happens on iOS only, not on Android. Strange...

It seems it is not the Depacketize itself, but maybe creating the 6184 Frame:

RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(frame);

often throws an Exception (iOS only).
May 23, 2016 at 11:43 AM
Because I thought, the Constructor of the RFC6184Frame does work, but maybe not add all it's packets, I changed my Code (called by OnFramechanged):
        private byte[] DepacketizeH264(RtpFrame frame)
        {
            byte[] result = new byte[0];
            try
            {
                //using (RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(frame))
                RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(frame);
                if (profileFrame.Count != frame.Count)
                {
                    //  Console.Out.WriteLine("FEHLER");
                    profileFrame = new RFC6184Media.RFC6184Frame(99);
                    foreach (RtpPacket packet in frame.Packets)
                        profileFrame.Packets.Add(packet);
                }
                profileFrame.Depacketize();
                if (profileFrame.HasDepacketized)
                {
                    result = profileFrame.Buffer.ToArray();
                }
                Media.Common.BaseDisposable.SetShouldDispose(frame, true);
            }catch (Exception e)
            {
                Console.Out.WriteLine(e);
            }
            return result;
        }
The idea was to fill packets manually for testing when there is no packet or a difference.

But this Code crashes in if (profileFrame.Count != frame.Count) it seems with this exception:

{System.NullReferenceException: Object reference not set to an instance of an object
at Media.Rtp.RtpFrame.get_Count () [0x00000] in <filename unknown>:0
at sks_Client.Manager.RTPClientManager.DepacketizeH264 (Media.Rtp.RtpFrame frame) [0x00010] in C:\Projekte\sks-Kinkel\Repos\Apps Gateway 2.0\sks-Client\sks-Client\sks_Client\Manager\RTPClientManager.cs:518 }
Coordinator
May 23, 2016 at 8:48 PM
Sorry about this, it was due to changes in the event model.

You should never receive a frame with 0 packets now.

Glad to hear performance is also better, I am trying to make thing as efficient as possible.

let me know if there are any other issues.
Marked as answer by juliusfriedman on 5/23/2016 at 1:49 PM
May 24, 2016 at 6:25 AM
Edited May 24, 2016 at 12:10 PM
Hello Julius,

thanks for the update.

Trying 112071 now when starting RTP on an Android device I get the following exception:
        public MemorySegment(long size, bool shouldDispose = true) 
            : base(shouldDispose)
        {
            if (size < 0) throw new ArgumentException("size");

            m_Length = size;

            //If the amount is unaligned then aligned it
            size += Machine.CalulcateAlignedSize(ref size);
05-24 08:22:45.794 I/mono-stdout( 6665): System.TypeInitializationException: The type initializer for 'Media.Common.Machine' threw an exception. ---> System.InvalidOperationException: Did not detect the NativePointerSize correctly.
System.TypeInitializationException: The type initializer for 'Media.Common.Machine' threw an exception. ---> System.InvalidOperationException: Did not detect the NativePointerSize correctly.
05-24 08:22:45.807 I/mono-stdout( 6665): at Media.Common.Machine..cctor () [0x00126] in C:\Projekte
et7mma-112071\Common\Classes\Machine.cs:756
at Media.Common.Machine..cctor () [0x00126] in C:\Projekte
et7mma-112071\Common\Classes\Machine.cs:756


It seems "Machine" is unknown or not set peroperly.
May 24, 2016 at 6:35 AM
If I remove the line with the machine, I get the next exception here (in the setter):
        /// <summary>
        /// Gets or Sets the unsigned 16 bit SequenceNumber field in the RtpHeader.
        /// </summary>
        public int SequenceNumber
        {
            [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.AggressiveInlining)]

            //The sequence number is stored in Netword Byte Order @ + 0x00 from the second octet (relative offset of 0x02 from the beginning of any header pointer)
            get { /*CheckDisposed();*/ return (ushort)Binary.ReadU16(SegmentToLast10Bytes.Array, SegmentToLast10Bytes.Offset, BitConverter.IsLittleEndian); }
            [System.Runtime.CompilerServices.MethodImpl(System.Runtime.CompilerServices.MethodImplOptions.AggressiveInlining)]

            set { /*CheckDisposed();*/ Binary.Write16(SegmentToLast10Bytes.Array, SegmentToLast10Bytes.Offset, BitConverter.IsLittleEndian, (ushort)value); }
        }
05-24 08:33:59.361 I/mono-stdout( 8280): System.TypeInitializationException: The type initializer for 'Media.Common.Binary' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Media.Common.Machine' threw an exception. ---> System.InvalidOperationException: Did not detect the NativePointerSize correctly.
System.TypeInitializationException: The type initializer for 'Media.Common.Binary' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Media.Common.Machine' threw an exception. ---> System.InvalidOperationException: Did not detect the NativePointerSize correctly.
at Media.Common.Machine..cctor () [0x00126] in C:\Projekte
et7mma-112071\Common\Classes\Machine.cs:756

Seems my bugfix is not perfect yet :-)
May 24, 2016 at 6:57 AM
I then added the line with the size again and just removed the line where you throw the exception "Did not detect the NativePointerSize correctly."

After that everything starts okay, and I have Sound but do not get a Picture anymore.

If it helps, in this case I am logging this:

05-24 08:54:13.576 I/mono-stdout(11361): Media.RFC3550+CommonHeaderBits@Finalize Completed
Media.RFC3550+CommonHeaderBits@Finalize Completed
[0:] Media.RFC3550+CommonHeaderBits@Finalize Completed
Media.Rtp.RtpPacket@Finalize Completed
[0:] Media.Rtp.RtpPacket@Finalize Completed
05-24 08:54:13.599 I/mono-stdout(11361): Media.Rtp.RtpPacket@Finalize Completed
Media.RFC3550+CommonHeaderBits@Finalize Completed
05-24 08:54:13.608 I/mono-stdout(11361): Media.RFC3550+CommonHeaderBits@Finalize Completed
[0:] Media.RFC3550+CommonHeaderBits@Finalize Completed
Media.RFC3550+CommonHeaderBits@Finalize Completed
[0:] Media.RFC3550+CommonHeaderBits@Finalize Completed
05-24 08:54:13.622 I/mono-stdout(11361): Media.RFC3550+CommonHeaderBits@Finalize Completed
Media.Rtp.RtpHeader@Finalize Completed
[0:] Media.Rtp.RtpHeader@Finalize Completed
05-24 08:54:13.629 I/mono-stdout(11361): Media.Rtp.RtpHeader@Finalize Completed
Media.Rtp.RtpPacket@Finalize Completed
[0:] Media.Rtp.RtpPacket@Finalize Completed
05-24 08:54:13.637 I/mono-stdout(11361): Media.Rtp.RtpPacket@Finalize Completed
Media.Rtp.RtpHeader@Finalize Completed
[0:] Media.Rtp.RtpHeader@Finalize Completed
05-24 08:54:13.644 I/mono-stdout(11361): Media.Rtp.RtpHeader@Finalize Completed
Media.Rtp.RtpHeader@Finalize Completed
[0:] Media.Rtp.RtpHeader@Finalize Completed
05-24 08:54:13.652 I/mono-stdout(11361): Media.Rtp.RtpHeader@Finalize Completed
Media.RFC3550+CommonHeaderBits@Finalize Completed
[0:] Media.RFC3550+CommonHeaderBits@Finalize Completed
05-24 08:54:13.660 I/mono-stdout(11361): Media.RFC3550+CommonHeaderBits@Finalize Completed
Media.Rtp.RtpPacket@Finalize Completed
[0:] Media.Rtp.RtpPacket@Finalize Completed
05-24 08:54:13.666 I/mono-stdout(11361): Media.Rtp.RtpPacket@Finalize Completed
Media.Rtp.RtpFrame@Finalize Completed
05-24 08:54:13.684 I/mono-stdout(11361): Media.Rtp.RtpFrame@Finalize Completed
[0:] Media.Rtp.RtpFrame@Finalize Completed
Media.Rtp.RtpHeader@Finalize Completed
May 24, 2016 at 12:13 PM
Hi Julius,

I also tested on iOS now. the machine errror does not appear, but just like in 65 I get audio, but not a Video anymore.
It seems I am only getting nalu types 1. I Need at least one time a 7 and a 8 to configure sps and pps.

The iOS is working with 88 both Video and vidirektional Audio, but I guess the Performance would be better with one of the later Versions.
Android is working with 65 Video and bidirektional Audio pretty good.

Thanks for your Support :-)
Coordinator
May 24, 2016 at 2:06 PM
Edited May 24, 2016 at 2:09 PM
Can you step the Machine constructor and let me know what the exception is? I have added more information in the exception to ensure I can debug if necessary.

I would imagine that ARM I am returning 32 bit on accident in some cases :) Can you tell me if IsArm works as well as isX86 and isX64?

Just to be clear, 65 works well with audio and video on both iOS and Android? or was that 88?

Also can you please test the latest version and let me know what you find.

Thanks again for all of your help.

I will be moving on to further improvements in the RtpFrame hopefully this week once all of the regression from recent changes is sorted out.

Overall each version should be a more stable and performant library than previous versions, lets hope that is the case :)

To give you an idea of some of the next changes I am thinking about:

OnInterleavedData will become OnOutOfBand(byte[], int offset, int length)

OnOutOfBand(RtpPacket) will be added

OnOutOfBand(RtcpPacket) will be added

AutomaticallyCreateContexts option will be added

OnMarkerPacket(RtpFrame) will be added (will fire on on marker packets)

I will also probably add another frame, NextFrame so that I can increase the chance that late packets arrive and are delivered to sources.

All that will give the consumers of the library the chance to reduce the amount of events they consume while still providing the overall function of the library.

I am also thinking about an OnLoss event but I would need to further modify the TransportContext, such a concept would definitely be useful for NACK / re-transmission.

Let me know what you find!
Marked as answer by juliusfriedman on 5/24/2016 at 7:06 AM
May 25, 2016 at 6:18 AM
Edited May 25, 2016 at 6:22 AM
Hi Julius,

we are using Visual Studio 2015 with Xamarin, and Building Android with "Any CPU" and iOS with "iPhone" Build Targets. I will try to find out.

Currently Android works very well with 112065, iOS with older 88, both with Audio and Video.

Retransmition could be interesting and changing Video Quality during Playback (if the Connection Quality changes) will be interesting in our case.

And I will try with a Windows 10 phone next weeks to get it working, I will try to use your Decoding and Rendering in this case also :-)

Thanks!
May 25, 2016 at 6:47 AM
Edited May 25, 2016 at 7:08 AM
Testing on my Nexxus with Android 5 and your 112074, it starts again without an exception. But I see no Video (Audio is okay). I am still getting Video data, but it seems, in the RFC6184Frame ContainedUnitTypes there is always only "1".
I would at least expect a 7 and 8 (SPS and PPS) one time starting and each second again or something like that.
And in the older versions at least on Android I had at lot of 28s.

I will test it later with Android 6 Motorola, where the exception occured yesterday and tell you, if I get the exception there still.
May 25, 2016 at 11:37 AM
Edited May 25, 2016 at 11:37 AM
I also tested on my Android 6.0 Motorola now, where yesterday the exception with the machine pointer size occured.

It now connects and activates the rtpclient without an exception, but no Video, just as descriped above.

I will be back on monday, have a nice week!

Thanks and see you :-)
Marked as answer by juliusfriedman on 5/25/2016 at 4:59 AM
Coordinator
May 25, 2016 at 11:59 AM
Thank you for the feedback.

What was the last version for you that showed Audio and Video with good performance?

I will be posting another update today.
Marked as answer by juliusfriedman on 5/25/2016 at 4:59 AM
May 25, 2016 at 12:18 PM
Edited May 25, 2016 at 12:19 PM
Android 65 has really good Performance. iOS still working with older 88 has okay Performance.
Do I have to set Compiler / build directives when Building your library?
Coordinator
May 25, 2016 at 12:44 PM
In general no.

If the code is configured for 64 bit but prefers 32 bit I still have my own optimizations to make in Machine to ensure I respect that.

My code should automatically detect and use the best possible functions for the processor it detects when the static constructors run.

Finally, what prevents iOS from running 65? Just the Machine exception? If you comment out that line does it run? If not why?

Thank you again for your help with testing!

Hopefully the next release or two should get things 100% stable and working on all platforms.

Then I can finally resume my development on RtpFrame etc.
Marked as answer by juliusfriedman on 5/25/2016 at 5:44 AM
May 25, 2016 at 1:23 PM
The machine error was only on Android, not on iOS. In 71 only.

iOS gets no Video. Audio seems to be fine. No exceptions or something. iOS gets both AV in older 88.
Coordinator
May 25, 2016 at 1:35 PM
Weird, is there a version which functions on iOS and Android e.g. 88?

I think it has to do with the amount of memory the device has but I need to confirm a few things, either ways I will hopefully make some changes which should prevent it from mattering in all but the most extreme cases.

I am just trying to isolate a few potential issues with Finalization which seems to be the cause of the video packets appearing to not display, they get collected before then can be used. Can you watch the Debug window when you don't get Video or Audio and tell me if you see statements indicating "Finalize" and what classes?

This shouldn't be happening with SetShouldDispose(x, false, false) but in certain conditions it can potentially be a race against the GC so I have added a SuppressFinalizerClass to help with this.

What I imagine is happening is that the data rate of the source is higher than the decoding rate, the frame gets disposed just before it's handled.

If you could also try setting ShouldDipose(frame, false, false) immediately in the OnRtpFrame event handler and then call SetShouldDispose(frame, true, true) when your done with it, it SHOULD allow the video to be displayed without the finalization issues.

Overall though your application layout should be as it was without worrying about the frames being disposed, I will narrow down the finalization issues very soon hopefully.

What is interesting to me is that you don't seem to experience them in 88 or 65, you seem to experience them in between those versions when you say something about clicking in the audio and artifacts in the video, that symptom is also an indication of GC.
Marked as answer by juliusfriedman on 5/31/2016 at 2:00 PM
May 30, 2016 at 10:57 AM
Hi Julius,

the latest 112085 showed a green Screen for a second, then a part of the Video stream. With many artefacts and the Audio was delayed but there.

If it helps I can post the Code we are currently using. I will have less time to try new versions from now on, Need to do important other stuff, but will have a look and test something from time to time.

Have a nice week :-)
May 30, 2016 at 2:25 PM
PS: I will test again tomorrow on another Android smartphone, I had some strange Problems here, maybe there is something wrong with my test-phone....

We also want to add Windows phone now. I opened a new thread under "Suggestions". Thanks :-)
May 31, 2016 at 6:17 AM
I also tried 112085 on another Android device. In this case no Picture and sometimes Sound bidirektinal.

Interesting maybe this log, sometimes appearing now:

05-31 08:12:09.431 W/AudioTrack(19051): releaseBuffer() track 0x7af58968 disabled due to previous underrun, restarting

For Android we are using the audiotrack to Play back Audio data.
Coordinator
May 31, 2016 at 9:01 PM
I will check into the disposable but things seem fairly stable in in the last updates I posted.

Maybe I can get a test version of your application running on my one of my test handsets (Samsung) to see exactly what your talking about..

-Julius
Marked as answer by juliusfriedman on 5/31/2016 at 2:01 PM
Jun 6, 2016 at 6:54 AM
Hi Julius,

I just downloaded 112103 and tested it on two Android devices. The Picture Shows only some few Little blocks again, and the Audio is a bit broken and delayed, but it is there. when we are using 112065 with the same Code it works good. Sequencenumbers etc looked okay on first view. Just to let you know, to get it working on a Windows 10 phone is much more important for us at this Moment.

If you Need more info please let me know.

06-06 08:48:44.331 I/mono-stdout( 3682): 131096693243264640@SessionDescription
06-06 08:48:44.331 I/mono-stdout( 3682): v=0
06-06 08:48:44.331 I/mono-stdout( 3682): o=root 58766182 58766182 IN IP4 192.168.1.132
06-06 08:48:44.332 I/mono-stdout( 3682): s=Asterisk PBX GIT-master-60a15fe
06-06 08:48:44.332 I/mono-stdout( 3682): c=IN IP4 192.168.1.132
06-06 08:48:44.332 I/mono-stdout( 3682): b=CT:384
06-06 08:48:44.332 I/mono-stdout( 3682): t=0 0
06-06 08:48:44.332 I/mono-stdout( 3682): m=audio 22762 RTP/AVP 0 8
06-06 08:48:44.332 I/mono-stdout( 3682): a=rtpmap:0 PCMU/8000
06-06 08:48:44.333 I/mono-stdout( 3682): a=rtpmap:8 PCMA/8000
06-06 08:48:44.333 I/mono-stdout( 3682): a=maxptime:150
06-06 08:48:44.333 I/mono-stdout( 3682): a=sendrecv
06-06 08:48:44.333 I/mono-stdout( 3682): m=video 7610 RTP/AVP 99 34
06-06 08:48:44.333 I/mono-stdout( 3682): a=rtpmap:99 H264/90000
06-06 08:48:44.333 I/mono-stdout( 3682): a=rtpmap:34 H263/90000
06-06 08:48:44.333 I/mono-stdout( 3682): a=sendrecv
06-06 08:48:44.333 I/mono-stdout( 3682):
06-06 08:48:44.333 I/mono-stdout( 3682): @Finalize Completed
[0:] 131096693243264640@SessionDescription
v=0
o=root 58766182 58766182 IN IP4 192.168.1.132
s=Asterisk PBX GIT-master-60a15fe
c=IN IP4 192.168.1.132
b=CT:384
t=0 0
m=audio 22762 RTP/AVP 0 8
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=maxptime:150
a=sendrecv
m=video 7610 RTP/AVP 99 34
a=rtpmap:99 H264/90000
a=rtpmap:34 H263/90000
a=sendrecv

@Finalize Completed
AudioData received 160 Audio Bytes
06-06 08:48:44.360 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.375 W/AudioTrack( 3682): releaseBuffer() track 0x632b5200 disabled due to previous underrun, restarting
06-06 08:48:44.376 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.385 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.393 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.401 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.409 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.417 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.454 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.463 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.473 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
Got a RTP Frame PacketCount 12 IsComplete False IsFinal True Seqnumber 52813 Timestamp 0 Time: 06.06.2016 08:48:44:466
06-06 08:48:44.482 I/mono-stdout( 3682): Got a RTP Frame PacketCount 12 IsComplete False IsFinal True Seqnumber 52813 Timestamp 0 Time: 06.06.2016 08:48:44:466
06-06 08:48:44.482 I/mono-stdout( 3682): Video is starting
Video is starting
AudioData received 160 Audio Bytes
06-06 08:48:44.483 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52815 Timestamp 3604 Time: 06.06.2016 08:48:44:497
06-06 08:48:44.498 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52815 Timestamp 3604 Time: 06.06.2016 08:48:44:497
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52817 Timestamp 7200 Time: 06.06.2016 08:48:44:503
06-06 08:48:44.504 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52817 Timestamp 7200 Time: 06.06.2016 08:48:44:503
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52819 Timestamp 10807 Time: 06.06.2016 08:48:44:508
06-06 08:48:44.509 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52819 Timestamp 10807 Time: 06.06.2016 08:48:44:508
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52821 Timestamp 14400 Time: 06.06.2016 08:48:44:515
BUSY WORKER
06-06 08:48:44.516 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52821 Timestamp 14400 Time: 06.06.2016 08:48:44:515
06-06 08:48:44.517 I/mono-stdout( 3682): BUSY WORKER
BUSY WORKER
06-06 08:48:44.521 I/mono-stdout( 3682): BUSY WORKER
AudioData received 160 Audio Bytes
06-06 08:48:44.523 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52823 Timestamp 18001 Time: 06.06.2016 08:48:44:523
BUSY WORKER
06-06 08:48:44.529 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52823 Timestamp 18001 Time: 06.06.2016 08:48:44:523
AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
06-06 08:48:44.531 I/mono-stdout( 3682): BUSY WORKER
06-06 08:48:44.532 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
06-06 08:48:44.545 I/mono-stdout( 3682): BUSY WORKER
BUSY WORKER
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52825 Timestamp 21603 Time: 06.06.2016 08:48:44:550
06-06 08:48:44.551 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52825 Timestamp 21603 Time: 06.06.2016 08:48:44:550
BUSY WORKER
06-06 08:48:44.553 I/mono-stdout( 3682): BUSY WORKER
06-06 08:48:44.555 W/ACodec ( 3682): cannot describe color format 0x13 = 19 with stride=0 and sliceHeight=0
06-06 08:48:44.558 I/mono-stdout( 3682): BUSY WORKER
BUSY WORKER
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52827 Timestamp 25201 Time: 06.06.2016 08:48:44:562
06-06 08:48:44.562 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52827 Timestamp 25201 Time: 06.06.2016 08:48:44:562
BUSY WORKER
AudioData received 160 Audio Bytes
06-06 08:48:44.566 I/mono-stdout( 3682): BUSY WORKER
[0:] AudioData received 160 Audio Bytes
06-06 08:48:44.570 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
BUSY WORKER
06-06 08:48:44.577 I/mono-stdout( 3682): BUSY WORKER
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52829 Timestamp 28802 Time: 06.06.2016 08:48:44:580
AudioData received 160 Audio Bytes
06-06 08:48:44.581 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52829 Timestamp 28802 Time: 06.06.2016 08:48:44:580
06-06 08:48:44.582 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
Thread started: #14
AudioData received 160 Audio Bytes
06-06 08:48:44.612 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
BUSY WORKER
06-06 08:48:44.621 I/mono-stdout( 3682): BUSY WORKER
06-06 08:48:44.622 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
BUSY WORKER
06-06 08:48:44.638 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
06-06 08:48:44.638 D/Mono ( 3682): GC_BRIDGE waiting for bridge processing to finish
06-06 08:48:44.639 I/mono-stdout( 3682): BUSY WORKER
06-06 08:48:44.639 D/Mono ( 3682): GC_OLD_BRIDGE num-objects 146 num_hash_entries 156 sccs size 156 init 0.00ms df1 0.47ms sort 0.34ms dfs2 0.77ms setup-cb 0.14ms free-data 0.10ms links 0/0/0/0 dfs passes 0/0
06-06 08:48:44.640 D/Mono ( 3682): GC_MINOR: (Nursery full) pause 6.62ms, total 9.28ms, bridge 0.00ms promoted 2064K major 2064K los 451K
[0:] AudioData received 160 Audio Bytes
AudioData received 160 Audio Bytes
06-06 08:48:44.647 I/mono-stdout( 3682): AudioData received 160 Audio Bytes
[0:] AudioData received 160 Audio Bytes
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52831 Timestamp 32403 Time: 06.06.2016 08:48:44:648
BUSY WORKER
06-06 08:48:44.654 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52831 Timestamp 32403 Time: 06.06.2016 08:48:44:648
inputBufferIndex 2 Time: 06.06.2016 08:48:44:657
BUSY WORKER
06-06 08:48:44.656 I/mono-stdout( 3682): BUSY WORKER
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52833 Timestamp 36003 Time: 06.06.2016 08:48:44:661
06-06 08:48:44.657 I/mono-stdout( 3682): inputBufferIndex 2 Time: 06.06.2016 08:48:44:657
06-06 08:48:44.659 I/mono-stdout( 3682): BUSY WORKER
BUSY WORKER
06-06 08:48:44.661 I/mono-stdout( 3682): Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52833 Timestamp 36003 Time: 06.06.2016 08:48:44:661
06-06 08:48:44.663 I/mono-stdout( 3682): BUSY WORKER
BUSY WORKER
Got a RTP Frame PacketCount 2 IsComplete False IsFinal True Seqnumber 52835 Timestamp 39602 Time: 06.06.2016 08:48:44:670
Jun 6, 2016 at 6:59 AM
We are currently using this Code:
        internal void OnSourceFrameChanged(object sender, RtpFrame frame = null, RtpClient.TransportContext tc = null, bool final = false)
        {
            if (frame != null)
            {
                if ((frame.PayloadType == Globals.H264PlayloadType) && final)
                {
                    if (Globals.DoDetailedLog)
                        ClientLogger.Instance.Log(string.Format("Got a RTP Frame PacketCount {0}  IsComplete {1}  IsFinal {2}  Seqnumber {3}  Timestamp {4}  Time: {5}:{6}", frame.Count, frame.IsComplete, final, frame.HighestSequenceNumber, frame.Timestamp, DateTime.Now.ToString(), DateTime.Now.Millisecond));

                    if (!isVideoOn)
                    {
                        isVideoOn = true;
                        if (RefreshUI != null)
                            RefreshUI("Video is starting", Xamarin.Forms.Color.Green);
                    }
                    Media.Common.BaseDisposable.SetShouldDispose(frame, false);
                    _videoQueue.Add(frame);
                    if (Globals.DoDetailedLog && frame.IsMissingPackets)
                    {
                        ClientLogger.Instance.Log(string.Format("Attention! Data loss in RTPClient OnSourceFrameChanged at Time: {0}:{1} ", DateTime.Now.ToString(), DateTime.Now.Millisecond));
                    }
                }
                else if ((frame.PayloadType == Globals.MuLawPlayloadType) && final)//tc.MediaDescription.MediaType == MediaType.audio)
                {
                    if (!isAudioOn)
                    {
                        isAudioOn = true;
                        if (RefreshUI != null)
                            RefreshUI("Audio is starting", Xamarin.Forms.Color.Green);
                    }
                    Media.Common.BaseDisposable.SetShouldDispose(frame, false);
                    _audioQueue.Add(frame);
                    _audioQueue.Add(DepacketizeMuLaw(frame));
                }
            }
            else
            {
                ClientLogger.Instance.Log("NULL - RTP frame received");
            }
        }
        
        private void _videoQueue_DataAvailable(object sender, ThreadingQueueDataEventArgs<RtpFrame> e)
        {
            if (VideoDataReceived != null)
                VideoDataReceived(this, new DataEventArgs() { Data = DepacketizeH264(e.Item) });
            Media.Common.BaseDisposable.SetShouldDispose(e.Item, true);
        }

        private byte[] DepacketizeH264(RtpFrame frame)
        {
            byte[] result = new byte[0];

            using (RFC6184Media.RFC6184Frame profileFrame = new RFC6184Media.RFC6184Frame(frame))
            {
                profileFrame.Depacketize();
                if (profileFrame.HasDepacketized)
                {
                    result = profileFrame.Buffer.ToArray();
                }
            }
            return result;
        }
VideoDataReceived just gives it to our h.264 Decoder. The _videoQueue is a own Queue which works data in it's own thread.

If we should try something else, please let us know. If you Need further info the same :-)
Coordinator
Jun 6, 2016 at 2:31 PM
It seems your running in DEBUG mode.

DEBUG mode causes a bit slower execution because it forces each instance created to be tracked for dispose via the AppDomain.

This is just for debugging purposes to test the amount of GC which occurs and memory used.

If you run in RELEASE mode you will probably have much less issues :)

Please verify ad let me know!
Marked as answer by juliusfriedman on 6/6/2016 at 7:31 AM
Jun 6, 2016 at 3:04 PM
I tried both, also release had the same Problems, maybe it moved that few blocks a Little faster :-)
Coordinator
Jun 6, 2016 at 4:10 PM
What you are describing is due to GC...

Try GC.KeepAlive(frame) and see if you get video.

What I imagine is that the fragmentation of the packets being persisted is causing the GC to want to compact more frequently.

You can try 'AddMemoryPressure(Int64)'

Take a look also at this article: CodeProject Performance Tips For Android

I find it hard to believe that 65 works and these versions do not, as the event model is the same in those versions.

If you see you are calling 'RefreshUI' during the event handler and before you specify the packet should not be disposed.

Then you set ShouldDispose = false after you leave that method....

It seems to me that you should first call SetShouldDispose to ensure that the GC doesn't step in and finalize the object between those calls.
Marked as answer by juliusfriedman on 6/6/2016 at 9:10 AM
Jun 7, 2016 at 8:03 AM
Hello Julius,

I downloaded the new Version 112128. Thanks a lot for it!

Tried with Android: 112065 working good in our source, the new one has a delayed Audio and only Fragments of Video.

Audio does work but is delayed, I get a green Picture and then again only some Fragments.

I added GC.KeepAlive(frame) to the top of the FrameChanged but it does no difference, a release build also does no big difference.
I removed all SetShouldDispose but it also does no difference at the start, later I seem to get crashes then.

Our idea with the SetShoulddispose was to use our own Queue which works in it's own thread. We want to Keep the Frame, add it to the Queue, then dispose it.
I will post the Code of it next to give you an idea. Works great with 112065. The Code where we feed it and receive it is the same as above.

I realized before in our H.264 Decoder we got a OutputFormatChanged once, now we do not seem to get it anymore:
                    if (outputBufferIndex == (int)MediaCodecInfoState.OutputBuffersChanged) // -3
                    {
                        //_codec.GetOutputBuffers();
                        if (Globals.DoDetailedLog)
                            ClientLogger.Instance.Log("decoder output buffers changed");
                    }
                    else if (outputBufferIndex == (int)MediaCodecInfoState.OutputFormatChanged) // -2
                    {
                        MediaFormat formatNew = _codec.OutputFormat;
                        //mediaformat changed
                        if (Globals.DoDetailedLog)
                            ClientLogger.Instance.Log("decoder mediaformat changed: "+formatNew.ToString());
                        _codec.Configure(formatNew, _surface, null, 0);
                    }
Tried with iOS: 111988 (old) working good in our source, the new one has a delayed Audio no Video at all.

For iOS we are still using the very old RTP and work with the Byte Array instead of the Frame still. We did not get it working any other way until today.

I tried just to replace the assemblies, and then tried to use it the same way as Android using the Frame and our Queue. In both ways I did not get Video (maybe the Decoder is less stable and quits earlier, Android had at least some Fragments) but got Audio.



I think something could be wrong with the stream still. Or it is our Queue, which works good with our Queue in the old 65 Version, but not anymore because of Memory optimizations or something like this.


Thanks a lot for your help!
Jun 7, 2016 at 8:07 AM
Here is the Queue we are using for Android (and want to use it for iOS too).
It is feed from Framechanged and gives data to the Decoder with _videoQueue_DataAvailable . Both codes above.

The idea is to run this Queue in ist own thread to prepare the Decoder.
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;



namespace sks_Client
{
    public class ThreadingQueue<T> where T : class
    {
        public event EventHandler ErrorOccured;
        public event EventHandler QueueFinished;
        public event EventHandler<ThreadingQueueDataEventArgs<T>> DataAvailable;

        private ManualResetEvent _mre = new ManualResetEvent(true);
        private object _mreLock = new object();
        private Queue<T> _queue = new Queue<T>();
        
        private bool _isRunning;
        public bool IsRunning
        {
            get { return _isRunning; }
        }

        public int WakeInterval { get; set; }

        public ThreadingQueue()
        {
            WakeInterval = 500;
            Start();
        }

        public void Add(T item)
        {
            lock (_mreLock)
            {
                _queue.Enqueue(item);
                _mre.Set();
                if (_queue.Count > 10 && Globals.DoDetailedLog)
                    Console.Out.WriteLine("BUSY WORKER");
            }
        }

        public void Add(T[] items)
        {
            lock (_mreLock)
            {
                foreach(T item in items)
                    _queue.Enqueue(item);
                _mre.Set();
                if (_queue.Count > 10 && Globals.DoDetailedLog)
                    Console.Out.WriteLine("BUSY WORKER");
            }
        }

        public void Start()
        {
            if (!_isRunning)
            {
                _isRunning = true;
                new Thread(new ThreadStart(Worker)).Start();
            }
        }

        public void Stop()
        {
            if (!_isRunning)
            {
                _isRunning = false;
            }
        }

        private void Worker()
        {
            while (_isRunning)
            {
                T item = null;
                lock (_mreLock)
                {
                    if (_queue.Count > 0)
                        item = _queue.Dequeue();
                    if (_queue.Count == 0)
                        _mre.Reset();
                }
                if (item != null)
                {
                    try
                    {
                        if (DataAvailable != null)
                            DataAvailable(this, new ThreadingQueueDataEventArgs<T>() { Item = item });
                    }
                    catch (Exception e)
                    {
                        try
                        {
                            if (ErrorOccured != null)
                                ErrorOccured(this, new ThreadingQueueErrorEventArgs() { Error = e });
                        }
                        catch (Exception) { }
                    }
                }

                _mre.WaitOne(WakeInterval);
            }
            try { 
            if (QueueFinished != null)
                QueueFinished(this, EventArgs.Empty);
            }
            catch (Exception e)
            {
                try
                {
                    if (ErrorOccured != null)
                        ErrorOccured(this, new ThreadingQueueErrorEventArgs() { Error = e });
                }
                catch (Exception) { }
            }
        }

        ~ThreadingQueue()
        {
            Stop();
        }
    }

    public class ThreadingQueueDataEventArgs<T> : EventArgs
    {
        public T Item { get; set; }
    }

    public class ThreadingQueueErrorEventArgs : EventArgs
    {
        public Exception Error { get; set; }
    }
}
If you see an error or I should just try something please let me know.
Coordinator
Jun 7, 2016 at 11:48 AM
Edited Jun 7, 2016 at 9:56 PM
There are QUITE a few things I see with this implementation...

Why not use the ConcurrentQueue or my ConcurrentLinkedQueue?

Lets also remove the Queue from the test, just try to generate a h.264 file and see if you can play the results, I have a feeling there is something wrong with your SDP parsing code since I made some changes recently, that may be why you don't get the Format Changed message.

Let me know if you have time to debug this as I am rather interested in how 65 work but the later versions don't...

You are also welcome to try ThreadEvents on the RtpClient but I doubt that will help you in this particular case...

Keep in mind a single frame comes 10 times at 10 FPS, so sleeping for half of one seconds drops 5 frames.... This is likely what is happening when the count is > 10 and you don't add or remove items.

@ TryAdd => Console.Out.WriteLine("BUSY WORKER"); is pointless because you already entered the lock

When you are in the 'Thread' You lock when you before you check the count and then you reset the MRE but within Add you only Set the MRE within a lock...

You should really be using Interlocked or Volatile Read if you plan on having multiple consumers or producers, if your only having a single producer and consumer you don't need to but then you also don't need locking either.

Your events use structure and thus are passed by value copying memory on each event which is only shortly lived.

The ! Can and will cause more issues than you want in a certain memory models and processors, use false == especially in threading scenarios or better yet false.Equals.

Probably more...

Just use ConcurrentLinkedQueue or ConcurrentQueue and you will be a lot better off.

Your lucky this worked at all in any version and essentially explains the weirdness of the errors and the symptoms in the application :p

After you make those changes you should be in good shape.
Marked as answer by juliusfriedman on 6/7/2016 at 5:13 AM
Jun 8, 2016 at 7:22 AM
Hi Julius,

the Problem is not, or not only our Queue.

I downloaded 112129. I copied the assemblies and started on my Android device. Again I get only some Fragments of the Video (blocky small parts) and a delayed Audio.
Then I removed the Queue totally. But after it, I had the same result. There must be something else wrong, maybe there is something missing in the stream now...

My Code now:
        internal void OnSourceFrameChanged(object sender, RtpFrame frame = null, RtpClient.TransportContext tc = null, bool final = false)
        {
            if (frame != null)
            {
                if ((frame.PayloadType == Globals.H264PlayloadType) && final)
                {
                    if (Globals.DoDetailedLog)
                        ClientLogger.Instance.Log(string.Format("Got a RTP Frame PacketCount {0}  IsComplete {1}  IsFinal {2}  Seqnumber {3}  Timestamp {4}  Time: {5}:{6}", frame.Count, frame.IsComplete, final, frame.HighestSequenceNumber, frame.Timestamp, DateTime.Now.ToString(), DateTime.Now.Millisecond));

                    if (VideoDataReceived != null)
                        VideoDataReceived(this, new DataEventArgs() { Data = DepacketizeH264(frame) });

                    if (Globals.DoDetailedLog && frame.IsMissingPackets)
                    {
                        ClientLogger.Instance.Log(string.Format("Attention! Data loss in RTPClient OnSourceFrameChanged at Time: {0}:{1} ", DateTime.Now.ToString(), DateTime.Now.Millisecond));
                    }
                }
                else if ((frame.PayloadType == Globals.MuLawPlayloadType) && final)//tc.MediaDescription.MediaType == MediaType.audio)
                {
                    if (AudioDataReceived != null)
                        AudioDataReceived(this, new DataEventArgs() { Data = DepacketizeMuLaw(frame) });
                }
            }
            else
            {
                ClientLogger.Instance.Log("NULL - RTP frame received");
            }
        }
        
VideoDataReceived is an Event that gives it directly to the Android H.264 Decoder, which works with the old 65 Version this way, there is nothing changed.
AudioDataReceived is the Event that gives it to our own MulawDecoder, it works with both versions, but with the new Version there is more delay.
Jun 8, 2016 at 8:03 AM
Addon for iOS:

If somebody else wants to use iOS, he or you Need to do some changes in order to get it work with the rtpclient:

A Suggestion for showing the error that happens in socket.Connect:
//Assign the RemoteRtp EndPoint and Bind the socket to that EndPoint
                    try
                    {
                        RtpSocket.Connect(RemoteRtp = remoteRtp);
                    }catch(Exception error){                        
                        System.Console.WriteLine(String.Format("LocalRtp Adress {0} Port {1} RemoteRtp Adress {2} Port {3}", localRtp.Address, localRtp.Port, remoteRtp.Address, remoteRtp.Port), error);
                        RemoteRtp = remoteRtp;
                    }
This way it works in iOS, the RtpSocket.Connect does Crash, but Setting the RemoteRtp works.

There will be also be a Problem with the localIP, we got it working this way:

In RTPClient FromSessionDescription:
            IPAddress remoteIp = IPAddress.Parse(connectionLine.Host), localIp = Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(remoteIp.AddressFamily);
            localIp = IPAddress.Parse("0.0.0.0"); // todo beh
If you do These changes, you get the Connection with the rtpclient working on an iOS device.

And then in our case there is the same Problem as on Android. We have working Audio with a delay, and no Video Shows up.
Coordinator
Jun 8, 2016 at 11:55 AM
That's a pretty weird fix...

Are you sure that is how it should be fixed?

What is the Local Endpoint of the socket and the Remote Endpoint?

I would like to maybe debug this with you and see what's going on.

I can't see why any IP would be the preferred use unless you don't have access to your IPAddress or NetworkInterface .

I'll be around, let me know.

Also produce a h264 file and capture the traffic when you do so, send me the h264 file and the capture and I will do a difference on the ouput and see if I can't determine something else.

Thanks again and we will sort this out one way or the other without version 65 :)
Marked as answer by juliusfriedman on 6/8/2016 at 4:55 AM
Jun 8, 2016 at 12:14 PM
On iOS the local endpoint seems to be 127.0.0.1 (localhost) in this case.

It works with our Workaround, maybe there is a better solution :-)
Coordinator
Jun 8, 2016 at 12:32 PM
So if your using Mobile and not wifi this will essentially switch to whatever interface it can resolve the destination on?

That's fine but I am wondering if it's not better to bind on your external IPAddress if you wanted that behavior.

You should also be able to bind and connect from the wifi interface which may be where the 0.0.0.0 comes from because in Ios it's not assigned or something.

I could add the try and catch but I don't think the fallback to 0.0.0.0 is appropriate because it indicates any IP bound and causes routing tables to be loaded and resolved etc, 127.0.0.1 I would be more comfortable with as it will not do that and always be resolving to localhost but I don't know if the absence of DNS there makes any difference honestly.

Finally, I guess I could add some notes for that, what is the exact exception you get so that I can see if it makes more sense and where to handle up to incase there are different scenarios.

Lastly, quick question on Ios, someone in another thread is asking how to use the ios decoder, apparently you can't use MediaCodec in Ios and also AvToolkit only works with filestream or uri so there's a gap.

I told him to mux the stream to mp4 and pass it in and I haven't heard back but I am just curious if that's correct and there's no decoder than how are you testing on ios?
Marked as answer by juliusfriedman on 6/8/2016 at 5:32 AM
Coordinator
Jun 8, 2016 at 3:34 PM
Edited Jun 8, 2016 at 3:36 PM
Eric...

FromSessionDescription uses GetFirstUnicastIPAddress.

Media.Common.Extensions.NetworkInterface.NetworkInterfaceExtensions.GetNetworkInterface

Is definitely what you want it to use there and not GetFirstUnicastIPAddress, this is because it's conceivable that you could have two wireless interfaces or even an Ethernet connection through the data port.

I use that because I want the first address I can find which matches the address family of RemoteIp,AddressFamily.

You should really be creating your socket and passing to the function to avoid this error totally.

Doesn't this look better?
System.Net.NetworkInformation.NetworkInterface localInterface;

            if (false.Equals(existingSocket.Equals(null)) && existingSocket.IsBound)
            {
                //This interface should be the interface you plan on using for the Rtp communication
                localIp = Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(remoteIp.AddressFamily, out  localInterface);
            }
            else
            {
                //Use the localIp of the exsisting socket.
                localIp = ((System.Net.IPEndPoint)existingSocket.LocalEndPoint).Address;
            }
You could always pass in a socket and not have to worry about the Connect 'Patch' also....

I could also add an option to FromSessionDescription which allows a NetworkInterface but the semantic is the same.
Marked as answer by juliusfriedman on 6/8/2016 at 8:36 AM
Jun 9, 2016 at 6:00 AM
Edited Jun 9, 2016 at 6:01 AM
Hi Julius,

some info for your iOS user:
let him read here, that should help a lot:

http://stackoverflow.com/questions/29525000/how-to-use-videotoolbox-to-decompress-h-264-video-stream/
http://stackoverflow.com/questions/26035380/how-avsamplebufferdisplaylayer-displays-h-264

I will try your Suggestion for connecting to iOS, thank you.

My main Problems are to get Windows phone running or to even get a start there, and it would be good to have a newer Version running again at all :-)
Jun 9, 2016 at 6:59 AM
Edited Jun 9, 2016 at 7:15 AM
Hi Julius,

the new Version 1121542 crashes when starting the rtpclient.

in this line

RTPClient = Media.Rtp.RtpClient.FromSessionDescription(sessionDescription, rtpPort: 0, rtcpEnabled: false);

I get

{System.NullReferenceException: Object reference not set to an instance of an object
at Media.Rtp.RtpClient.FromSessionDescription (Media.Sdp.SessionDescription sessionDescription, Media.Common.MemorySegment sharedMemory, Boolean incomingEvents, Boolean rtcpEnabled, System.Net.Sockets.Socket existingSocket, Nullable1 rtpPort, Nullable1 rtcpPort, Int32 remoteSsrc, Int32 minimumSequentialRtpPackets, Boolean connect, System.Action`1 configure) [0x00026] in c:\Projekte\net7mma-112142\Rtp\RtpClient.cs:255
at sks_Client.Manager.RTPClientManager.Init (Independentsoft.Sip.Sdp.SessionDescription sdp) [0x0005e] in C:\Projekte\sks-Kinkel\Repos\Apps Gateway 2.0\sks-Client\sks-Client\sks_Client\Manager\RTPClientManager.cs:107 }

Debugging Shows in your RTPClient around line 255 existingSocket == null and crashes in this line with a strange construction ;-)

if (false.Equals(existingSocket.Equals(null)) && existingSocket.IsBound)

If I change to
if (existingSocket!=null && existingSocket.IsBound)

it crashes in the else part below, because existingSocket is used and null.

When Setting this back to the source of the older Version again I only get some few small fragements instead of a Picture and delayed Audio.

If I just undo the changes and have my 65 assemblies back, it works. With or without my threading Queue...
Coordinator
Jun 9, 2016 at 12:07 PM
Edited Jun 9, 2016 at 12:23 PM
Are you passing a socket or no?

Ill work around the issue with Equals and IsBound buy I need to understand what the goals is.

If you pass a socket it should already have the local ip and remote Ip.

65 shouldn't be any different from the current version.

The major differences in that version only relate to the RtspClient your not even using.

I'll do a compare against 65 today in an attempt to hopefully realize something different.

Windows 10 can be targeted! You can't target the portable subset, nor should you want to because of other known Issues with it at the current time as well. E.g. Marshal is missing for some unknown reason even though it's available from windows 8 phone and windows 10 phone directly it's not available under the Portable subset.


P.s. Thanks for the links I'll take a look and pass the info on!
Marked as answer by juliusfriedman on 6/9/2016 at 5:07 AM
Jun 9, 2016 at 12:53 PM
Hi Julius,

we are starting and activating our RTPClient like this:
            try
            {
                if (!IsRTPConnected)
                {

                    Media.Sdp.SessionDescription sessionDescription = new Media.Sdp.SessionDescription(sdp.ToString());
                    sessionDescription.SessionName = sdp.Name;
                    sessionDescription.SessionId = sdp.Owner.SessionID.ToString();

                    ClientLogger.Instance.Log("RTPClient Created from SessionDescription: " + sessionDescription);

                    RTPClient = Media.Rtp.RtpClient.FromSessionDescription(sessionDescription, rtpPort: 0, rtcpEnabled: false);

                    foreach (RtpClient.TransportContext context in RTPClient.GetTransportContexts())
                    {
#if __ANDROID__
                        Media.Common.ISocketReferenceExtensions.SetReceiveBufferSize(context, Globals.BufferSize); 
#endif
                        Media.Common.ISocketReferenceExtensions.SetReceiveTimeout(context, 4);
                        Media.Common.ISocketReferenceExtensions.SetSendTimeout(context, 2);
                    }

                    RTPClient.FrameChangedEventsEnabled = true;

                    RTPClient.RtpFrameChanged += OnSourceFrameChanged;
                    
                    foreach (var context in RTPClient.GetTransportContexts())
                    {
                        if (context.MediaDescription.MediaType == MediaType.audio)
                        {
                            AudioPort = (context.RtpSocket.LocalEndPoint as System.Net.IPEndPoint).Port;
                        }
                        else if (context.MediaDescription.MediaType == MediaType.video)
                        {
                            VideoPort = (context.RtpSocket.LocalEndPoint as System.Net.IPEndPoint).Port;
                        }
                    }
                    RTPClient.Activate();
Works for both Android with 65 and iOS with older 88 if I do some minor changes to iOS RTPClient.

Later versions also work but something is missing there, the Picture is broken and I get only Fragments, the Audio is delayed. But still communications is working. Just Problems with Decoders but no exceptions or something you can easily find.
Coordinator
Jun 9, 2016 at 6:47 PM
Those timeouts are whats probably causing you the problems, check MSDN Here (Suprisingly not Here but still applies)

So those values get translated into 500 Milliseconds regardless of your input for 1 >= X <= 499, 0 is indefinite... -1 is also indefinite... seems indefinitely definitive that 0 and -1 are the same for this value; 500 >= X <= int.MaxValue works as expected.

That's really the only thing I see with the initialization procedure you posted there which I would really change.

You can also remove the SetReceiveBufferSize call, it's done automatically for you during Initialize on the TransportContext, it always was but I recently changed TransportContext to allow someone to specify exactly how much they wanted including 0 (which is probably what I would recommend for this case).

You don't want to buffer receives if you can avoid it, it costs more memory and it causes an additional memory copies.

With that being said UDP typically has a high rate of data transfer and using a value for 0 receive buffer size can cause you to drop packets if your not receiving them fast enough.

In your case though you probably want to drop those packets anyway and it seems to me like a more natural solution than letting the decoder possibly back up on frames which are no longer even relevant.

Give a few experiments with changing the ReceiveBufferMultiplier on the TransportContext to 0, see if you can get the decoder to work that way and how much better or worse the result is.

After you think your at the point where you can't optimize the frame handling anymore, try increasing the ReceiveBufferMultiplier until you find the minimum value which works for your application.

You can determine this at runtime by checking the 'FailedRtpTransmissions' and 'FailedRtpReceptions', if you find the values growing this indicates your missing packets.

You can also use 'RtpBadSeq' to determine this but it's slightly harder to say if your just receiving out of order or missing packets but the same logic applies.

Once you realize that you are missing packets (either by looking at the properties of the TransportContext from time to time or encountering a frame which IsMissingPackets) you can then call SetReceiveBufferSize again and again keeping track of how many times you did so.

This will allow you to adjust in real time to packet loss etc and compensate as required.

Let me know if your still having trouble and I will have no problem doing another debug session to see if we can't narrow it down together!

Thank you again for your patience and testing!

We will get the Windows Phone 10 / Universal Windows Platform version running in due course.

Lets get these smaller existing issues addressed so we can be sure that we don't encounter them at a later date and on other systems and then I will devote time to getting the library running on Windows Phone 10 / Universal Windows Platform (even though I am fairly confident at this point that if you target just one of those environments instead of the portable environment that it will work as it is, an if not I apologize in advance and need to understand why so I can address it)

Type to you soon!

-Julius
Marked as answer by juliusfriedman on 6/9/2016 at 11:47 AM
Jun 10, 2016 at 8:14 AM
Edited Jun 10, 2016 at 8:15 AM
Hi Julius,

thanks a lot for your help!

we tested with Android and the latest 112146. As before, only (very few) Fragments of the Video and delayed Audio.

Then we removed from the transportcontext:
  • SetReceiveBufferSize no Change,
  • SetReceiveTimeout and SetSentTimeout no Change
Or did you talk about other timeout, too?

Then we set BufferMultiplier to 0, 1, 10, 100, 200, 409 and in OnFramechanged we increased it if Frame.IsPacketMissing is true.

That way we did not see any changes.

I will be here from now for 6 hours still and will turn on skype later. If you have some time to talk about it and debug together, please let me know. Or if you have any other idea what I can test, please tell me.

I am also available each day of next week from monday to friday.

Thanks a lot!
Coordinator
Jun 13, 2016 at 1:41 PM
I will have no problem doing a quick debug session to determine where the issues is especially because I cannot replicate it.

If you can provide the Wireshark Capture and accompanying h.264 file you create in the Application I may be able to visualize what is going on without exactly seeing it happen.

Otherwise let me know how you want to proceed, I have a few updates pending which will further improve performance in certain areas related to Rtsp parsing but nothing much in the RtpClient yet.

I am seeing if I will get enough time to work on adding new events and possibly revise the event model yet again for the new events, not to mention work on the RtpFrame -> Depacketize etc.

-Julius
Marked as answer by juliusfriedman on 6/13/2016 at 11:22 AM
Jun 14, 2016 at 8:13 AM
Hi Julius,

Support with Debug together would be great, I want to get the latest Version running now, best would be today, or tomorrow.

I am just Debugging 112149.

First try again I get on Android data but no Video like before.

So I switched to iOS to see if I get there anything at all.

The exceptions are gone that's good, and I can connect.

But I did not receive data.

First Problem others with iOS will also run into:
            //If the socket is NOT null and IS BOUND use the localIp of the same address family
            if (false.Equals(existingSocket == null) && existingSocket.IsBound)
            {
                //If the socket is IP based
                if(existingSocket.LocalEndPoint is IPEndPoint)
                {
                    //Take the localIp from the LocalEndPoint
                    localIp = (existingSocket.LocalEndPoint as IPEndPoint).Address;
                }
                else
                {
                    throw new NotSupportedException("Please create an issue for your use case.");
                }
            }
            else // There is no socket existing.
            {
                //If the remote address is the broadcast address or the remote address is multicast
                if (System.Net.IPAddress.Broadcast.Equals(remoteIp) || Common.Extensions.IPAddress.IPAddressExtensions.IsMulticast(remoteIp))
                {
                    //This interface should be the interface you plan on using for the Rtp communication
                    localIp = Media.Common.Extensions.Socket.SocketExtensions.GetFirstMulticastIPAddress(remoteIp.AddressFamily, out  localInterface);
                }
                else
                {
                    //This interface should be the interface you plan on using for the Rtp communication
                    localIp = Media.Common.Extensions.Socket.SocketExtensions.GetFirstUnicastIPAddress(remoteIp.AddressFamily, out  localInterface);
                }
            }
That gives not the correct localIP, it return something like 127.0.0.1 and so I do not receive Frames.

I added
        localIp = IPAddress.Parse("192.168.1.110");
just to test. now I receive data in iOS, too.
Jun 14, 2016 at 9:19 AM
Edited Jun 14, 2016 at 10:06 AM
I made a wireshark from starting the app (release build), then starting a call from my doorbell with camera which has av packets.

I did not see or hear av.

https://www.dropbox.com/s/zd2vm1dvq413f95/Wireshark-AndroidClient-ReleaseBuildWith112149.pcapng?dl=0

Thanks for your help. I will examine further and be online via skype from next hour on.
Jun 14, 2016 at 10:05 AM
Edited Jun 14, 2016 at 10:42 AM
Hi Julius,

I made a "movie". In fact it is not. This is what arrives in my Decoder after Depackatize.

https://www.dropbox.com/s/5s07yk4z4bjomkq/ownvideo14.06.2016%20115256.h264?dl=0

It is very small. When I made something like that the last time, it is over 2 MB, this time only 400kb. Duration should be similar.
And I cannot Play it. I guess there is too much data missing.

It would be great if you could take a look, I will be online in around half an hour for the next 5 hours...

thanks a lot!

PS: This movie did not check for "final" in SoruceFrameChanged. I turned it off for testing. I guess it would be even smaller when I turn that on again...
Coordinator
Jun 14, 2016 at 12:03 PM
You didn't capture enough of the stream for me to really do much investigating :(

In reference to the iOS error, are any other IP Addresses listed? or is the call to 'GetFirstUnicastIPAddress' not even getting past 'System.Net.NetworkInformation.NetworkInterface.GetIsNetworkAvailable()'

Skype me if you want to debug, I will be signing in shortly!
Marked as answer by juliusfriedman on 6/14/2016 at 5:03 AM
Jun 14, 2016 at 12:07 PM
Edited Jun 14, 2016 at 12:09 PM
Debug anytime you want to, I am here! Thanks!
I thought the stream would be enough, it is 4,4 MB large. What is missing?

We could investigate the iOS connecting issue to make your Library iOS compatible for others.
And find out why I do not get complete Video anymore and missing so much data, that would be great for me :-)

We are also currently investigating how to get PCL assemblies:
https://visualstudiogallery.msdn.microsoft.com/e2dcdf06-444d-4501-8dae-732e76bc94aa?SRC=Home
If we get something working, I will tell you how. I am still pretty sure that we Need those for UWP (Win10).
Coordinator
Jun 14, 2016 at 9:25 PM
I meant the h.264 file was not large enough in comparison to the wire shark capture for me to see anything or diagnose anything.

I appreciate your continued testing and patience while attempting to diagnose and resolve whatever issues were manifested between 65 and now.

I also extremely appreciate your continued patience with the UWP / Windows 10 issue; I wish I had more time and more of a reason to work with UWP / Windows 10 but at this time I don't really have any ambition to do so other than getting your project working.

I will ensure that we resolve the Android issues as well as iOS issues you experience in the newer versions and finally we will ensure you can deploy on Windows 10.

I also have quite a few other things besides improvements to this library on my plate which takes time away from this project but it's important to me that what I offer works as it is advertised.

I will work with you again tomorrow and we will get to the bottom of all this very quickly!
Marked as answer by juliusfriedman on 6/14/2016 at 2:25 PM
Jun 15, 2016 at 7:32 AM
Edited Jun 15, 2016 at 7:32 AM
Hi Julius,

I just tested the 112151 with andorid release build and my original Code (before we made the changes yesterday).
Still some Fragments of a Video only without complete Picture, and a delayed Audio that sounds a Little broken.

I will be available today for a debug session from the same time like yesterday before.

Thanks a lot for helping me out :-)
Coordinator
Jun 15, 2016 at 11:49 AM
Thanks for your continued testing.

Give me about 30 minutes to get situated and I will Skype you.
Marked as answer by juliusfriedman on 6/15/2016 at 4:49 AM
Jun 15, 2016 at 3:32 PM
Jun 15, 2016 at 4:19 PM
Coordinator
Jun 15, 2016 at 9:18 PM
Edited Jun 15, 2016 at 10:27 PM
Did a bit of bench-marking / testing and.... I didn't find much. :(

I was using the default settings, buffer sizes and GC Latency settings, e.g. bufferSize = 8192, System.Runtime.GCSettings.LatencyMode = Interactive

I know your calls run for about 20 seconds so I did tests for well over that and I couldn't honestly even get the GC run a collection on my computer... CLR Profiler, See also How to use the CLR Profiler

There is also a version for UWP / Windows Store apps here: CLRProfiler45

Even when trying to cause memory issues with the application by switching from ThreadEvents to not in the middle of sending requests and various other methods including introducing random sleeps or my favorite yet, simply not calling dispose I wasn't able to replicate what your experiencing.

I am hoping that the few changes I made possibly get the solution working for you on Android again, if not I will do some additional debugging until we both can verify the stable operation of the RtpClient.

If I am not able to find anything out within the first few hour or so of debugging I will have revise how I am disposing frames in the RtpClient and hopefully that will resolve the issue, depending on how that works out and only if needed I will totally revise the Send/Receive and Event model ahead of my schedule.

I will add the other events I was speaking about in the RtpClient and I will add at least one other cycle before frames are disposed.

If that still doesn't solve the issue I will have to get the library running on an Android phone I have locally to really debug it and find out whats going on.

What I will do is get a simple Android based unit test started and utilize from my library from that unit test, in the unit test I will just setup a Rtsp or Rtp client and consume a demo stream which is either given at run-time or via a setting in the application.

I should then be able to playback the stream on my handset and experience similar symptoms to that of which you are experiencing on your handset on my handset and I can debug from there to isolate and finally fix the issue when I find it.

Lets hope for the former rather than the latter :)

I apologize for any inconvenience this has caused you and sincerely appreciate your patience while we resolve this matter.

I will be available tomorrow to debug further!

I am also running a few profile sessions as I write this so if they come up with anything I will either make another update or review in morning.

Most Sincerely,
Julius
Marked as answer by juliusfriedman on 6/15/2016 at 2:18 PM
Jun 16, 2016 at 6:35 AM
Hi Julius,

I downloaded and tried 112155. Again I get no Video. In set a breakpoint in the Android h264 Decoder, it gets data, but does not start Decoding:
        public void ProcessDecoderOuput()
        {
            try
            {
                MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();

                int outputBufferIndex = _codec.DequeueOutputBuffer(bufferInfo, timeout);
                while (outputBufferIndex != (int)MediaCodecInfoState.TryAgainLater)
It never gets into the while Loop.

The Audio is there. but delayed. But the Debug and the many console writings could be the cause for this? I will try a release for the Audio.
Jun 16, 2016 at 6:40 AM
With a release build it takes much longer until it is delayed and sounds broken (than in the debug build).

In both cases no Video at all, no Fragments or something.
Coordinator
Jun 16, 2016 at 1:38 PM
Interesting... :(

I have updated the source code again with some testing I was doing last night.

I have to do a few quick things this morning related to something else but I will be available within 2 hours or so;

For a quick test, you can enable Holding, e.g. comment it and then comment the code which disposes LastFrame, but do cycle LastFrame, e.g. make LastFrame CurrentFrame.

I will Skype you as soon as I can debug, it shouldn't be more than 2 hours or so.

Thank you again for your patience!
Marked as answer by juliusfriedman on 6/16/2016 at 6:38 AM
Jun 16, 2016 at 2:03 PM
I Need to go to the Dentist today, so I will leave earlier. Maybe we can debug tomorrow, that would be nice!

Thanks for helping.

A quick test of 112156 shows some fragements for a short while, the the Picture freezes. Audio is there at the start, then seems to be gone after a few seconds.

Do not have much time left to try your suggestions today anymore, sorry for that. See you tomorrow I hope :-)
Coordinator
Jun 16, 2016 at 2:12 PM
Yes, I will be available.

I am actually available now, I am just getting settled in.

If you have some time today let me know.

One thing I would like you to try please...

Take the Code from 'http://net7mma.codeplex.com/SourceControl/list/changesets' Starting from 112065 and try each version deliberately as it is on the handset.

Let me know if you can find a version which works from the code in source control on your handset without issues.

Once you find that version if you could let me know which version after that it stops working I would be able to do some more comparison between only the versions which were working and then which don't rather than speculating from the current code.

Thanks and please feel better!
Marked as answer by juliusfriedman on 6/16/2016 at 7:12 AM
Jun 16, 2016 at 2:49 PM
Edited Jun 16, 2016 at 2:54 PM
Hi Julius,

I am home and cannot try from here, but I can check things tomorrow.

I am pretty sure I tested nearly all Versions from 65 until the latest and None of those worked. I also check the original 65 and it did not work I think.

Did you realize any difference between the assemblies I sent you and the original maybe?

Thanks I am fine, I think my Dentist visit today should not be painful :-)
Coordinator
Jun 16, 2016 at 3:03 PM
Here is some code which I am using to verify that from the application I am testing with that I receive frames which have sequential sequence numbers and timestamps and that there are no missing packets.
static void DumpFrame(Media.Rtp.RtpFrame frame, ref bool final)
        {
            if (Media.Common.IDisposedExtensions.IsNullOrDisposed(frame)) return;

            int frameCount = frame.Count;

            if (frameCount.Equals(0)) return;

            System.Text.StringBuilder output = new System.Text.StringBuilder();

            if (final) output.Append("Final @ ");

            output.Append(frame.Timestamp + " / ");

            output.Append(frameCount);

            if (frameCount > 0)
            {
                output.Append(" - [" + frame.HighestSequenceNumber + " , " + frame.LowestSequenceNumber + "]");
            }
            else
            {
                output.Append(" - [" + frame.HighestSequenceNumber + "]");
            }

            output.Append(" MarkerCount = " + frame.MarkerCount);

            if (final)
            {
                bool complete = frame.IsComplete;

                if (false.Equals(complete)) ++missingPacketsOnFinal;

                output.Append(" Complete = " + frame.IsComplete);
            }
            else
            {
                output.Append(" IsMissingPackets = " + frame.IsMissingPackets);
            }

            System.Diagnostics.Trace.WriteLine(output);
        }
From the event e.g. 'OnSourceFrameChanged' / 'Client_RtpFrameChanged'
 Media.Rtp.RtpClient.TransportContext context = tc ?? ((Media.Rtp.RtpClient)sender).GetContextByPayloadType(frame.PayloadType);

            //If there is no context or the packet does not correspond to a video media type, return
            if (Media.Common.IDisposedExtensions.IsNullOrDisposed(context) || 
                false.Equals(context.MediaDescription.MediaType == Media.Sdp.MediaType.video))
            {
                return;
            }

            //Not the final event
            if (false.Equals(final))
            {
                //if the frame is not complete then dump it and return
                if (false.Equals(frame.IsComplete))
                {
                    DumpFrame(frame, ref final);

                    return;
                }
            }

/*
Here the frame seems complete, it is OKAY to attempt to decode this frame early, if the decoder doesn't have enough data the application will receive another OnRtpFrameChanged Event with the same Timestamp (if the sender sends such a packet), which will be put into the buffer so the decoder can decode it. 

The only thing important to remember here is that if the decoder advanced past the buffer and did not decode anything from that buffer then that existing buffer must be re-queued to the decoder before the new data is queued to the decoder for the remaining data of the frame so the decoder does not skip it.

E.g. Timestamp 0, 3 packets, Sequence 0, 1M, 2M

RtpFrameChanged => Timestamp = 0, 1 Packet, No Marker

RtpFrameChanged => Timestamp = 0, 2 Packets, 1 Marker, IsComplete = true

Depacketize

Decode

Not enough data

Requeue buffer

RtpFrameChanged  => Timestamp = 0, 1 Packet, 1 Marker, IsCompelte = true

Depacketize

Decode

Finally enough data

*/

//Dispose this packet NOW so that duplicate events are not raised.
Media.Common.BaseDisposable.SetShouldDispose(frame, true, true);
This lets you decode as early as possible and also forces the disposal of the frame which was already decoded or buffered to the decoder.

This will give you slightly more responsive decoding than waiting for 'final' as you are actively decoding data which is available and as soon as it becomes available.

I am still testing and hopefully I think I may have found a few more 'nuances' with Dipose in the base classes.

I will keep you updated as I continue to debug this.

Sorry again for the inconvenience this has caused.
Marked as answer by juliusfriedman on 6/16/2016 at 8:03 AM
Coordinator
Jun 16, 2016 at 3:06 PM
The changes in the older version's as far as the RtpClient is concerned don't make any meaningful change IMHO that would cause such a negative impact.

It appears that the Common.BaseDisposable and Common.SuppressFinalizeDisposable still have a bug or two;

I found yet another thing / bug and I am testing to see if I find anything else.

I will post that update shortly.
Marked as answer by juliusfriedman on 6/16/2016 at 8:06 AM
Coordinator
Jun 16, 2016 at 7:38 PM
I have done more testing and profiling.

I only found 1 or two real bugs, the changes in the BaseDisposable were minor but probably related to the issues you were experiencing (At least I hope).

Let me know if that's the case so I at least know that your issues with Android are resolved, I can then focus on anything for iOS and UWP afterwards.

There were more changes in the RtspServer than anything else and definitely a major improvement in the RtspServer so I am hoping that I am at least getting closer to finding the issue with the RtpClient if I didn't already find it and fix it.

I will continue to do some bench-marking and profiling and see if anything else comes up!

Thanks again for your patience and testing!
Marked as answer by juliusfriedman on 6/16/2016 at 12:38 PM
Coordinator
Jun 16, 2016 at 8:41 PM
Edited Jun 16, 2016 at 8:43 PM
112158 has minor revisions which seem to improve the RtspServer further.

The RtspClient should be close to where it needs to be as well as the RtpClient if it's not already.

I already have a few ideas related to where I can improve if needed but I am hoping that I have it solved with the latest code.

Talk to you soon!

-Julius
Marked as answer by juliusfriedman on 6/16/2016 at 1:41 PM
Coordinator
Jun 16, 2016 at 10:26 PM
Just to confirm, my recent tests and the profiling results of the RtspServer against multiple streams (while viewing them at the same time as the test).

I have made a few more changed but I have not upload them yet as they only pertain to the RtspServer to make it highly available.

The tests ran for over about 30 minutes, connecting as many clients as possible to the server while watching the streams at the same time through VLC and other players.

I was never able to be locked out of the server for more than a few seconds, in the seconds I was it was because I was also using about 50 clients a second to connect to the RtspServer.

I never dropped a packet to the client and I never had higher than 50% usage on the RtspServer for more than a few minutes, the peaks and valleys look fairly healthy to me. (See Proccesor Time vs Gen 2 Collections)

Anyway...

I had finally got the GC to run!!!

But I had over 3000 connected clients...

You can find all of the graphics below!

EntireGraph

TimeInGC

Gen2Collections

ProcessorTime

I am going to take a break now... I think we are on track.

Let me know if I am excited for no reason :)
Marked as answer by juliusfriedman on 6/16/2016 at 3:26 PM
Jun 17, 2016 at 6:41 AM
Edited Jun 17, 2016 at 9:32 AM
Hi Julius,

thanks for the update. I am not using RTSP, only RTP with an existing SIP Client.

A first quick test of 112158 with Android release build showed more fragements I think or at least gray blocks, but no complete Picture.

I added your DumpFrame Code from above now and run in debug (always logging using Audio and Video currently):
10617 / 2 - [63781 , 63780] MarkerCount = 0 IsMissingPackets = False
[0:] 10617 / 2 - [63781 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:00.936 I/mono-stdout(17847): 10617 / 2 - [63781 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:01.266 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:01.288 I/art (17847): Explicit concurrent mark sweep GC freed 1313(70KB) AllocSpace objects, 0(0B) LOS objects, 40% free, 16MB/27MB, paused 380us total 21.731ms
06-17 09:02:01.300 D/Mono (17847): GC_OLD_BRIDGE num-objects 1226 num_hash_entries 1249 sccs size 1249 init 0.00ms df1 2.99ms sort 0.31ms dfs2 2.61ms setup-cb 1.38ms free-data 1.04ms links 24/24/24/1 dfs passes 2499/1273
06-17 09:02:01.300 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 73.87ms, total 74.65ms, bridge 0.00ms major 6432K/0K los 904K/0K
10617 / 3 - [63782 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:01.497 I/mono-stdout(17847): 10617 / 3 - [63782 , 63780] MarkerCount = 0 IsMissingPackets = False
[0:] 10617 / 3 - [63782 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:02.831 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:02.851 I/art (17847): Explicit concurrent mark sweep GC freed 1157(59KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 16MB/27MB, paused 368us total 19.643ms
06-17 09:02:02.862 D/Mono (17847): GC_OLD_BRIDGE num-objects 1177 num_hash_entries 1192 sccs size 1192 init 0.00ms df1 2.77ms sort 0.31ms dfs2 2.33ms setup-cb 1.80ms free-data 0.84ms links 16/16/16/1 dfs passes 2385/1208
06-17 09:02:02.862 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 78.21ms, total 79.18ms, bridge 0.00ms major 6496K/0K los 911K/0K
10617 / 4 - [63783 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:03.065 I/mono-stdout(17847): 10617 / 4 - [63783 , 63780] MarkerCount = 0 IsMissingPackets = False
[0:] 10617 / 4 - [63783 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:04.165 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:04.191 I/art (17847): Explicit concurrent mark sweep GC freed 957(49KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 16MB/27MB, paused 402us total 25.705ms
06-17 09:02:04.202 D/Mono (17847): GC_OLD_BRIDGE num-objects 988 num_hash_entries 1003 sccs size 1003 init 0.00ms df1 2.21ms sort 0.49ms dfs2 2.56ms setup-cb 1.22ms free-data 0.56ms links 16/16/16/1 dfs passes 2007/1019
06-17 09:02:04.202 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 72.54ms, total 73.20ms, bridge 0.00ms major 6560K/0K los 964K/0K
10617 / 5 - [63784 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:04.398 I/mono-stdout(17847): 10617 / 5 - [63784 , 63780] MarkerCount = 0 IsMissingPackets = False
[0:] 10617 / 5 - [63784 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:05.407 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:05.430 I/art (17847): Explicit concurrent mark sweep GC freed 902(46KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 16MB/27MB, paused 468us total 23.151ms
06-17 09:02:05.441 D/Mono (17847): GC_OLD_BRIDGE num-objects 938 num_hash_entries 953 sccs size 953 init 0.00ms df1 2.04ms sort 0.47ms dfs2 1.28ms setup-cb 1.20ms free-data 0.80ms links 16/16/16/1 dfs passes 1907/969
06-17 09:02:05.441 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 77.18ms, total 77.70ms, bridge 0.00ms major 6560K/0K los 921K/0K
10617 / 6 - [63785 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:05.644 I/mono-stdout(17847): 10617 / 6 - [63785 , 63780] MarkerCount = 0 IsMissingPackets = False
[0:] 10617 / 6 - [63785 , 63780] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:06.525 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:06.547 I/art (17847): Explicit concurrent mark sweep GC freed 821(42KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 16MB/27MB, paused 427us total 21.942ms
06-17 09:02:06.557 D/Mono (17847): GC_OLD_BRIDGE num-objects 863 num_hash_entries 878 sccs size 878 init 0.00ms df1 2.05ms sort 0.46ms dfs2 0.93ms setup-cb 1.01ms free-data 0.50ms links 16/16/16/1 dfs passes 1757/894
06-17 09:02:06.557 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 77.12ms, total 77.87ms, bridge 0.00ms major 6576K/0K los 924K/0K
06-17 09:02:06.762 I/mono-stdout(17847): 14218 / 1 - [63786 , 63786] MarkerCount = 0 IsMissingPackets = False
14218 / 1 - [63786 , 63786] MarkerCount = 0 IsMissingPackets = False
[0:] 14218 / 1 - [63786 , 63786] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:07.522 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:07.545 I/art (17847): Explicit concurrent mark sweep GC freed 736(38KB) AllocSpace objects, 0(0B) LOS objects, 40% free, 16MB/27MB, paused 380us total 21.879ms
06-17 09:02:07.553 D/Mono (17847): GC_OLD_BRIDGE num-objects 781 num_hash_entries 796 sccs size 796 init 0.00ms df1 2.12ms sort 0.43ms dfs2 0.94ms setup-cb 0.33ms free-data 0.46ms links 16/16/16/1 dfs passes 1593/812
06-17 09:02:07.553 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 73.29ms, total 73.98ms, bridge 0.00ms major 6608K/0K los 961K/0K
14218 / 2 - [63787 , 63786] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:07.761 I/mono-stdout(17847): 14218 / 2 - [63787 , 63786] MarkerCount = 0 IsMissingPackets = False
[0:] 14218 / 2 - [63787 , 63786] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:08.508 I/art (17847): Starting a blocking GC Explicit
06-17 09:02:08.530 I/art (17847): Explicit concurrent mark sweep GC freed 708(36KB) AllocSpace objects, 0(0B) LOS objects, 39% free, 16MB/27MB, paused 415us total 21.614ms
06-17 09:02:08.539 D/Mono (17847): GC_OLD_BRIDGE num-objects 758 num_hash_entries 773 sccs size 773 init 0.00ms df1 1.75ms sort 0.42ms dfs2 1.00ms setup-cb 0.50ms free-data 0.47ms links 16/16/16/1 dfs passes 1547/789
06-17 09:02:08.539 D/Mono (17847): GC_MAJOR: (LOS overflow) pause 81.09ms, total 81.80ms, bridge 0.00ms major 6608K/0K los 933K/0K
17815 / 1 - [63788 , 63788] MarkerCount = 0 IsMissingPackets = False
06-17 09:02:08.756 I/mono-stdout(17847): 17815 / 1 - [63788 , 63788] MarkerCount = 0 IsMissingPackets = False
[0:] 17815 / 1 - [63788 , 63788] MarkerCount = 0 IsMissingPackets = False
It seems Makercount is always 0. does that help?

I will be available for Debugging today. :-)

Thanks Julius!
Coordinator
Jun 17, 2016 at 11:50 AM
Weird, it gives me something else to look into..

I am just getting situated here, I need about 30 - 45 minutes to get ready!

Skype you soon!
Jun 17, 2016 at 12:16 PM
Thanks a lot! I am online for next 3 hours. If not, just left for a coffee or something like that :-)
Jun 20, 2016 at 8:55 AM
Hi Julius,

I just tested 112159 but see no big Change, there is no Video.

How is the Android testing working? Can I help you with something?
Would a debug session next couple of days help?

Thanks and have a nice monday :-)
Coordinator
Jun 20, 2016 at 12:14 PM
The Android testing is at the point where I have to hook up the decoder, before I do that though I was just going to get a basic interface going where I could enter in an address and click start playing/stop playing.

The file logger already logs the sdp and rtsp messages so I was going to also hook up something to view those logs in the application also.

I have a meeting today at 3:30 so I am not sure how far I will get on the Android stuff before then, probably tommorow tommorow I will have something more.

I also have a meeting on Thursday so hopefully between Tuesday and Thursday nothing else comes up to delay that further.

If I get enough time today I will update you on my progress.

I will also be looking into the Uwp stuff while I have the chance but after the video problem is sorted out.

-Julius
Marked as answer by juliusfriedman on 6/20/2016 at 5:15 AM
Jun 20, 2016 at 12:46 PM
Hi Julius,

okay, thanks for the info!
Coordinator
Jun 20, 2016 at 4:05 PM
Got about an hour or so fee time right now, I will be updating the TestApp and sending you what I have so far when I finish.

-Julius
Marked as answer by juliusfriedman on 6/20/2016 at 9:05 AM
Coordinator
Jun 20, 2016 at 5:48 PM
Sent you an email with the updated View and Code.

If I can get some more time today I will shoot you another email otherwise probably tomorrow!

-Julius
Marked as answer by juliusfriedman on 6/20/2016 at 10:49 AM
Jun 21, 2016 at 6:26 AM
Hi Julius,

I have something else to do this morning and yesterday, so I did not try the sample yet.
But we can later debug and try the sample when you stood up :-)

I only do not have that much time to test as Long and late as last week.

Thanks for searching for the bug! I will be online in skype later today.
Jun 21, 2016 at 7:34 AM
PS: There is a nuGet Package for System.Runtime.InteropServices you can also use in a PCL. Maybe that helps for the later Windows Phone compability :-)
Coordinator
Jun 21, 2016 at 10:55 AM
I am just getting situated, I will be able to Skype and debug in about 30 minutes.


-Julius
Marked as answer by juliusfriedman on 6/21/2016 at 3:55 AM
Jun 21, 2016 at 1:45 PM
Hi Julius,

thanks a lot for your sample. I get one build error when I insert the 112159 assemblies:

if (client != null) matched = RtspClient.Client.GetContextForPacket(rtcpPacket);

MainActivity, Line 926:

Severity Code Description Project File Line Suppression State
Error CS1503 Argument 1: cannot convert from 'Media.Rtcp.RtcpPacket' to 'Media.Rtp.RtpPacket' TestApp C:\Projekte\FirstAPp\FirstAPp\MainActivity.cs 926 Active
Coordinator
Jun 22, 2016 at 8:36 PM
It's maybe because of the access change of the function, 112162 has the changes required which I verified and works on my devices and should have the following public functions on RtpClient:
    public TransportContext GetContextByPayloadType(int payloadType);
    public TransportContext GetContextBySocket(Socket socket);
    public TransportContext GetContextBySocketHandle(IntPtr socketHandle);
    public TransportContext GetContextForFrame(RtpFrame frame);
    public TransportContext GetContextForMediaDescription(MediaDescription mediaDescription);
    public TransportContext GetContextForPacket(RtcpPacket packet);
    public TransportContext GetContextForPacket(RtpPacket packet);
You should be able to load and run the TestApp using 112162
Marked as answer by juliusfriedman on 6/22/2016 at 1:36 PM
Coordinator
Jun 24, 2016 at 9:29 PM
112165 should run the same or better than 112162.

-Julius
Marked as answer by juliusfriedman on 6/24/2016 at 2:29 PM
Jun 27, 2016 at 7:16 PM
Hi Julius,
sorry, we had a company summer party yesterday, I did not read your mail :-)
have a nice weekend!
eric

--
EMail: [email removed]
Besuch meine HomePage www.ericbehme.de
Gesendet: Freitag, 24. Juni 2016 um 21:35 Uhr
Von: juliusfriedman <[email removed]>
An: [email removed]
Betreff: Re: Managed Media Aggregation Stability & Performance on Xamarin Devices (Android and iOS) [net7mma:654743]

From: juliusfriedman

I am just getting situated, I will be able to Skype and debug in about 30 minutes.


-Julius
Coordinator
Jun 28, 2016 at 1:03 AM
That reply was sent via gmail a few days ago when we debugged the last time :)

It's no worries, take your time and let me know when you have either the issue located in my TestApp which I can reproduce or if your find a work around or that another version works as the old 65 did.

UWP / Native support I am looking into but done have any major ambitions to support at the moment beyond what is offered in the library already, native apps should be able to be built from my library as the source is provided and hopefully if I get the time and resources to look into why you are having issues in that environment I will do so but without a reason to do so it may takes longer than you would like.

Sincerely,
Julius
Marked as answer by juliusfriedman on 6/27/2016 at 6:03 PM
Coordinator
Jun 28, 2016 at 3:48 PM
I took another quick look at the UWP stuff.... I simply am at a loss to what your specific problem is, I know the error you get and I can't seem to understand why... It probably has something to do with the Visual Studio version your using.

You may also want to check out this tutorial on how to use Visual Studio 2013 to build a Native project (Unless your using Visual Studio 2015) then you should already be able to do this.

Adding Shared Projects Support For Xamarin In Visual Studio 2013

Using a similar approach you should be able to reference my libraries and utilize them from your application on Windows Phone, iOS or Android without issues.

If you need a variation of the library which is built against a different framework e.g. the Portable Subset for whatever reason then changes will need to be made to the Media Solution and all other solutions to be built against whatever framework you wish to target and not the Standard .Net Framework

See MSDN

There is only one place this Portable Subset becomes useful and that's on Windows Phone and unfortunately that's Microsoft's own fault.

I honestly think that once Dot Net Core takes off there will be little use for the Portable Sub Set and they will likely be killed off... See what others say about how this is handled Here, And here

It's not like the IL Language is different or there is anything which causes the incompatibility in the first place e.g. like with Micro Framework... This is Microsoft's issue in so many ways its not even funny...

See also apparently you don't even need really to build for UWP...
Marked as answer by juliusfriedman on 6/28/2016 at 8:48 AM
Coordinator
Jul 5, 2016 at 6:16 PM
I have fixed the 'bugs' with the RFC6184 Frame which were the cause of the distortion in the decoder.

I have verified under TCP and UDP.

Please let me know if your still having trouble.
Marked as answer by juliusfriedman on 7/5/2016 at 11:16 AM
Jul 20, 2016 at 7:24 AM
Hi Julius,

sorry for the late Reply, I had Holidays for one and a half week and was very budy since then. I will try and let you know, thanks for the update.
Jul 20, 2016 at 8:13 AM
Edited Jul 20, 2016 at 8:20 AM
The (very) good new: Version 112188 Tested on 2 different Android devices, it works. I got Picture and Audio. That's great we have a Version where it works again!

The bad news: there is a delay, which is very small at the start, but grows a lot and adds artefacts after a while. At the beginning everything Looks good.

But it is great to have a working Version again. :-)

Thanks!
Coordinator
Jul 20, 2016 at 7:51 PM
Thanks for your feedback.

Please let me know how you like the changes to the layout of the RtpClient class as well as how some of the issues are being resolved.

If you would please also take the time to try and use the SIPMessage for setting up the connection it would help me start find and resolve issues in that area as well.

I have some ambitions to work on Http, specifically Http Tunneling as well as normal Http communication, Http 2 will require compression classes whereas Http and Rtsp etc would only benefit from them.

I also have some ambitions to work more on the RtpFrame class as well as the PaD concept; your feedback in that area is also welcome.

Finally, I have some ambitions to if nothing else to create a interface or class which can be used to implement Encoders or Decoders if not directly perform the Encoding or Decoding themselves.

I will see how much time I get done in the way of working on these things as well as the other concepts I have presented hopefully this week.

Thank you for your continued feedback.
Marked as answer by juliusfriedman on 7/20/2016 at 12:51 PM
Jul 21, 2016 at 8:14 AM
Edited Jul 21, 2016 at 8:15 AM
Hi Julius,

I am late with the development of my app and in a big hurry. I am sure I won't be able to try something this week, and I am not sure about next week.

I am still using the old Version because I do not even have the time to find out what slows down the new Version (which is working again that's great).

I hope I'll be able to look for the delay and try on iOS next month, maybe I can try the sipmessage too if it is not to much efford. Windows phone has been stopped at this Moment, I do not know if we continue with it.

thanks for helping and sorry I do not have the time at the moment
Coordinator
Jul 21, 2016 at 6:06 PM
Time is surely not of the essence for me in relation to this library nor any of the underlying conventions portrayed, conceived or associated with such therein.

I thank you again for your time and interest and remind you to keep the RFC6184Frame class updated with my changes such that it would be easy to find the problems in such cases as the last time.

Further I remind you that profiling is required to determine exactly were such issues may or may not arise [such as but not limited to slows down the new version]; based on the FACT that the library only does provide the events from the socket you are already using in your specific case (UDP); You utilize SIP from independent soft to setup and control the device and only use my library to enable the radiation of data [display and audible] which was transferred to your application from the operating system.

To eliminate such concerns utilize the built in Android RtpStream classes or even the higher level Intent class

e.g.
                Android.Content.Intent intent = new Android.Content.Intent(Android.Content.Intent.ActionView);

                intent.SetData(Android.Net.Uri.Parse(VideoUri));

                StartActivity(intent);
I appreciate your continued interest and feedback and hope to provide more updates to the library in due course;

Regards,
Marked as answer by juliusfriedman on 7/21/2016 at 11:07 AM
Jul 26, 2016 at 8:00 AM
Hi Julius,

today I had a short time to try out 112200.

First I got an Build error

in exceptionextension
System.Diagnostics.StackFrame = new System.Diagnostics.StackFrame(stackFrameDepth, fNeedFileInfo);
Error 56 'System.Diagnostics.StackFrame' is a 'type', which is not valid in the given context c:\Projekte\net7mma-112200\Common\Extensions\ExceptionExtensions.cs 95 17 Media.Common

I removed that line.

On Android it works, no artefacts and looking good but in my case it is slower than before and getting a delay which grows.
Right now I do not have the time to find out why. But the Picture is good.

On iOS I got timeouts or exceptions here:
rtpclient.method.cs
do received += justRecieved = socket.ReceiveFrom(buffer.Array, buffer.Offset + received, buffer.Count - received, System.Net.Sockets.SocketFlags.None, ref remote);

and in the catch block of action() here:
ios in action() catch
{System.Net.Sockets.SocketException: Invalid arguments
at System.Net.Sockets.Socket.SetSocketOption (SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue) [0x0001e] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net.Sockets/Socket.cs:2872
at System.Net.Sockets.Socket.set_SendBufferSize (Int32 value) [0x0001d] in /Users/builder/data/lanes/3234/d8bedd03/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net.Sockets/Socket.cs:687
at Media.Rtp.RtpClient+TransportContext+<>c__DisplayClass3e.<ConfigureRtpRtcpSocket>b__2f () [0x00000] in <filename unknown>:0
at Media.Common.Extensions.Exception.ExceptionExtensions.ResumeOnError (System.Action action) [0x00000] in c:\Projekte\net7mma-112200\Common\Extensions\ExceptionExtensions.cs:72 }

Those maybe the old iOS socket Errors. In the RTPClient.Constant FromSessionDescription I tried to set the localIP like it worked in the older versions before:

localIp = IPAddress.Parse("0.0.0.0");

but that did not help. Must be some socket Problems on iOS it seems. Sorry but this month I have no time to investigate, maybe next.

Just some short Infos for you, maybe it helps a bit.
Coordinator
Jul 26, 2016 at 12:25 PM
just as a FYI I think you could have also just built the project in release rather than commenting it out... You are linking against those libraries anyway in most cases.

You might have not have actually used a a reference to System.Diagnostics.StackFrame which is used in the code so the advanced linking process used by Xamarin might have removed it during such;

https://developer.xamarin.com/api/type/System.Diagnostics.StackFrame/

"0.0.0.0" binds to any IP; if you have your cell radio and your wifi radio then you have 2 interfaces... The cell is probably the primary....

Since the exception is caught and the logic included already handled any data received then I am not sure what you mean; with receive buffering even if the receive times out the data will be received later (on the next reception; so long as the receive buffer is not exceeded)

There is little I can do about this besides implement the socket layer myself;

If it would satisfy you I will create such an issue on the Issue Tracker such that it may be widely recognized;

-Julius
Marked as answer by juliusfriedman on 8/17/2016 at 9:28 AM