Support forUWP (Windows 10 Universal App) together with existing Xamarin Support

Topics: Suggestion
May 30, 2016 at 11:57 AM
Edited May 30, 2016 at 11:59 AM
Hi Julius,

as you know, I am using Xamarin Forms 2 to develop an App with AV Streaming on Android and iOS. And I am using your library (great stuff) for AV Streaming.

Now I have my Windows 10 Phone on the table.

I installed the Windows Software Development Kit (SDK) for Windows 10 from here:
https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk

Then I tried to add your DLLs to my UWP Project I added to my app.

It then tells me:

A reference ... MediaCodec.dll could not be added. The Project targets ".NetCore" while the file reference Targets ".NetFramework"...

I talked to Google, which told me in a UWP app you can only add DLL which is build as a portable class library.

I opened your Project properties with the build Settings and could not Switch it to something like that.

Just for a try, I created a new solution with my visual Studio 2015 and chosed "Class library (Portalble for iOS, Android and Windows" as Project template.

After that, in the Project properties build Settings, I have multiple Targets including .Net Framework 4.5, Windows Phone, Xamarin Android, Xamarin iOS.

So I think this Kind would both Support .net Framework like before, and all other xamarin etc stuff. You would be totally compatible :-)

Would you consider in making this Change? I could then use it for Windows phone, too.
And I could give you Feedback again for Windows phone and also have an Argument for testing new versions of your library :-)

Thanks a lot and have a nice week!
Coordinator
May 31, 2016 at 9:07 PM
I will check into Universal Windows Projects later this week hopefully, as you stated it's fairly easy to do when compiling the library for another platform, it should just be a matter of having the appropriate configuration in the project's and possibly the solution, I don't think it's more than a few lines....

If you post up the modified portion of the 'sln' and 'proj' files required I will be able to integrate it sooner that I would if I had to set everything up myself.

Finally Media.Codec.dll is not really tested or ready for production use, how are you using it? Maybe you could also provide some examples to help with the testing / documentation.

-Julius
Marked as answer by juliusfriedman on 5/31/2016 at 2:07 PM
Jun 1, 2016 at 6:36 AM
This would be great, thanks Julius!

Right now, we are using the RTPClient of yours to receive the AV stream using Framechanged and sending Audio data using SendRTPPacket, works great under Xamarin with both Android and iOS.

For Receiving we are using Depacketize and then give it to the System Decoders.

For Sending we are Creating a new RTPPacket from a byte-Array, increase the Headers Sequencenumber and Timestamp, Setting a unique SynchronizationSourceIdentifier and then send it with SendRtpPacket.

For the Windows 10 Phone UWP we would like to try to use your codec Image class to Display the Video stream, if we get your assemblies working with it.

Thanks :-)
Coordinator
Jun 1, 2016 at 4:50 PM
I updates the latest code release.

I added a project.json but take note that I think that it won't even be required soon:

https://blogs.msdn.microsoft.com/dotnet/2016/05/23/changes-to-project-json/

I still don't really understand what is the difficultly using my library in Windows Phone, I will evidently have to install the emulator and phone dev kits to try and see what the hold up is...

There shouldn't be anything in my code which doesn't work on Windows Phone unless your talking about Concepts which may never make the production cut.

Media.Codec will eventually be production ready but there are several other issues which need to be addressed before work on Codecs start again, for instance the RtpFrame depacketization API coupled with the ability to packetize or depacketize arbitrary data (possibly according to a rtp profile if desired)

There is also the matter of Getting the SipClient to work with SessionDescriptions, the SipClient can send messages and receive them but it needs the methods to JoinSession(SessionDescription) Join(MediaDescription) and a few others.

-Julius
Marked as answer by juliusfriedman on 6/1/2016 at 9:50 AM
Jun 2, 2016 at 6:32 AM
My Solution includes 4 Projects:

A shared library (containing views, Control, viewmodels with xamarin c# and xaml)

A Droid Project
A iOS Project

and new:
A UWP (Universal Windows) Project, wihich is meant for Windows Mobile / Windows 10 Phone.

I just downloaded your latest Version to be sure.

When I add one of your assemblies I get:
A reference to ...Media.codec.dll could not be added. The Project Targets .NetCore while the file reference Targets .Net Framework. This is a not supported Scenario.
So I think I Need a portable class library, which would Support both Scenarios? Or do you have an other Suggestion?

I could try to do this by myself. But then it would be hard to get later updates of your library when I would have to do this every time...

Thanks for your help :-)
Jun 2, 2016 at 7:22 AM
PS: I also copied the new assemblies from 112091 to my Android Project. I got some broken Audio, and the Video Shows something green, then a part of the Picture with artefacts still. Just to let you know :-)
Coordinator
Jun 2, 2016 at 11:15 AM
Edited Jun 2, 2016 at 11:25 AM
The broken audio and video, is that under release mode or debug and can you provide outputs of the decoder so I can see what exactly it complains about.

The UWP requirements are 100 times less than they used to be, you used to have to have different code bases. I checked winphone out originally and very quickly deemed it useless with respect to .Net in general for that reason. Fortunately that has mostly changed.

That's resolved depending on how advanced your scenario was but the main point is that it's a solution level configuration for the most part... you should be able to replace net core with whatever the target framework is for Windows Phone now in the project and it will work.

To make it seamlessly integrate a duplicate csproj would be created and then be opened in the ide for Windows phones.

That's not a big update.. and I think that the only benefit is that it takes between 30 seconds and 3 minutes or less to create the new project and add the exisitng files.

Since windows phone is the only environment where you have to do this outside of MicroFx I have been reluctant to provide such facilities as it doesn't benefit the project and it takes additional time to test and support.

If a separate code base was required I could see your point and might oblige more quickly depending on the direction.

Right now you don't really need different code bases but you do need to have a project which can be built under the framework derivations...

Just create a new project for each that you need starting with Common and go from there until your done.

Send me copies of the new cjproj files, I will differ them and add anything else necessary and the result is that there will be a csproj and .windows.phone.cs project.

I will make a Windows phone directory and keep the sln and csproj's there and use them when required for building that version.

Just like I will have to do for silver light and wpf eventually also.

What is the need for Media.Codec.dll anyway? Windows Phone exposes it's media capabilities in Windows.Media similar to silver light which has its own concept of encoders and decoders.

You could also just pass the data to the player in a complaint container format which would eliminate most if not all of the platform level difficulties.
Marked as answer by juliusfriedman on 6/2/2016 at 4:15 AM
Jun 3, 2016 at 9:24 AM
Edited Jun 3, 2016 at 9:30 AM
Hello Julius,

I found this from Microsoft, where they describe how to port to .NET Core and they talk about Advantages / disadvantages and requirements:

https://blogs.msdn.microsoft.com/dotnet/2016/02/10/porting-to-net-core/

Just to check your Suggestion, I made a new solution for trying to load your assemblies:

I added various Projects (using Visual Studio 2015):

Blank App Universal Windows
Blank App Universal Windows 8.1
DirectX12App Universal Windows

None of it opened your assemblies, all had the same Problem.

So I added a new portalble class library. I added all your assemblies to the references, and then tried to add that assembly to my exiting app UWP Project, but again it did not work...

I can delete and add the Windows phone Project, because it is empty still, just do not know how to add it in order to be able to deploy on the phone and get your assemblies loaded and my shared xamarin Project...

I added the existing UWP Project to my dropbox, maybe you can look at the Project files if it helps:

https://www.dropbox.com/s/9r1j4kut9tox0oh/sks_Client.UWP.zip?dl=0
Coordinator
Jun 3, 2016 at 1:59 PM
I will take a look as soon as possible.

I do want my project to be useful in as many environments as possible...

I imagine all you have to do is important my exisitng project and build it using the Windows Phone as the target rather than the full framework to link the libraries correctly to the runtime that will be using them.

You can't use binaries which were targeted at 4.5 or 4.6 full framework for the full fx on Windows phone.

Same way for silver light requires additional libs or wpf or Windows forms for that matter.
Marked as answer by juliusfriedman on 6/3/2016 at 6:59 AM
Coordinator
Jun 4, 2016 at 12:08 AM
It looks like it's some re-arranging in the project and solution files mostly...

The big changes in the code come into play because of the change in the RunTime with Reflection mostly.

Rtcp uses GetType so I will have to work on that to revise it to use TypeInfo where required.

I will do the same for other places probably next week!

-Julius
Marked as answer by juliusfriedman on 6/3/2016 at 5:08 PM
Jun 4, 2016 at 5:44 AM
Hi Julius,

thank you, that sounds good! It would be great if we could get a first DLL for testing if we can add it to our Project. The RTPClient would be great for a start :-)

Later for the Windows phone if possible we would like to use your codec Image DLL this time, too, to draw the Video if possible...


Have a nice Weekend!
Coordinator
Jun 6, 2016 at 3:19 PM
Edited Jun 6, 2016 at 3:57 PM
I looked into this over the weekend and I have a few gripes...

1) I have no problem making changes where I use Type to get members or other places where I use GetMethods to use TypeInfo rather than type.

Since Microsoft is serious about splitting Type for various reasons then this will have to be eventually anyway.

I would imagine that .Net 5 will remove the compatibility <= 4.6 and force all code to use the new TypeInfo for that purpose, if not then there is no reason to change any code to use TypeInfo vs Type because it will semantically be the same except in advanced usage scenarios.

2) System.Net.IPAddress is missing, it can be added back quite easily but there is a bigger issue which effects the project;

3) System.IO is gone... there is StorageFile and StorageFolder, That means no FileStream.

4) System.Runtime.InteropServices.MethodImpl is different meaning MethodImplOptions.Synchronized cannot be used, thus all places this code is used now has to change to 'lock(this)' which is the same semantic but really unnecessary.

Before I go on and on and on, let me just say this.

I simply will not convert to using Windows.Networking or Windows.Anything.

Quite frankly it's bullshit that I have to put up with that I can use the TcpClient but not Socket, TcpClient has to have a 'Socket' to work.

Why Socket couldn't be stubbed out to use TcpClient or UdpClient underneath is beyond me, totally beyond me.

Thus I would have to Make the 'Sockets' solution the 'shared' library in this scenario, I started this before and stopped because it's no longer needed except when you want to do something advanced e.g. Raw Sockets especially since Windows Phone 10 allows Socket to be used and also contains IPAddress.

To support Windows 8 Phone I would have to implement UdpSocket, TcpSocket etc and use the System.Windows.Networking classes where Socket was not support just to use TcpClient...

This is resolved in Windows Phone 10 but not in Windows Phone 8... So essentially what I am saying is that Windows Phone 8 cannot be targeted by this library due to Microsoft's own design choices.

Windows 10 Phone recognizes some of this and fixes it and thus you can use the library as is in Windows 10 Phone once some minor changes to where Reflection is used.

I will compensate for the reflection differences by refactoring my code to use the new methods but again I think that there shouldn't be this requirement, it should be stubbed like it is on the Full Fx for compatibility or it should be changed to ensure everyone is using the new API which would eliminate this difference totally.

I think it's extremely stupid and an oversight that I can take a Windows 8 phone and put Android on it and run the code but I can't run it on Windows 8 stock?

That being said... I honestly believe they are 75% of the way there to allowing Full FX and Net Core to be 100% inter-operable

If you take a look at the adapter there is really only such a small amount of differences it's almost painful to see such large pieces of functionality missing:

See github

Microsoft even indicates that they can still load Full FX assemblies and call their methods so why not provide that bridge directly to user code?

I believe and hope that .Net 5.0 will resolve all of this and remove such 'Targeting' issues.

If you can run the code on Full Fx and Mono you should be able to run it on the phone also...

The phone specific API features can and should be put into a separate library e.g. Phone where they can be abstracted appropriately.

I shouldn't have to change my entire code base just because the code is running on a phone... Just like with Android.

In closing

The changes for reflection are so minor I can probably pump them out today or tomorrow.

After that you should be able to target Windows 10 just like Full Fx without any issues.

Windows Phone 8 is just not supported and I am sure you will find many other developers who also take the same stance.

Finally,

Univeral Windows Platform

https://developer.microsoft.com/en-us/windows/downloads/sdk-archive

This is how you can Target Windows 10 and subsequently other platforms from a single project.

Unfortunately you need Visual Studio 2015 to do so...
Marked as answer by juliusfriedman on 6/6/2016 at 8:20 AM
Coordinator
Jun 6, 2016 at 4:14 PM
Edited Jun 6, 2016 at 6:14 PM
I made an update which should resolve this for Windows 10 / UWP environments.

These changes should rectify any issues with UWP / Windows 10 / Dot Net Core and as you can see the change is quite minimal.

If you notice anyplace else where I fail to use TypeInfo just me know and I will compensate.


Update

You can see the totality of the effects of the System.Type changes here:

Reflection API Changes

There is only minimal more work to be done to make it explicitly compatible e.g. I have to either provide extension methods to filter access and member names or I have to iterate all and break where I have the member I want.

As we already have Linq it's almost not worth creating the extension method right now but in the future I will add ReflectionExtensions which will pretty much emulate the missing methods from the Reflection API where possible.

E.g.

I have recently changes everything to use DeclaredFields, DeclaredMembers, DeclaredConstructors and DeclaredProperties.

I will post the update soon, I just am working more in the Intrinsics layout right now and how that will be effected by switching to the new reflection API.
Marked as answer by juliusfriedman on 6/6/2016 at 9:14 AM
Coordinator
Jun 6, 2016 at 10:17 PM
Edited Jun 7, 2016 at 1:01 AM
I made another update which should rectify all the issues with Reflection which should essentially be EVERYTHING which prevents the library from being used in a portable environment. (With ofcourse the exception of Marshal, but that is a known bug)

I will see about changing the CSPROJ to reference the Portable Subset later on but only if it doesn't cause build problems without it or if there's a way to work around any issues with Marshal but I don't use it outside of the Native code paths anyway which is completely optional.

I further have allowed for MethodInfo to be able to replaced directly in CLR methods with InjectionHelper which doesn't even require mmap or mprotect or any other PInvoke and thus should also work in Windows Phone as well as Android etc.

Lastly, I show how I can provide the Method and Fallback logic for Intrinsics and that they can automatically revert when required due to hardware support or otherwise.

I implement Rdtsc again via PlatformRdtsc to show how the new inheritance will work.

I will make the change to other intrinsics shortly and I will be adding more intrinsics for bit manipulation and the like.

Let me know what you think!
Marked as answer by juliusfriedman on 6/6/2016 at 3:17 PM
Jun 7, 2016 at 11:08 AM
Edited Jun 7, 2016 at 11:22 AM
Hi Julius,

thanks a lot for your help and updates.

If I mentioned Windows Phone 8 somewhere, then this was a wrong info sorry, we are only targeting Windows 10 Phones (UWP) not 8 phones, and the already working Android and iOS platforms.

But waiting for .NET Framework 5.0 will take much too Long for our app ;-)

We currently directly use the RTP and H.264 Codec only and because of it's dependencies I added These Assemblies from your library:

Media.Codec.dll
Media.Codecs.Video.dll
Media.Codecs.Video.H264.dll
Media.Common.dll
Media.Compression.dll
Media.Container.dll
Media.Cryptography.dll
Media.Http.dll
Media.Ntp.dll
Media.Rtp.dll
Media.Sdp.dll
Media.Sip.dll
Media.Sockets.dll

and I guess for Windows phone 10 additional Codecs.Image. to fully use your solution if possible.

I currently cannot add those because of the not matching .NetFramework 4.6 to .NET Core incompatibilty issue as you know. If I should try myself or if I can help you somehow please let me know. Thanks a lot :-)
Coordinator
Jun 7, 2016 at 11:14 AM
Good to know! I believe that I'm on track to support wp10.

I may also have a way to work around the visual studio 2013 issues by manually building the project where required.

The Marshal issue is interesting, Windows Phone 8 can even access it, not sure why it's not portable.

In cases where Marshal cannot be used there always still is an alternative but I will worry about that when I need it.

Let me know if you still have issues targeting windows 10 and I will compensate further if required.
Marked as answer by juliusfriedman on 6/7/2016 at 4:15 AM
Jun 7, 2016 at 11:30 AM
Edited Jun 7, 2016 at 11:31 AM
I am still having the Problem I cannot add areference to for example Media.RTP or Media.Codec Assembly into my new created UWP Project.
A reference to ...Media.codec.dll could not be added. The Project Targets .NetCore while the file reference Targets .Net Framework. This is a not supported Scenario.
If I solve that by myself in your code I would have Problems later with testing new updates ;-)
Coordinator
Jun 7, 2016 at 11:45 AM
Edited Jun 7, 2016 at 12:18 PM
The code wouldn't change... Only the CSPROJ would change, you need to have an indication in the '<PropertyGroup>' to build the library as a Portable Class...

Make a new CSPROJ and then add the property group as required to the project... I will double check that is the only thing you have to do after I get time to install the SDK and build again.

NATIVE already has it's own build path so the issue is something else...

The CSPROJ in my project is targeting the full fx so as I stated you need to have a property group to allow building under the UWP / Portable Subset.

I will modify the Property Group in the CSPROJ when I get a chance, it's literally about 10 - 15 lines to add.

Then when you build you must choose to target the UWP Platform and not The Full Framework of Net Core, which is specified from the Build Properties of the project.

Which, Actually I think if you just go there and change the target framework it will just work just like you would do if you changed from 4.6 to 4.5 etc.

I should have time to confirm and address any remaining issues this week.
Marked as answer by juliusfriedman on 6/7/2016 at 5:14 AM
Jun 13, 2016 at 6:16 AM
Hi Julius,

any News on the compatible assemblies yet? I Need to start with Windows phone now. :-)

Thanks a lot!
Coordinator
Jun 13, 2016 at 1:36 PM
I wrote in the other thread...

I am fairly confident at this point that if you target just one of those environments instead of the portable environment that the project will build and function as it is, an if not I apologize in advance and need to understand why so I can address it.

You can't use the Portable Subset right now (as it's not really Portable it just eliminated classes such as, FileStream, IPAddress, Sockets which are defined by the Platform)

Soon, how soon I am not sure but the end goal I imagine is that there will be much less fragmentation and that you will just have to worry about if .Net can run on the Device and what Profile, until then you need to build the library for use in the Environment you plan on using it in.

If you want to use this library for Windows Phone you need to target the .Net Framework subset which is present on the phone and not the Portable Subset because again the Portable Subset isn't even really 100% portable at the current time.

See

Among many others...

The only way to work around this is to target a different framework profile (besides the portable subset)

There are no other changes to the assemblies which are required.
Marked as answer by juliusfriedman on 6/13/2016 at 6:36 AM
Jun 29, 2016 at 11:47 AM
Hi Julius,

thanks a lot for your Infos and help. I took too much time on the RTP stuff last weeks, and have to do other important stuff the rest of this week.

Next week I got a free week (Holiday).

I will get back to you after next week. Thanks a lot and have a good time!
Coordinator
Jun 29, 2016 at 2:07 PM
Np, thank you also and have a great holiday!
Marked as answer by juliusfriedman on 6/29/2016 at 7:07 AM