Kinect plugged, visual studio started and....

Topics: Question
Dec 1, 2014 at 7:33 PM
Hello,
and what to do after ?
i browsed your code and found a encode method in Media.RtspServer ---> RFC2250Media :

public byte[] WriteMPEGSequence(Bitmap img)

But you told me to use mjpeg and i can't find the encode fonction.


i am getting bite array for colordata from the kinect.
i have to encode these bites arrays and packetize them after

can you show me the way to make the kinect work with your librairy please ?
Dec 2, 2014 at 5:25 PM
ok then can you juste tell me wich method i have to modify ?
Coordinator
Dec 3, 2014 at 5:45 PM
MJPEG Is in the RFC2435 Frame, If I already told you then what other information do you need?

You shouldn't have to modify anything.
Marked as answer by juliusfriedman on 12/3/2014 at 9:45 AM
Dec 3, 2014 at 7:22 PM
Edited Dec 3, 2014 at 8:04 PM
i am french and i am beginner so sorry if i ask questions stupid question.
i tried to read your article i also read the part about your dog but i didn't understood all.....seems you didn't had luck with your neighbourg...hope karma will work against them..


i have to convert the bytes array received from the kinect to image object so as to pass it to the function :
public virtual void Packetize(System.Drawing.Image image) right ?
and after just have to launch the start method of my rfc2435 object ?


can you explain me the parameters that i to give to instantiate a new rfc2435 object;

public RFC2435Media(string name, string directory = null, bool watch = true)
Coordinator
Dec 4, 2014 at 11:28 PM
Thanks for the advice about Karma, I hope the same.

Yes the Packetize method will take a Image in whatever format it is and then Encode it as JPEG.

The constructor is documented, it will watch the given directory so you can just put new JPEG files inside and it will packetize and send them when they are created.

The Loop Property allows the stream to start over once it hits the end and no more pictures are available.

If you look at the Screen example in the test solution you will see how I take images of the Desktop and send them out in Rtp.

If you need anything else let me know!
Dec 28, 2014 at 9:54 AM
Hello,
The code that follow with the help of your media aggregation librairy is able to stream the colorstream from the kinect on vlc using the RFC2435 as you asked me...
This is a great step for me but it is working only for 20 seconds after that vlc screen get freeze on the client side.
i noticed that i had the same problems with your screen exemple in the test solution.
Do you think the problem is coming from my networks or my computers ?
No one have that kind of probleme with your librairy ?
here is the code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using Media;
using Microsoft.Kinect;
using System.Threading;
using System.IO;
using System.Drawing;
using System.Runtime.InteropServices;

namespace Portail
{
    /// <summary>
    /// Logique d'interaction pour MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        string assemblyLocation = System.Reflection.Assembly.GetExecutingAssembly().Location;
        static Media.Rtsp.RtspServer server = new Media.Rtsp.RtspServer(System.Net.IPAddress.Any, 554)
          {
              Logger = new Media.Rtsp.Server.RtspServerDebuggingLogger()
          };
        
Media.Rtsp.Server.MediaTypes.RFC2435Media screenShots = new Media.Rtsp.Server.MediaTypes.RFC2435Media("Screen", null, false, 640, 480, false);
        Thread updateVideoThread;
        byte[] colorData = null;
        KinectSensor myKinect;
        WriteableBitmap colorImageBitmap = null;
        


        public MainWindow()
        {
            InitializeComponent();
        }

        private void Window_Loaded(object sender, RoutedEventArgs e)
        {
            string localPath = System.IO.Path.GetDirectoryName(assemblyLocation);



           


            try
            {
                myKinect = KinectSensor.KinectSensors[0];
                myKinect.ColorStream.Enable();
                myKinect.Start();
            }
            catch
            {
                MessageBox.Show("Kinect initialise failed", "Camera Viewer");
                Application.Current.Shutdown();
            }
            myKinect.ColorStream.Enable();
            myKinect.Start();
            server.AddMedia(screenShots);
            updateVideoThread = new Thread(new ThreadStart(videoDisplay));
            updateVideoThread.IsBackground = true;
            updateVideoThread.Priority = ThreadPriority.Highest;



            server.Start();
            updateVideoThread.Start();

        }


        void videoDisplay()
        {
            while (server.Listening)
            {
                using (ColorImageFrame colorFrame = myKinect.ColorStream.OpenNextFrame(10))
                {
                    if (colorFrame == null) continue;

                    Bitmap bmp = ImageToBitmap(colorFrame);
                    


                    screenShots.Packetize(bmp);
                    //System.Threading.Thread.Sleep(0);

                    //REST
                    //System.Threading.Thread.Sleep(50);


                    
                }
            }

            // when we get here the program is ending – stop the sensor

          //  myKinect.Stop();
        }


    
       unsafe Bitmap ImageToBitmap(ColorImageFrame Image)
        {
            byte[] pixeldata = new byte[Image.PixelDataLength];
            Image.CopyPixelDataTo(pixeldata);
            Bitmap bmap = new Bitmap(Image.Width, Image.Height, System.Drawing.Imaging.PixelFormat.Format32bppArgb);
            System.Drawing.Imaging.BitmapData bmapdata = bmap.LockBits(
                new System.Drawing.Rectangle(0, 0, Image.Width, Image.Height),
                System.Drawing.Imaging.ImageLockMode.WriteOnly,
                bmap.PixelFormat);
            IntPtr ptr = bmapdata.Scan0;
            Marshal.Copy(pixeldata, 0, ptr, Image.PixelDataLength);
            bmap.UnlockBits(bmapdata);
            return bmap;
        }
    }
}
Coordinator
Jan 26, 2015 at 9:06 AM
110851 should resolve this!

let me know if you need anything else!
Feb 10, 2015 at 10:24 AM
Hello,
i tried the 110851 and other version that you post after that one.
And i still have trouve with the screen exemple and VLC.
The image on vlc is freezing after 10 seconds.
Feb 10, 2015 at 10:27 AM
May i use another RFC than RFC2450 to transmit video and audio data over rtp with your librairie ?
Coordinator
Feb 10, 2015 at 3:45 PM
The 10 second issue is an rtp identity issue which arrises due to invalid handling of the 'ssrc' parameter sent.

I am doing testing to see if there's a way to resolve it in a way which is compliant.

You can use quick time or m player or another player and the issue won't exist.

You can use any profiles you want.

I would suggest the raw video profile.

I haven't completely implemented it but its not hard and requires no decoder.

Vlc doesn't implement it either though so a viewer would have to be written, which also isn't too hard.

Let me know if you need anything else!
Marked as answer by juliusfriedman on 2/10/2015 at 7:45 AM
Coordinator
Feb 11, 2015 at 12:57 AM
Give the new release a try, I don't seem to have any issues with VLC there either with TCP or UDP.

Mplayer also works good.

If you need anything else let me know!

Hopefully this gives me a reason to implement these:

https://tools.ietf.org/html/rfc4175
https://tools.ietf.org/html/rfc4421

Hope to hear from you soon!
Feb 12, 2015 at 7:29 PM
hello,

Your "screen" example is woking perfectly with mplayer
i didn't know that software before.

i still have lag with the kinect stream but i work on it....

This is the question section so as programmer for sip sorcery i am sur you have a little idea about it:
I would like the program to be able to make voip call ?
is that possible and difficult to have to your librairie the capacity for doing that
Coordinator
Feb 12, 2015 at 8:29 PM
E.g. 'Mirror' :)

Thanks!

You have Lag with the Connect stream probably due to encoding and decoding to JPEG, It would work better if the data didn't have to be Encoded and Decoded each time, hence the uncompressed video RFC I linked to.... I just need to get around to it..

You would also then need a way to view it as I am not aware of a public viewer for uncompressed video (complaint to the RFC).

I am aware of SIP (the RFC) and the software Sip-sorcery (which is actually going to be using this stack for Rtp and Hopefully Rtsp soon)

You can send any type of Rtsp or Rtp data with this library although you can even send SIP or STUN or something else if you needed to, I just don't yet have classes to create the serialized messages for or handle their response for you. 'See SendData' on the RtpClient....

In short you can even use SOAP to setup the Rtp Session if you wanted or interface with any middle layer protocol you required (standard or proprietary).

After you can agree how the SETUP will work (RTSP, SIP, ETC) You will have to transmit and optionally receive the media from and to the end point.

Where you get your sound data from is up to you. (For now)

You can also receive any data, how you decode that data is again up to you. (For now)

I don't think it's any more difficult than it would be SIP Sorcery @ all and probably would actually allow you to use RTP over TCP which you can't do right now with SIP Sorcery.

So in short, if you wanted to make a phone call, I think you just need to send a SIP OFFER and the phone will ring.

Once the OFFER is accepted you can then setup the RTP the same way you would with SIP Sorcery...

If you need anything else let me know!
Marked as answer by juliusfriedman on 2/12/2015 at 12:29 PM