Irrlicht Beginner 2D. Live video tweaking.

Announce new projects or updates of Irrlicht Engine related tools, games, and applications.
Also check the Wiki
Post Reply
irrlicht_ist_toll
Posts: 11
Joined: Fri Oct 09, 2015 3:17 pm

Irrlicht Beginner 2D. Live video tweaking.

Post by irrlicht_ist_toll »

Hi all I've used several 3D engines in the past and have some experience with 3d meshes and all.
But I'm just starting with Irrlicht and win32, getting away from .NET

For now I'm using irrlicht to show some 2D stuff on a texture on a viewport as part of my
visual recognition project.

Image

This is really fun, I'm just getting the live video frame data from a webcam into a buffer, and after doing any desired modification to the data, I lock and update the texture.

Code: Select all

 ptr= (unsigned int*)(texture->lock(ETLM_READ_WRITE,0)); 
 
   for(i=0;i<size;i++){ ptr[i] = source[i]; }   //copy
   //for(i=0;i<size;i++){ ptr[i] = 0xFFFF0000;} //red-fill-test
 
 texture->unlock();//
One of the best parts is that the program has worked consistently on different computers I've tested on, without having to install anything.
chronologicaldot
Competition winner
Posts: 684
Joined: Mon Sep 10, 2012 8:51 am

Re: Irrlicht Beginner 2D. Live video tweaking.

Post by chronologicaldot »

Cool beans :D
What did you use to decode the video?
irrlicht_ist_toll
Posts: 11
Joined: Fri Oct 09, 2015 3:17 pm

Re: Irrlicht Beginner 2D. Live video tweaking.

Post by irrlicht_ist_toll »

There is this whole convoluted formula to go from YUY2 to RGB. See below.
YUY2 goes like y0, u , y1, v,
so Y luma is per-pixel while uv is shared by two pixels.
The conversion is pretty convoluted but it's performing surprisingly fast.
I guess one could also use a GPU shader on the application, but i haven't tried that since it doesn't really benefit me here, as i need the RGB data in application memory to manipulate it.

Code: Select all

 
LRESULT CALLBACK Camera_onFrameCall(HWND hWnd, LPVIDEOHDR lpVHdr){
 
 //Copy rgb version to application buffer
 //This convertion is pretty long but still performed really fast.
 //still need to optimize this 
 
 unsigned int s= lpVHdr->dwBufferLength;
 
 if(camera.user_frame_array_rgb != NULL){//if user set pointer
   int rgb,r,g,b,p;
   int y0,u,y1,v, C, C2, D, E;
   //
   p=0;
   for(i=0;i<s;i+=2){
     y0 = lpVHdr->lpData[p];
      u = lpVHdr->lpData[p+1];
     y1 = lpVHdr->lpData[p+2];
      v = lpVHdr->lpData[p+3];
     p +=4;//yuyu2 is 16bpp
     if(p>=s){break;}
     //
     C  = y0 - 16;  
     C2 = y1 - 16;  
     D  = u  - 128;
     E  = v  - 128;
     //
     //Pixel 1
         r = (298*C+409*E+128)/256;
     if(r<0){r=0;}if(r>255){r=255;}
     g = (298*C-100*D-208*E+128)/256;
     if(g<0){g=0;}if(g>255){g=255;}
     b = (298*C+516*D+128)/256;
     if(b<0){b=0;}if(b>255){b=255;}
         camera.user_frame_array_rgb[i]=(r<<16)|(g<<8)|(b);  //rgb
     //
     //Pixel 2
         r = (298*C2+409*E+128)/256;
     if(r<0){r=0;}if(r>255){r=255;}
         g = (298*C2-100*D-208*E+128)/256;
     if(g<0){g=0;}if(g>255){g=255;}
         b = (298*C2+516*D+128)/256;
     if(b<0){b=0;}if(b>255){b=255;}
     camera.user_frame_array_rgb[i+1]=(r<<16)|(g<<8)|(b);  //rgb
     //
     //camera.user_frame_array_rgb[i]=0xFFFF0000;//red-fill-test
   }//
 }//
}
chronologicaldot
Competition winner
Posts: 684
Joined: Mon Sep 10, 2012 8:51 am

Re: Irrlicht Beginner 2D. Live video tweaking.

Post by chronologicaldot »

Messy indeed. Good job at making it work.
Post Reply