Theora video wrapper

Post those lines of code you feel like sharing or find what you require for your project here; or simply use them as tutorials.
Post Reply
zet.dp.ua
Posts: 66
Joined: Sat Jul 07, 2007 8:10 am

Theora video wrapper

Post by zet.dp.ua »

If someone is interested:

Simple version of ogg-theora wrapper class to play video files.
YUV to RGB conversion function was taken from nebuladevice.

OnUpdate function decodes next frame and updates texture's buffer and texture. It's a question how to do update in the right way to minimize memory lags in multithreaded game design

Image

demo with source is here:
http://www.mediafire.com/?dmodc4tgtzd
and here:
http://www.megaupload.com/?d=ZMKNRHPL

header file

Code: Select all

// Theora lib wrapper class

#ifndef __C_THEORA_PALYER_H_INCLUDED__
#define __C_THEORA_PALYER_H_INCLUDED__


//! Irrlicht
#include "irrlicht.h"

using namespace irr;

#include "theora/theora.h"

enum CTheoraPlayerStates
{
  CTPS_IDLE = 0,
  CTPS_PLAYING,
};

class CTheoraPlayer 
{
public:

  CTheoraPlayer(IrrlichtDevice* device);
  ~CTheoraPlayer();

  //! Create
  bool  Create();
  //! Get next frame and update texture
  bool  OnUpdate(u32 timeMs);

  //! Get playing state
  CTheoraPlayerStates GetState() { return iState; }

  //! Get texture interface
  video::ITexture* GetTexture();
  //! Get image interface
  video::IImage*   GetImage();

  //! Start playing
  bool  Play(const c8* fileName, bool loop = true);
  bool  Stop();


private:

  bool  ProcessNextFrame();

  void  UpdateBuffer();
  void  UpdateTexture();

  bool  PrepareOgg();
  bool  PrepareBuffers();

  //! Helper Theora functions
  s32   queue_page(ogg_page *page);
  s32   buffer_data();

  // Irrlicht classes
  IrrlichtDevice*  iIrrDevice;
  video::IVideoDriver* iIrrVideoDriver;
  ILogger*         iIrrLog;

  video::ITexture* iBlankTexture;
  video::ITexture* iTexture;
  video::IImage*   iImage;
  io::IReadFile*   iVideoFile;

  /* Ogg and codec state for demux/decode */
  ogg_sync_state   iOggSyncState;
  ogg_page         iOggPage;
  ogg_stream_state iOggStreamState;
  theora_info      iTheoraInfo;
  theora_comment   iTheoraComment;
  theora_state     iTheoraDecoderState;
  int              iTheoraPacketsCount;
  ogg_packet       iOggPacket;


  bool             iIsLooped;
  u32              iCurrFrame;
  u32              iCurrTime;           


  CTheoraPlayerStates   iState;
};

#endif //__C_THEORA_PALYER_H_INCLUDED__
cpp file

Code: Select all

#include "CTheoraPlayer.h"

//#include "OHRTimer.h"
//
//OHRTimer gTimer;

//------------------------------------------------------------------------------
//! Ctor
//! 
CTheoraPlayer::CTheoraPlayer(IrrlichtDevice* device) :
  iIrrDevice(device), iTexture(NULL), 
  iVideoFile(NULL), iTheoraPacketsCount(0), 
  iState(CTPS_IDLE), iImage(NULL)
{
  iIrrVideoDriver = device->getVideoDriver();
  iIrrLog = device->getLogger();
}

//------------------------------------------------------------------------------
//! Dtor
//! 
CTheoraPlayer::~CTheoraPlayer()
{
  Stop();
}

//------------------------------------------------------------------------------
//! Create
//! 
bool CTheoraPlayer::Create()
{
  // create blank texture
  core::dimension2di size(1,1);
  iBlankTexture = iIrrVideoDriver->addTexture(
    size, "CTheoraPlayerBlankTexture");

  u8* data = (u8*) iBlankTexture->lock();
  if (data != NULL)
  {
    memset(data, 0xFF, iBlankTexture->getPitch()*iBlankTexture->getSize().Height);
    iBlankTexture->unlock();
  }

  return true;
}

//------------------------------------------------------------------------------
//! Play
//! 
bool CTheoraPlayer::Play(const c8* fileName, bool loop)
{
  // stop previous video
  Stop();

  iIsLooped = loop;

  // try to open file
  iVideoFile = iIrrDevice->getFileSystem()->createAndOpenFile(fileName);
  if (iVideoFile == NULL)
  {
    // failed to open file
    return false;
  }

  // initialize ogg decoder
  if (!PrepareOgg())
    return false;

  // prepare texture and buffer
  if (!PrepareBuffers())
    return false;

  iState = CTPS_PLAYING;

  return true;
}

//------------------------------------------------------------------------------
//! Stop
//! 
bool CTheoraPlayer::Stop()
{
  if (iState == CTPS_IDLE)
    return true;

  if (iTheoraPacketsCount)
  {
    ogg_stream_clear(&iOggStreamState);
    theora_clear(&iTheoraDecoderState);
    theora_comment_clear(&iTheoraComment);
    theora_info_clear(&iTheoraInfo);
  }
  ogg_sync_clear(&iOggSyncState);

  if (iVideoFile != NULL) { iVideoFile->drop(); iVideoFile = NULL; }
  if (iTexture != NULL) { iTexture->drop(); iTexture = NULL; }
  if (iImage != NULL) { iImage->drop(); iImage = NULL; }

  iState = CTPS_IDLE;

  return true;
}

//------------------------------------------------------------------------------
//! PrepareOgg
//! 
bool CTheoraPlayer::PrepareOgg()
{
  // set start pos
  iVideoFile->seek(0);

  /*
  Ok, Ogg parsing. The idea here is we have a bitstream
  that is made up of Ogg pages. The libogg sync layer will
  find them for us. There may be pages from several logical
  streams interleaved; we find the first theora stream and
  ignore any others.

  Then we pass the pages for our stream to the libogg stream
  layer which assembles our original set of packets out of
  them. It's the packets that libtheora actually knows how
  to handle.
  */

  int stateflag = 0;

  /* start up Ogg stream synchronization layer */
  ogg_sync_init(&iOggSyncState);

  /* init supporting Theora structures needed in header parsing */
  theora_comment_init(&iTheoraComment);
  theora_info_init(&iTheoraInfo);

  iTheoraPacketsCount = 0;
  iCurrFrame = 0;
  iCurrTime = 0;


  /* Ogg file open; parse the headers */

  /* Vorbis and Theora both depend on some initial header packets
  for decoder setup and initialization. We retrieve these first
  before entering the main decode loop. */

  /* Only interested in Theora streams */
  while (!stateflag)
  {
    int ret = buffer_data();
    if (ret == 0)
      break;

    while (ogg_sync_pageout(&iOggSyncState,&iOggPage)>0)
    {
      ogg_stream_state test;

      /* is this a mandated initial header? If not, stop parsing */
      if (!ogg_page_bos(&iOggPage))
      {
        /* don't leak the page; get it into the appropriate stream */
        queue_page(&iOggPage);
        stateflag=1;
        break;
      }

      ogg_stream_init(&test,ogg_page_serialno(&iOggPage));
      ogg_stream_pagein(&test,&iOggPage);
      ogg_stream_packetout(&test,&iOggPacket);

      /* identify the codec: try theora */
      if (!iTheoraPacketsCount && theora_decode_header(&iTheoraInfo, &iTheoraComment, &iOggPacket)>=0)
      {
        /* it is theora -- save this stream state */
        memcpy(&iOggStreamState,&test,sizeof(test));
        iTheoraPacketsCount = 1;
      }
      else
      {
        /* whatever it is, we don't care about it */
        ogg_stream_clear(&test);
      }
    }
    /* fall through to non-initial page parsing */
  }

  /* we're expecting more header packets. */
  while (iTheoraPacketsCount && iTheoraPacketsCount<3)
  {
    int ret;

    /* look for further theora headers */
    while (iTheoraPacketsCount && (iTheoraPacketsCount < 3))
    {
      ret = ogg_stream_packetout(&iOggStreamState, &iOggPacket);
      if (ret < 0) 
      {
        iIrrLog->log("CTheoraPlayer: Error parsing Theora stream headers; corrupt stream?\n");
        return false;
      }
      if (theora_decode_header(&iTheoraInfo, &iTheoraComment, &iOggPacket))
      {
        iIrrLog->log("CTheoraPlayer: Error parsing Theora stream headers; corrupt stream?\n");
        return false;
      }
      iTheoraPacketsCount++;
    }


    /* The header pages/packets will arrive before anything else we
    care about, or the stream is not obeying spec */
    if(ogg_sync_pageout(&iOggSyncState, &iOggPage)>0)
    {
      queue_page(&iOggPage); /* demux into the stream state */
    }
    else
    {
      int ret=buffer_data(); /* need more data */
      if (ret == 0)
      {
        iIrrLog->log("CTheoraPlayer: End of file while searching for codec headers.\n");
        return false;
      }
    }
  }

  /* Now we have all the required headers. initialize the decoder. */
  if (iTheoraPacketsCount)
  {
    theora_decode_init(&iTheoraDecoderState, &iTheoraInfo);
    static c8 buffer[512] = {0};
    snprintf(buffer, 511, "Ogg logical stream %x is Theora %dx%d %.02f fps video\nEncoded frame content is %dx%d with %dx%d offset\n",
      (unsigned int)iOggStreamState.serialno,iTheoraInfo.width,iTheoraInfo.height, 
      (double)iTheoraInfo.fps_numerator/iTheoraInfo.fps_denominator,
      iTheoraInfo.frame_width, iTheoraInfo.frame_height, iTheoraInfo.offset_x, iTheoraInfo.offset_y);
    iIrrLog->log(buffer);
  }
  else
  {
    /* tear down the partial theora setup */
    theora_info_clear(&iTheoraInfo);
    theora_comment_clear(&iTheoraComment);
  }

  /* queue any remaining pages from data we buffered but that did not
  contain headers */
  while (ogg_sync_pageout(&iOggSyncState, &iOggPage) > 0)
  {
    queue_page(&iOggPage);
  }

  return true;
}

//------------------------------------------------------------------------------
//! Create texture if needed and buffer for it.
//! If texture output is on we create buffer with the same parameters
//! to use memcpy function in synchronization section
bool CTheoraPlayer::PrepareBuffers()
{
  // create texture
  // unset auto mipmaps flag
  bool oldMipmapFlag = iIrrVideoDriver->getTextureCreationFlag(
    video::ETCF_CREATE_MIP_MAPS);
  iIrrDevice->getVideoDriver()->setTextureCreationFlag(
    video::ETCF_CREATE_MIP_MAPS, false);

  // create texture
  core::dimension2d<s32> size(iTheoraInfo.frame_width, iTheoraInfo.frame_height);
  iTexture = iIrrVideoDriver->addTexture(
    size, "CTheoraTexture", video::ECF_R8G8B8);

  // restore auto mipmaps flag
  iIrrVideoDriver->setTextureCreationFlag(
    video::ETCF_CREATE_MIP_MAPS, oldMipmapFlag);

  if (iTexture == NULL)
  {
    iIrrLog->log("CTheoraPlayer: Failed to create texture\n", ELL_ERROR);
    return false;
  }
  iTexture->grab();

  // we should care what buffer and texture 
  // have the same size so we can use memcpy operation
  // create buffer image
  video::ECOLOR_FORMAT colorFormat = iTexture->getColorFormat();
  core::dimension2d<s32> imageSize = iTexture->getSize();
  u32 pitch = iTexture->getPitch();
  u32 height = imageSize.Height;

  u32 bufferSize = pitch*height;
  void* textureBuffer = (void*) new u8[bufferSize];
  if (textureBuffer == NULL)
  {
    iIrrLog->log("CTheoraPlayer: Failed to allocate buffer memory\n", ELL_ERROR);
    return false;
  }

  // create IImage, i couldn't get working parameters for createImageFromData()
  // to use textureBuffer as Data in CImage class and correctly delete it
  iImage = iIrrVideoDriver->createImageFromData(
    colorFormat, imageSize, textureBuffer);
  delete [] textureBuffer;
  if (iImage == NULL)
  {
    iIrrLog->log("CTheoraPlayer: Failed to create image\n", ELL_ERROR);
    return false;
  }

  return true;
}

//------------------------------------------------------------------------------
//! OnUpdate
//! Get next frame and update texture
bool CTheoraPlayer::OnUpdate(u32 timeMs)
{
  if (iState != CTPS_PLAYING)
    return true;

  //timeMs = 1000;

  // calculate how many frames need to be decoded
  iCurrTime += timeMs;
  u32 neededFrame = (u32)(1.0f*(iCurrTime/1000.0f)*iTheoraInfo.fps_numerator/iTheoraInfo.fps_denominator);
  u32 framesToDo = neededFrame - iCurrFrame;

  // now decode
  u32 i, wasLastFrameDecoded = false;
  for (i = 0; i < framesToDo; i++)
  {
    wasLastFrameDecoded = ProcessNextFrame();
    if (iState != CTPS_PLAYING)
      break;
  };

  if (wasLastFrameDecoded)
  {
    /* dumpvideo frame */
    UpdateBuffer();
    UpdateTexture();
  }

  return true;
}

//------------------------------------------------------------------------------
//! ProcessNextFrame
//! Decode next frame from ogg input stream.
//! Returns true if frame was successfully decoded
bool CTheoraPlayer::ProcessNextFrame()
{
  /*
  It's one Theora packet per frame, so this is pretty 
  straightforward if we're not trying to maintain sync
  with other multiplexed streams.

  the videobuf_ready flag is used to maintain the input
  buffer in the libogg stream state. If there's no output
  frame available at the end of the decode step, we must
  need more input data. We could simplify this by just 
  using the return code on ogg_page_packetout(), but the
  flag system extends easily to the case were you care
  about more than one multiplexed stream (like with audio
  playback). In that case, just maintain a flag for each
  decoder you care about, and pull data when any one of
  them stalls.

  videobuf_time holds the presentation time of the currently
  buffered video frame. We ignore this value.
  */

  /* single frame video buffering */
  int          videobuf_ready=0;
  ogg_int64_t  videobuf_granulepos=-1;
  double       videobuf_time=0;

  while (!videobuf_ready)
  {
    /* theora is one in, one out... */
    if (ogg_stream_packetout(&iOggStreamState, &iOggPacket)>0)
    {
      theora_decode_packetin(&iTheoraDecoderState, &iOggPacket);
      videobuf_granulepos=iTheoraDecoderState.granulepos;
      videobuf_time=theora_granule_time(&iTheoraDecoderState, videobuf_granulepos);
      videobuf_ready=1;
      iCurrFrame++;
    }

    // TODO: handle end of file
    if (!videobuf_ready && (iVideoFile->getPos() == iVideoFile->getSize()))
    {
      if (iIsLooped)
      {
        PrepareOgg();
      }
      else
      {
        Stop();
        return false;
      }
    }

    if (!videobuf_ready)
    {
      /* no data yet for somebody.  Grab another page */
      buffer_data();
      while(ogg_sync_pageout(&iOggSyncState, &iOggPage)>0){
        queue_page(&iOggPage);
      }
    }
  }

  return videobuf_ready != 0;
}


//------------------------------------------------------------------------------
//! Stop
//! 
video::ITexture* CTheoraPlayer::GetTexture()
{
  if (iTexture != NULL)
  {
    return iTexture;
  }

  return iBlankTexture;
}

//------------------------------------------------------------------------------
//! Stop
//! 
video::IImage* CTheoraPlayer::GetImage()
{
  return iImage;
}

//------------------------------------------------------------------------------
//! UpdateTexture
//! Copy buffer data to texture
void CTheoraPlayer::UpdateTexture()
{
  if (iImage == NULL)
    return;

  u32 texturePitch = iTexture->getPitch();
  const core::dimension2d<s32>& textureSize = iTexture->getSize();

  // try to lock texture
  u8* textureData = (u8*) iTexture->lock();
  if (textureData == NULL)
    return;
  
  iImage->copyToScaling(
    textureData, 
    textureSize.Width, 
    textureSize.Height, 
    iTexture->getColorFormat(),
    texturePitch);

  iTexture->unlock();
}


//------------------------------------------------------------------------------
//! UpdateBuffer
//! Write out the planar YUV frame, uncropped
void CTheoraPlayer::UpdateBuffer()
{
  if (iImage == NULL)
    return;

  //gTimer.Start();

  yuv_buffer yuv;
  theora_decode_YUVout(&iTheoraDecoderState, &yuv);

  const core::dimension2d<s32>& imageSize = iImage->getDimension();
  _IRR_DEBUG_BREAK_IF(imageSize.Height < yuv.y_height ||
    imageSize.Width < yuv.y_width);


  u8* bufferData = (u8*)iImage->lock();
  u8* yData = yuv.y;
  u8* uData = yuv.u;
  u8* vData = yuv.v;

  int y, x;

  u32 rowSize = iImage->getPitch();
  switch (iImage->getColorFormat())
  {
    case video::ECF_A8R8G8B8:
    {
      for (y = 0; y < yuv.y_height; y++)
      {
        int xsize = yuv.y_width;
        int uvy = (y/2)*yuv.uv_stride;
        int yy = y*yuv.y_stride;
        int by = y*rowSize;
        for (x = 0; x < xsize; x++)
        {
          int Y = yuv.y[yy + x] - 16;
          int U = yuv.u[uvy + (x/2)] - 128;
          int V = yuv.v[uvy + (x/2)] - 128;
          int R = ((298*Y         + 409*V + 128)>>8);
          int G = ((298*Y - 100*U - 208*V + 128)>>8);
          int B = ((298*Y + 516*U         + 128)>>8);
          if (R<0) R=0; if (R>255) R=255;
          if (G<0) G=0; if (G>255) G=255;
          if (B<0) B=0; if (B>255) B=255;

          bufferData[by + x*4 + 0] = B;
          bufferData[by + x*4 + 1] = G;
          bufferData[by + x*4 + 2] = R;
          bufferData[by + x*4 + 3] = 0xFF;
        };
      };
    } break;

    case video::ECF_R8G8B8:
    {
      for (y = 0; y < yuv.y_height; y++)
      {
        int xsize = yuv.y_width;
        int uvy = (y/2)*yuv.uv_stride;
        int yy = y*yuv.y_stride;
        int by = y*rowSize;
        for (x = 0; x < xsize; x++)
        {
          int Y = yuv.y[yy + x] - 16;
          int U = yuv.u[uvy + (x/2)] - 128;
          int V = yuv.v[uvy + (x/2)] - 128;
          int R = ((298*Y         + 409*V + 128)>>8);
          int G = ((298*Y - 100*U - 208*V + 128)>>8);
          int B = ((298*Y + 516*U         + 128)>>8);
          if (R<0) R=0; if (R>255) R=255;
          if (G<0) G=0; if (G>255) G=255;
          if (B<0) B=0; if (B>255) B=255;

          bufferData[by + x*3 + 0] = B;
          bufferData[by + x*3 + 1] = G;
          bufferData[by + x*3 + 2] = R;
        };
      }
    } break;
  }

  //gTimer.Stop();

  //float timeDiff = gTimer.GetMS();
  //printf("UpdateBuffer: %.2f\n", timeDiff);
}

//------------------------------------------------------------------------------
//! queue_page
//! helper: push a page into the steam for packetization
irr::s32 CTheoraPlayer::queue_page(ogg_page *page)
{
  if (iTheoraPacketsCount)
    ogg_stream_pagein(&iOggStreamState, &iOggPage);
  return 0;
}

//------------------------------------------------------------------------------
//! buffer_data
//! Helper; just grab some more compressed bitstream and sync it for
//! page extraction
irr::s32 CTheoraPlayer::buffer_data()
{
  char* buffer = ogg_sync_buffer(&iOggSyncState, 4096);
  irr::s32 bytes = iVideoFile->read(buffer, 4096);
  ogg_sync_wrote(&iOggSyncState, bytes);
  return (bytes);
}
bob
Posts: 57
Joined: Fri Jun 08, 2007 4:17 am
Location: Jacksonville, Fl (USA)
Contact:

Post by bob »

If someone is interested
Someone is interested! I've been thinking about implementing a custom VOIP using Theora. I'll try it out as soon as I get the chance.

Thanks! :D :D :D
genesisrage
Posts: 93
Joined: Tue Feb 08, 2005 12:19 pm

Post by genesisrage »

YES im very interested, i already use ogg-vorbis for all my sounds, now being able to use ogg-theora for video is awesome!!
pixeljunky
Posts: 19
Joined: Sun Feb 25, 2007 1:31 pm

Post by pixeljunky »

Hey,

just found this. Are you still improving this code ?
What happend to the VOIP code ?

Thanx
porcus
Posts: 149
Joined: Sun May 27, 2007 6:24 pm
Location: Germany

Post by porcus »

From where can I get the Theora static library ?
(If I try to compile it, I get an error.)
zet.dp.ua
Posts: 66
Joined: Sat Jul 07, 2007 8:10 am

Post by zet.dp.ua »

hi!

Theora's main site is www.theora.org. Theora and related libraries
can be gotten from www.theora.org or the main Xiph.Org site at
www.xiph.org. Development source is kept in an open subversion
repository, see http://theora.org/svn.html for instructions.
porcus
Posts: 149
Joined: Sun May 27, 2007 6:24 pm
Location: Germany

Post by porcus »

I didnt't found the library there, theres only the sourcecode and if I try
to compile it I get an error.
netpipe
Posts: 669
Joined: Fri Jun 06, 2008 12:50 pm
Location: Edmonton, Alberta, Canada
Contact:

Re: Theora video wrapper

Post by netpipe »

is the sound portion of this easy to integrate too ?
Live long and phosphor!
-- https://github.com/netpipe/Luna Game Engine Status 95%
Post Reply