Hi,
I have been using your terrific control with excellent results. However my application now has a new requirement to display live images from a camera (4K x 3K x 16-bits) at > 15FPS.
My current display algorithm scales the input image from 16-bits to 256 bits (stored in Emgu/OpenCV Mat classes), then creates an RGB bitmap and sets it to the "Image" property of the control. The best performance I can achieve is < 5 FPS (see simplified code below).
Have you used your control with OpenGL in C# to display images, and if so, do you have any example code? Or do I need to start from scratch with something like OpenTK to create my own control and handle zooming, etc. on my own? I did use OpenGL in C++ about 10 years ago, so I could bite the bullet and start down this path. Or is there a better method?
void GenerateBitmap()
{ ...
unsafe
{
...
//Both m_RgbData and tempImage8BitBuffer are byte arrays
fixed (byte* pRGB = &m_RgbData[0])
{
fixed (byte* pImageData = &tempImage8BitBuffer[0])
{
byte value;
byte* pDst = pRGB;
byte* pSource = pImageData;
for (int y = 0; y < rows; y++)
{
for (int x = 0; x < cols; x++)
{
value = *pSource++;
*pDst++ = value; // R
*pDst++ = value; // G
*pDst++ = value; // B
}//for (int x = 0; x < cols; x++)
}//for (int y = 0; y < rows; y++)
}//fixed (byte* pImageData = &tempImage8BitBuffer[0])
}//fixed (byte* pRGB = &m_RgbData[0])
}//unsafe
...
m_Bitmap = new Bitmap((int)nCurrentWidth, (int)nCurrentHeight, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
BitmapData bmd = m_Bitmap.LockBits(new Rectangle(0, 0, (int)nCurrentWidth, (int)nCurrentHeight),
System.Drawing.Imaging.ImageLockMode.ReadOnly, m_Bitmap.PixelFormat);
...
unsafe
{
byte* pDst = (byte*)bmd.Scan0;
for (int i = 0; i < nCurrentHeight; i++)
{
Marshal.Copy(m_RgbData, (int)(i * nCurrentWidth * 3), (IntPtr)pDst, (int)(nCurrentWidth * 3));
pDst += bmd.Stride;
}
}
m_Bitmap.UnlockBits(bmd);
imageBox.Image = m_Bitmap;
}
Thanks!
Hello,
Welcome to the forums, glad you found the control useful. And thanks for the interesting question!
I've never considered desktop applications in terms of FPS before; it's not something that has ever been a factor for the typical application I work with. With that said, I would have thought repainting the control 15 times a second should be more than doable, even at such huge resolutions.
Have you tried profiling your application to find out where the bottlenecks are? As far as I know, the code you're doing with unsafe pointers is the fastest way to manipulate bitmaps in C#, but I would still wonder if it's fast enough. On the other hand, I discovered that ImageBox has a performance hit if you continuously repaint it with a unchanged zoomed image, so there's potential for improvement there too. It could even be something else such as garbage collection. I'm not sure what to suggest without more information, I definitely think it would be best to start with profiling and identify what the slowest parts of the program are.
In regards to OpenGL, as a Windows Forms control, the ImageBox won't work with it. You could in theory take a lot of code and dump into a OpenTK application (assuming they've finished cleaning house yet) and use that, but I don't know if it would be better to start from scratch rather than trying to hammer OpenGL into ImageBox logic (for example the different co-ordinate systems used by WinForms and OpenGL). I haven't used OpenTK for rendering huge graphics, but its a safe bet that it's going to perform much better than a WinForms program.
Not sure if this is viable, but WPF is hardware accelerated using DirectX, perhaps this is another option? I haven't used WPF in anger though so I don't have much to offer on how to do image processing using it.
A bit of a wishywashy post I'm afraid - but if you do some profiling and find that there's aspects of ImageBox causing you an issue if you can provide more information I could look into improving it.
Hope something in this helps!
Regards;
Richard Moss
Hi,
Thanks for the quick response. I took your suggestion to profile the code (again) and found that I could eliminate the code listed above with just a few lines of Emgu/OpenCV code. I used one of their "Convert" functions to go from a one channel to a four channel Mat object(i.e. RGBG), then set the ImageBox.Image property equal to the Mat.Bitmap property. I can now get ~10 FPS, which will have to do in the short term. It does help to read the docs...
I will play with OpenTK or equivalent when I get a chance, to see what can be done in the future.
Thanks,
DR