So I decided it was time to add support for full screen mode in my game. Previously when entering full screen, the buffer would just get scaled up, but now I want to do it properly.
I'm using my own software renderer to do all my game's rendering, and then OpenGL does the final display of the texture that comes back from the game.
In order to keep things as simple as possible, I decided to allocate the memory for the texture that is passed to the game big enough for the whole screen. That way I don't need to reallocate it when the window changes size, and I can just pass in the new width & height to the game. As far as dealing with the OpenGL side of things, I first tried just changing the current width & height values, but that resulted in a GL_INVALID_VALUE (1281) error when calling glTexSubImage2D, and really, that's not too surprising since the texture was initialized with a smaller size in the setup code with the call to glTexImage2D.
Here's the setup code for OpenGL:
At the end of the frame, the buffer is drawn to the screen like this (at first, when the window changed size I would just update the values of _currentSize.width and _currentSize.height):
When I exited full screen mode, everything was fine again, so it's clearly an issue with calling glTexSubImage2D with the new & bigger size. But then I tried resizing the texture by calling glTexImage2D with the new size when the window changed size, but that still resulted in the same error. Next, I tried deleting the texture and re-generating it when the window changes size:
Instead of getting a GL_INVALID_VALUE error, I get a GL_INVALID_OPERATION error (1282). It's worth noting that the render method and the reshape method are called on a different thread, and I did try adding synchronization between the two methods but that didn't help.
The one way I did get it to work was to call glTexImage2D instead of glTexSubImage2D in the render call. As far as I understand though, glTexImage2D is for creating a texture and glTexSubImage2D is for copying pixels into the texture, so it's much more efficient to call glTexSubImage2D instead of glTexImage2D.
Any ideas what I'm doing wrong here or how I need to approach resizing the OpenGL texture when the window changes size?
I'm using my own software renderer to do all my game's rendering, and then OpenGL does the final display of the texture that comes back from the game.
In order to keep things as simple as possible, I decided to allocate the memory for the texture that is passed to the game big enough for the whole screen. That way I don't need to reallocate it when the window changes size, and I can just pass in the new width & height to the game. As far as dealing with the OpenGL side of things, I first tried just changing the current width & height values, but that resulted in a GL_INVALID_VALUE (1281) error when calling glTexSubImage2D, and really, that's not too surprising since the texture was initialized with a smaller size in the setup code with the call to glTexImage2D.
Here's the setup code for OpenGL:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | CGSize screenSize = ...; _textureSize.width = screenSize.width; _textureSize.height = screenSize.height; _currentSize.width = (GLsizei)self.bounds.size.width; // This is the actual width of the window. _currentSize.height = (GLsizei)self.bounds.size.height; // This is the actual height of the window. // Adds apron around the texture. GLsizei allocWidth = _textureSize.width + 2*CLIP_REGION_PIXELS; GLsizei allocHeight = _textureSize.height + 2*CLIP_REGION_PIXELS; _textureMemory = (uint32_t *)osx_allocate_memory(allocWidth * allocHeight * sizeof(uint32_t)); memset(_textureMemory, 0, allocWidth * allocHeight * sizeof(uint32_t)); _texture = _textureMemory + (intptr_t)(CLIP_REGION_PIXELS * allocWidth + CLIP_REGION_PIXELS); glPixelStorei(GL_UNPACK_ALIGNMENT, 1); glGenTextures(1, &_textureId); glBindTexture(GL_TEXTURE_2D, _textureId); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _currentSize.width, _currentSize.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); glBindTexture(GL_TEXTURE_2D, 0); |
At the end of the frame, the buffer is drawn to the screen like this (at first, when the window changed size I would just update the values of _currentSize.width and _currentSize.height):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | - (void)render {
glClear(GL_COLOR_BUFFER_BIT);
glBindTexture(GL_TEXTURE_2D, _textureId);
glEnable(GL_TEXTURE_2D);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _currentSize.width, _currentSize.height, GL_RGBA, GL_UNSIGNED_BYTE, _texture);
// ** here I would get the GL_INVALID_VALUE error **//
glBegin(GL_QUADS);
glTexCoord2f(0.f, 1.f); glVertex2f(-1.f, -1.f);
glTexCoord2f(0.f, 0.f); glVertex2f(-1.f, 1.f);
glTexCoord2f(1.f, 0.f); glVertex2f(1.f, 1.f);
glTexCoord2f(1.f, 1.f); glVertex2f(1.f, -1.f);
glEnd();
}
|
When I exited full screen mode, everything was fine again, so it's clearly an issue with calling glTexSubImage2D with the new & bigger size. But then I tried resizing the texture by calling glTexImage2D with the new size when the window changed size, but that still resulted in the same error. Next, I tried deleting the texture and re-generating it when the window changes size:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | - (void)reshape {
GLsizei newWidth = self.bounds.size.width;
GLsizei newHeight = self.bounds.size.height;
if (newWidth != _currentSize.with || newHeight != _currentSize.height) {
glDeleteTextures(1, &_textureId);
_textureId = 0;
glGenTextures(1, &_textureId);
glBindTexture(GL_TEXTURE_2D, _textureId);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, newWidth, newHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glBindTexture(GL_TEXTURE_2D, 0);
_currentSize.width = newWidth;
_currentSize.height = newHeight;
}
}
|
Instead of getting a GL_INVALID_VALUE error, I get a GL_INVALID_OPERATION error (1282). It's worth noting that the render method and the reshape method are called on a different thread, and I did try adding synchronization between the two methods but that didn't help.
The one way I did get it to work was to call glTexImage2D instead of glTexSubImage2D in the render call. As far as I understand though, glTexImage2D is for creating a texture and glTexSubImage2D is for copying pixels into the texture, so it's much more efficient to call glTexSubImage2D instead of glTexImage2D.
Any ideas what I'm doing wrong here or how I need to approach resizing the OpenGL texture when the window changes size?
Edited by Flyingsand
on
Reason: Initial post
You "change" texture size by simply allocating new texture with glTexImage2D. No need to delete texture object handle and allocate it again, just call glTexImage2D - that's it.
If you get any errors then that is because of other things.
For example, are you sure _currentSize.width and _currentSize.height values you are passing for glTexSubImage2D are equal or smaller than width/height values you are using for glTexImage2D?
"render method and the reshape method are called on a different thread, and I did try adding synchronization between the two methods but that didn't help" - if this is true, are you aware that GL context is typically bound to one thread? Are you creating multiple GL contexts and sharing between threads? Because that is not a good idea, multi-thread context's in GL always have been very unstable.
Btw, in your case it probably would be more efficient to use glTexImage2D in render() function instead of glTexSubImage2D. Because glTexSubImage2D will tell GL driver that you need to only update subregion of texture, which means it needs to "keep" old texture alive. But if GPU is till using old texture to draw previous frame, there will be a stall. If you would use glTexImage2D, GL driver will know that old data is irrelevant and it can put texture in new memory space in case old data is still being used.
Read "The problem" and "Buffer re-specification" sections here: https://www.khronos.org/opengl/wi...Streaming#Buffer_re-specification It talks about buffers, but the same concept applies to textures. And technically you can also use PBO buffers to upload texture data. In old days that was the fastest way to stream texture data which changes per every frame (nowadays there is a persistent mapping).
If you get any errors then that is because of other things.
For example, are you sure _currentSize.width and _currentSize.height values you are passing for glTexSubImage2D are equal or smaller than width/height values you are using for glTexImage2D?
"render method and the reshape method are called on a different thread, and I did try adding synchronization between the two methods but that didn't help" - if this is true, are you aware that GL context is typically bound to one thread? Are you creating multiple GL contexts and sharing between threads? Because that is not a good idea, multi-thread context's in GL always have been very unstable.
Btw, in your case it probably would be more efficient to use glTexImage2D in render() function instead of glTexSubImage2D. Because glTexSubImage2D will tell GL driver that you need to only update subregion of texture, which means it needs to "keep" old texture alive. But if GPU is till using old texture to draw previous frame, there will be a stall. If you would use glTexImage2D, GL driver will know that old data is irrelevant and it can put texture in new memory space in case old data is still being used.
Read "The problem" and "Buffer re-specification" sections here: https://www.khronos.org/opengl/wi...Streaming#Buffer_re-specification It talks about buffers, but the same concept applies to textures. And technically you can also use PBO buffers to upload texture data. In old days that was the fastest way to stream texture data which changes per every frame (nowadays there is a persistent mapping).
Edited by Mārtiņš Možeiko
on
Also, as far as I know, using a texture to update the Framebuffer is not how it's meant to be. Textures are meant for texture mapping, and there are for example some size constraints. Some drivers cannot handles texture sizes other than power-of-2 sizes. There are also maximum texture sizes dependent on the graphics drivers, so you might as well have hit the maximum size.
Maybe a better API to use is glWritePixels() / glDrawPixels()? Take all that with a grain of salt, since I've never used those and I'm not an OpenGL expert.
You could also use an OS-native API to upload the pixel data, instead of using OpenGL. I've made Win32 and X11 backends for a software renderer, it was pretty simple.
Maybe a better API to use is glWritePixels() / glDrawPixels()? Take all that with a grain of salt, since I've never used those and I'm not an OpenGL expert.
You could also use an OS-native API to upload the pixel data, instead of using OpenGL. I've made Win32 and X11 backends for a software renderer, it was pretty simple.
Edited by jstimpfle
on
jstimpfle
Also, as far as I know, using a texture to update the Framebuffer is not how it's meant to be. Textures are meant for texture mapping
Generally speaking - yes, true. But there is nothing wrong with uploading texture and drawing large triangle/quad. It's all good and "legal". In fact that is the most performant way to upload/draw full-window framebuffer per frame on modern OS'es.
Some drivers cannot handles texture sizes other than power-of-2 sizes.
This was true 10 or 15 years ago. Nowadays I don't think you'll find GPU/driver that does not support NPOT for regular RGBA8 texture.
There are also maximum texture sizes dependent on the graphics drivers, so you might as well have hit the maximum size.
True, but then glTexImage2D would fail, not glTexSubImage2D.
Maybe a better API to use is glWritePixels() / glDrawPixels()?
This will be significantly slower than using texture mapping. glRead/DrawPixels cause stall in pipeline.
Edited by Mārtiņš Možeiko
on
You "change" texture size by simply allocating new texture with glTexImage2D. No need to delete texture object handle and allocate it again, just call glTexImage2D - that's it.
Ok, that's what I thought. What was confusing me is that I verified that _currentSize.width and _currentSize.height are correct, but it looks like it was a threading issue. I don't think OpenGL liked that I tried to reallocate the texture with glTexImage2D on a different thread, even with synchronization in place, so instead I tried this just now in the render method and it works:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | GLint textureWidth, textureHeight;
glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_WIDTH, &textureWidth);
glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT, &textureHeight);
if (textureWidth != _currentSize.width || textureHeight != _currentSize.height) {
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _currentSize.width, _currentSize.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr);
}
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _currentSize.width, _currentSize.height, GL_RGBA, GL_UNSIGNED_BYTE, _texture);
glBegin(GL_QUADS);
glTexCoord2f(0.f, 1.f); glVertex2f(-1.f, -1.f);
glTexCoord2f(0.f, 0.f); glVertex2f(-1.f, 1.f);
glTexCoord2f(1.f, 0.f); glVertex2f(1.f, 1.f);
glTexCoord2f(1.f, 1.f); glVertex2f(1.f, -1.f);
glEnd();
|
Although since you say that glTexImage2D might actually be more efficient in my case instead of using glTexSubImage2D, I might just go with that.
And yes, I do know that multithreading with OpenGL is a bit of a minefield. I'm not explicitly creating multiple GL contexts and sharing them, I'm just using Apple's NSOpenGLView, and I initialize things inside the prepareOpenGL method as suggested. In order to sync with the monitor's refresh rate, I'm using CVDisplayLink, in which I register a callback function that runs every time a new frame is needed, and this is where I call my update and render function (so it's effectively the main thread of the game). After returning from the update and render function of the game, that is when I call render, so it gets called on this CVDisplayLink thread. As for when the window changes size, I get notified of that in the view's reshape method, which I verified gets called on the main thread (predictably so because it's UI-related).
You could also use an OS-native API to upload the pixel data, instead of using OpenGL. I've made Win32 and X11 backends for a software renderer, it was pretty simple.
True. I think I tried this at the very beginning, and it worked fine. I just ended up switching to OpenGL for some reason. And yes I know I'm using a very antiquated way of calling OpenGL, but I'm going to have to switch to using Metal on Apple and probably DirectX on Windows at some point anyway, so.. Although I'm sure OpenGL will keep working for some time, but with Apple you never know -- they like to force people to do things their way, and then want you to use Metal. :/
Edited by Flyingsand
on
If you were not explicitly sharing GL context's, then other GL context does not know anything about textures created in first GL context. GL context is kind of a namspace for all objects - state, textures, buffer, shaders. If you don't share them GLuint texture handles are not valid to use in different GL context.
If you were not explicitly sharing GL context's, then other GL context does not know anything about textures created in first GL context. GL context is kind of a namspace for all objects - state, textures, buffer, shaders. If you don't share them GLuint texture handles are not valid to use in different GL context.
Ok, yeah I think I see where I was going wrong. The thing that confused me is that the prepareOpenGL method that I override from NSOpenGLView (where I initially create the texture and other OpenGL stuff) is called on the main thread, but my render method is called in the CVDisplayLink thread. However, in the render method I have this:
1 2 3 4 5 6 7 | [[self openGLContext] makeCurrentContext]; CGLLockContext([[self openGLContext] CGLContextObj]); // rendering.. CGLFlushDrawable([[self openGLContext] CGLContextObj]); CGLUnlockContext([[self openGLContext] CGLContextObj]); |
So that clearly handles the sharing of the OpenGL context on across threads, and that's probably the synchronization I would have needed when I was trying to update the texture size with glTexImage2D in the reshape method. Oh well, the way I have it now is working, and doesn't require extra locking, so it's actually the better solution.
Oh, I see - you were using only one context. There is not sharing. You were binding one context into different threads when you want to use it. That should have worked. Doesn't CGLLockContext should happen before makeCurrentContext? Otherwise other thread can call makeCurrentContext outside of lock, no?
But I would not recommend it anyway. Not sure how it is on mac, but on Windows and Linux operation "makeCurrentContext" is reaaaally slow, as it fully flushes pipeline. You really want to avoid calling in your main rendering loop.
But I would not recommend it anyway. Not sure how it is on mac, but on Windows and Linux operation "makeCurrentContext" is reaaaally slow, as it fully flushes pipeline. You really want to avoid calling in your main rendering loop.
Edited by Mārtiņš Možeiko
on
Oh, I see - you were using only one context. There is not sharing. You were binding one context into different threads when you want to use it. That should have worked. Doesn't CGLLockContext should happen before makeCurrentContext? Otherwise other thread can call makeCurrentContext outside of lock, no?
Yes. Sorry if I wasn't clear on that. And yes, good catch on the lock happening after makeCurrentContext. It's been awhile since I looked at this code, and in fact, I tried searching for where I got it from. I remember seeing something similar to it that I ended up using here awhile back, so I was curious to see if I could track it down again to see why they were calling makeCurrentContext each time through the render loop.
Now that I'm going over this code again, however, I don't need the locks because I don't call OpenGL from anywhere else except in this render method (well, apart from initializing it in prepareOpenGL of course). I also experimented with calling makeCurrentContext only once to see if I get better performance. So my render method (in full) now looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | static bool GLContextActive = false; - (void)render { if (!GLContextActive) { [[self openGLContext] makeCurrentContext]; GLContextActive = true; } glClear(GL_COLOR_BUFFER_BIT); glBindTexture(GL_TEXTURE_2D, _textureId); GLint textureWidth, textureHeight; glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_WIDTH, &textureWidth); glGetTexLevelParameteriv(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT, &textureHeight); if (textureWidth != _currentSize.width || textureHeight != _currentSize.height) { glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _currentSize.width, _currentSize.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, nullptr); } glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, _currentSize.width, _currentSize.height, GL_RGBA, GL_UNSIGNED_BYTE, _texture); glBegin(GL_QUADS); glTexCoord2f(0.f, 1.f); glVertex2f(-1.f, -1.f); glTexCoord2f(0.f, 0.f); glVertex2f(-1.f, 1.f); glTexCoord2f(1.f, 0.f); glVertex2f(1.f, 1.f); glTexCoord2f(1.f, 1.f); glVertex2f(1.f, -1.f); glEnd(); CGLFlushDrawable([[self openGLContext] CGLContextObj]); } |
It turns out it makes a huge difference! Wow. At full screen resolution (2560x1440), this method took ~5ms! By only calling makeContextCurrent once, it's now down to ~2.5ms! What an eye-opener. :) Though even 2.5ms seems a bit long, so switching out of immediate mode would probably be a good idea. Or just using Metal. I'm using very minimal amount of OpenGL so it's not a huge amount of work, and I've already done some Metal programming.
Thanks so much for pointing that out!
Very nice! Next steps:
1) remove glClear(GL_COLOR_BUFFER_BIT), because you are drawing fullscreen quad anyway. No point spending time filling it with color if it will overwritten immediately afterwards
2) avoid using glGetTexLevelParameteriv. glGet* functions sometimes are slow (depends on GL library being used). Cache previous texture size yourself.
3) use only glTexImage2D (no glTexSubImage2D). As I explained above - TexSubImage will add extra synchronizatin in GL pipeline to make sure that drawing commands from previous frame have finished using texture before updating it for new frame. Which is something you don't need - as you are uploading full texture. TexImage will be a "hint" to GL pipeline that new texture is incoming without any dependencies on previous usage.
1) remove glClear(GL_COLOR_BUFFER_BIT), because you are drawing fullscreen quad anyway. No point spending time filling it with color if it will overwritten immediately afterwards
2) avoid using glGetTexLevelParameteriv. glGet* functions sometimes are slow (depends on GL library being used). Cache previous texture size yourself.
3) use only glTexImage2D (no glTexSubImage2D). As I explained above - TexSubImage will add extra synchronizatin in GL pipeline to make sure that drawing commands from previous frame have finished using texture before updating it for new frame. Which is something you don't need - as you are uploading full texture. TexImage will be a "hint" to GL pipeline that new texture is incoming without any dependencies on previous usage.
Edited by Mārtiņš Možeiko
on
I ended up doing 2 of those 3 things (1 & 3) before I read your message, and now I've just finished #2. I'm not seeing much difference this time, but oh well.
I do have a Windows version, so I'll be interested to see what the results are there. However, it's way behind my Mac version, so I have a bit of work ahead of me to get it up to speed.
This game is starting to feel like a proper prototype. :)
Thanks again!
I do have a Windows version, so I'll be interested to see what the results are there. However, it's way behind my Mac version, so I have a bit of work ahead of me to get it up to speed.
This game is starting to feel like a proper prototype. :)
Thanks again!