10
Stack Exchange
votes

iOS OpenGL Catch-22: OpenGL background rules and "app snapshot" for App Switcher

Posted on 6/2/2016
UQ ID:395
Technology
Stack Overflow
Original

Like many developers, I have an app that uses OpenGL via a UIView subclass whose layerClass: method returns [CAEAGLLayer class].

Note I am not using GLKit or GLKView or GLKViewController

When I click Home to put the app into the background, after applicationDidEnterBackground, iOS calls my view's layoutSubviews twice, with portrait and landscape sizes, trying to generate an "app snapshot" as explained here (see "prepare for the app snapshot"):

https://developer.apple.com/library/ios/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/StrategiesforHandlingAppStateTransitions/StrategiesforHandlingAppStateTransitions.html#//apple_ref/doc/uid/TP40007072-CH8-SW10

How can this possibly work?

There seems to be a direct contradiction here with the very clear advice on this page (see "Background Apps May Not Execute Commands on the Graphics Hardware"):

https://developer.apple.com/library/ios/documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/ImplementingaMultitasking-awareOpenGLESApplication/ImplementingaMultitasking-awareOpenGLESApplication.html#//apple_ref/doc/uid/TP40008793-CH5-SW1

that we must not draw anything with OpenGL after applicationDidEnterBackground

If we don't draw, we cannot generate the snapshots. We must violate one rule or the other.

But we also want good snapshots in both orientations, so that when the user double-clicks home and goes to the App Switcher, they see reasonable snapshot images.

Even if I temporarily change my code to fully implement the layoutSubviews after applicationDidEnterBackground by creating an OpenGL surface and drawing (which, contrary to the Apple dox, does not crash), and then I double-click home and look at the snapshot in different orientations, only the snapshot for the orientation I was in before is correct. The other one is a super-ugly nasty re-scaling of the other snapshot. Apple seems to be going through the motions of taking snapshots, but not actually taking them.

I am seeing this behavior on iOS 9.3.2 on an iPad Mini. The behavior doesn't show on most/all iPhone devices since they don't support a landscape App Switcher.

UPDATE: the problem also happens, and happens much worse, when using the new iOS 9 "Slide Over" multitasking feature and switching the same app between being a normal fullscreen app vs. being an app slid over another app. iOS only seems to capture a snapshot of the last app size, so after using the app at 640px wide and then trying to use the App Switcher to get to the app fullscreen, we see a grotesque pixely out-of-proportion snapshot in the App Switcher and also during the first second of launch. There has got to be some way to fix this!

UPDATE 2: I have seen a few iOS apps, which I know to be OpenGL-only apps, where if you use them in portrait, then go back Home and rotate to landscape, then double-click Home, you see the portrait launch image rather than a horrible, distorted, out-of-proportion image like I am seeing. While I would prefer to render snapshot images, I would even be happy to see the launch image. But the option everyone mentions, ignoreSnapshotOnNextApplicationLaunch, does not work because it only affects what you see at actual app launch time, not what is seen in the App Switcher when you double-click Home, and for many on StackOverflow it actually didn't even work at all (not even at launch time).

How do we get around this Catch-22?

This StackOverflow thread (unlike me, the OP here uses GLKit but the symptom is the same):

iOS OpenGL ES screen rotation while background apps bar visible

confirms that some OpenGL apps on iOS are able to have proper preview images in the Home double-press app switcher for both orientations. How do they do it?

How can I get proper snapshots shown in both orientations in the App Switcher?

Here is a log of AppDelegate (appdel), ViewController (eaglc) and View (eaglv) calls that come from iOS at the time that I click the Home button once to exit the app. You can see the attempts at snapshotting that come well after didEnterBackground:

+  189.57ms      appdel appWillResignActive
+    0.74ms        appdel appWillResignActive between_view_os_callbacks 0
+    4.11ms        appdel appWillResignActive between_view_os_callbacks 0 done
+    0.82ms        appdel appWillResignActive activation_changed
+    1.50ms        appdel appWillResignActive activation_changed done
+    0.47ms        appdel appWillResignActive between_view_os_callbacks 1
+    2.68ms          drawing rect [(144,1418)+(2,66)] (0 left)
+   44.28ms          swap_buffers glFlush()
+    6.16ms          swap_buffers presentRenderBuffer
+    9.01ms        appdel appWillResignActive between_view_os_callbacks 1 done
+    0.61ms        appdel save_state
                     ..app saving data, no OpenGL here..
+    0.49ms          appdel save_state calling glFinish
+    0.34ms        appdel save_state done
+    0.25ms      appdel appWillResignActive done
+  492.72ms      appdel applicationDidEnterBackground
+    0.56ms        appdel save_state
                     ..app saving data, no OpenGL here..
+    0.65ms          appdel save_state calling glFinish
+    0.54ms        appdel save_state done
+    0.65ms        eaglv let_go_of_frame_buffer_render_buffer
                     app drops OpenGL frame_buffer and render_buffer here
+    1.10ms      appdel applicationDidEnterBackground done

Now we are not supposed to do OpenGL, BUT...

+    6.30ms      eaglc supportedInterfaceOrientations
+    5.74ms      about_to_sleep between_view_os_callbacks
+    1.30ms        SKIPPING between_view_os_callbacks cuz app in background
+    0.66ms      about_to_sleep between_view_os_callbacks done
+  135.85ms      eaglc willRotateToInterfaceOrientation
+    2.49ms      appdel willChangeStatusBarFrame new=0,0 768x20
+    3.21ms      appdel didChangeStatusBarFrame old=0,0 1024x20

we get a portrait layoutSubviews....

+    1.26ms      eaglv layoutSubviews (initted=1, have_fbrb=0)
+    1.80ms        eaglv assure_frame_buffer_render_buffer
+    0.95ms          eaglv assure_fbrb scale ios=2 eaglv=2
+    0.90ms          eaglv assure_fbrb (frame=1536,2048)
+    1.04ms          eaglv assure_fbrb (layer frame=1536,2048)
+    0.92ms        eaglv assure_fbrb in bg: will make fbrb later
+    0.96ms      eaglv layoutSubviews done
+    3.11ms      eaglc didRotateFromInterfaceOrientation
+  149.07ms      eaglc willRotateToInterfaceOrientation
+    1.99ms      appdel willChangeStatusBarFrame new=0,0 1024x20
+    2.35ms      appdel didChangeStatusBarFrame old=0,0 768x20

then a landscape layoutSubviews...

+    1.91ms      eaglv layoutSubviews (initted=1, have_fbrb=0)
+    1.09ms        eaglv assure_frame_buffer_render_buffer
+    0.91ms          eaglv assure_fbrb scale ios=2 eaglv=2
+    1.65ms          eaglv assure_fbrb (frame=2048,1536)
+    0.92ms          eaglv assure_fbrb (layer frame=2048,1536)
+    0.93ms        eaglv assure_fbrb in bg: will make fbrb later
+    0.83ms      eaglv layoutSubviews done
+    2.79ms      eaglc didRotateFromInterfaceOrientation

and, adding insult to injury, we get this log message:

Snapshotting a view that has not been rendered results in an empty snapshot. Ensure your view has been rendered at least once before snapshotting or snapshot after screen updates.
Keywords & Tags
ios
objective-c
ipad
opengl-es
layoutsubviews
Model Answers (9)
Submitted answers from various AI models
Unsolved
No reviews yet

K2-Think

Pending UQ Validation
Needs Human Verification
MBZUAI-IFM
Qwen2.5-32B
Sep 18, 2025, 01:50 PM

To resolve the iOS OpenGL snapshot issue for the App Switcher while complying with background execution rules, follow these steps:

1. Capture the Current Framebuffer Content Before Entering Background

When the app is about to resign active (triggered by applicationWillResignActive), capture the current OpenGL framebuffer content as a UIImage. This ensures you have the latest visual st...

0 model reviews
0 human reviews
No reviews yet

GPT-5

Failed UQ Validation
Needs Human Verification
OpenAI
Sep 2, 2025, 06:56 AM

There’s no real contradiction. The “don’t use the GPU in the background” rule is correct. The system still needs to produce snapshots for the App Switcher, and it will ask your view hierarchy to relayout after your app has moved to the background. However, those snapshots are produced by Core Animation/BackBoard rendering the window’s layer tree, not by executing your OpenGL commands. If your topm...

0 model reviews
0 human reviews
No reviews yet

Claude Opus 4

Failed UQ Validation
Needs Human Verification
Anthropic
Aug 1, 2025, 12:00 AM

This is indeed a challenging situation where Apple's documentation seems to contradict itself. You're experiencing the classic OpenGL snapshot problem on iOS. Let me break down the issue and provide some solutions:

The Problem

  1. Background Rule: You cannot execute OpenGL commands after applicationDidEnterBackground
  2. Snapshot Rule: iOS wants to capture snapshots in multiple orient...
0 model reviews
0 human reviews
No reviews yet

o3 Pro

UQ Validated
Needs Human Verification
OpenAI
Aug 1, 2025, 12:00 AM

Why the OS calls your view again after the app is already in the background

When the user presses the Home button iOS

  1. sends your application the usual life-cycle messages
  2. immediately takes an image of all the windows that are on-screen (this is the picture that is shown if the user re-enters the application without having killed it)
  3. still while your process is alive, asks ...
0 model reviews
0 human reviews
No reviews yet

o3

UQ Validated
Needs Human Verification
OpenAI
Aug 1, 2025, 12:00 AM

The two things that are happening when you press the Home-button are

  1. As soon as applicationDidEnterBackground: returns your process is not allowed to touch the GPU any more.
  2. A fraction of a second after that, SpringBoard asks Core Animation for a bitmap (“app snapshot”) that it can show in the App- Switcher and while the app is relaunching.

Those two facts are not contradictory...

0 model reviews
0 human reviews
No reviews yet

DeepSeek R1

Failed UQ Validation
Needs Human Verification
DeepSeek
deepseek-ai/DeepSeek-R1
Aug 1, 2025, 12:00 AM
<think> We are given a detailed question about an iOS OpenGL ES app that uses a custom UIView subclass (with CAEAGLLayer) and is not using GLKit. The problem is that when the app enters the background, iOS tries to take snapshots for the App Switcher in both portrait and landscape orientations by calling `layoutSubviews` twice. However, Apple's guidelines state that we must not use OpenGL after `a...
0 model reviews
0 human reviews
No reviews yet

Gemini 2.5 Pro

Failed UQ Validation
Needs Human Verification
Google
Aug 1, 2025, 12:00 AM

Excellent and very detailed question. You've hit upon one of the most frustrating and poorly documented aspects of developing a non-GLKit OpenGL application on iOS.

You are absolutely correct. There is a fundamental contradiction between these two requirements, and the behavior you are seeing is the direct result of it.

  1. The Rule: You must release your OpenGL resources and cease all ren...
0 model reviews
0 human reviews
No reviews yet

o4 mini

Failed UQ Validation
Needs Human Verification
OpenAI
Aug 1, 2025, 12:00 AM

iOS is really trying to do the “right” thing – it will call your EAGL‐backed view’s ‑layoutSubviews one last time (in each orientation) after you go into the background so that it can take a bitmap snapshot for the App-Switcher. Unfortunately Apple also says that as soon as you are backgrounded you may not issue any more OpenGL ES commands, so you can’t rebuild your frame‐buffer/render‐buffer in ...

0 model reviews
0 human reviews
No reviews yet

Claude 3.7 Sonnet

Failed UQ Validation
Needs Human Verification
Anthropic
Aug 1, 2025, 12:00 AM

iOS OpenGL App Switcher Snapshot Dilemma

This is a complex issue involving OpenGL rendering, iOS lifecycle events, and app snapshots for the App Switcher. Let me explain the problem and potential solutions.

The Core Problem

You're facing a contradiction in Apple's documentation:

  1. iOS needs to take snapshots when an app goes to background (including different orientations)
  2. OpenGL comma...
0 model reviews
0 human reviews
Submit Your Model's Answer
Your submission will be validated by our LLM validator and then by human experts

You need to be logged in to submit an answer