macos platformview UI blinking #138936
Comments
|
Reproducible using the code sample provided above. Screen.Recording.2023-11-23.at.10.16.41.movflutter doctor -v |
|
EDIT: This is not the case (see the last comment). There is currently a full window surface for each of the platform view (because of the Flutter outline).
When obtaining backbuffer surface we don't check if is used by compositor, because generally it's not a problem if compositor still holds it at the beginning of frame for a few milliseconds. It does become a problem if compositor is using it for another 2-3 frames like in this example. I'm not entirely sure what the best solution here is. We may need to handle situation when jank is not caused by Flutter. |
|
Btw, have you considered using external textures for this? Using this many CAMetalLayers on platform thread is likely not going to result in good performance, it's just too heavy, but you should be able to render most of the workload to external texture and render those in flutter. |
|
@knopp it's just a stressing example to be sure it happens quickly. In my app, I use max 10 small 160x90 (x screen scale) at a time on screen. also I pause all rendering as soon as the window enter live resizing... but it still happens sometime.. if there is a way to share MTLTexture with flutter, and display them using the flutter renderer, would be faster for sure.. but I don't know how do it.. |
|
Look at FlutterTextureRegistry (it's same on macOS and iOS). The texture object that you register needs to provide a I'm suggesting this because even when the issue here gets fixed, you'll still be dropping frames with your current approach (dropping CA frames not caused by Flutter is the underlying reason for the flicker). |
|
@knopp thank you, I will look at it. For my app all the real rendering is done in background, so it's not too important if the UI drop some frame sometime.. but it still better if not and if I can spare some cpu cycles.. |
|
This is possibly relevant: #49757 |
|
Actually, I just noticed, that because every of these platform views has a flutter outline, there are 19 total surfaces on Flutter side. That is way, way too much, and perhaps even bigger reason why we're missing core animation deadline. Good news here is, I have a PR in progress just for this: flutter/engine#43301. I tested it, and it reduces the number of surfaces to 1 (2 when scrolling because of overlay). It seems to completely get rid of the issue. |
…utter#43301) Fixes flutter/flutter#129710 Fixes flutter/flutter#138936 Implements https://flutter.dev/go/optimized-platform-view-layers ## Example This scene would normally require 5 surfaces, but with the PR it comes down to 2 (when drawing over platform views) and 1 when all drawing is outside of platform views). https://github.com/flutter/flutter/assets/96958/091da832-bfbc-44a2-8da5-d55d84024c96 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
|
@knopp I think I got the fix on master branch, seems better.. but it's still happening sometime when using lots of camera inputs.. they must hang a lil the main thread when they are starting.. I think would be nice to have a frame counter or something like that to sync the compositor thread and the renderer thread in flutter.. in case something hang... I'll try to avoid the camera start on main thread still ;) |
|
I think a main thread display link would help here. It is on my TODO list. In any case, if you can provide another example that breaks, it would be quite helpful. |
|
Reopening because with opacity set on large number of platform views, there is still flicker because we're pushing more frames than the compositor can flip. This still needs to be addressed. |
|
There are two problems here:
|
|
@knopp I confirm, I got multiple computer freezes lately when minimizing and reverting back the app window. |
|
@knopp , little update, since one of the last sonoma update, apple has fixed the freezing.. it should be easier to debug this problem now.. |
|
I'm fairly confident that we're being too optimistic with single back buffer here. I really wanted it to work, but there's no way to know for sure that the compositor has picked up front surface. And when that happens (and compositor misses the frame), we end up drawing to the previous front surface while it is still in use. I think we're going to need similar logic to retrieve surface as FlutterMetalLayer is using. |
…use (#52082) Fixes flutter/flutter#138936 Currently the engine hold a single backbuffer per flutter view layer, which gets swapped with front buffer. This however is too optimistic, as the front buffer in some cases might not get immediately picked up by the compositor. When that happens, during next frame the cache may return a backbuffer that is in fact still being used by the compositor. Rendering to such surface will result in artifacts seen in the video. This seems more likely to happen with busy platform thread. It is also more likely to happen with the presence of "heavy" platform views such as `WKWebView`. IOSurface is able to report when it is being used by the compositor (`IOSurfaceIsInUse`). This PR ensures that the backing store cache never returns surface that is being used by compositor. This may result in transiently keeping more surfaces than before, as the compositor sometimes seems to holds on to the surface longer than it is displayed, but that's preferable outcome to visual glitches. This PR adds `age` field to `FlutterSurface`. When returning buffers to the surface cache (during compositor commit), the age of each surface currently present in cache increases by one, while the newly returned surfaces have their age set to 0. When returning surfaces from cache, a surface with lowest age that is not in use by compositor is returned. Surfaces with age that reaches 30 are evicted from the pool. Reaching this age means one of two things: - When surface is still in use at age 30 it means the compositor is holding on to it much longer than we expect. In this case just forget about the surface, we can't really do anything about it. - When surface is not in use at age 30, it means it hasn't been removed from cache for 29 subsequent frames. That means the cache is holding more surfaces than needed. Removing all surfaces from cache after idle period remains unchanged. Before: https://github.com/flutter/engine/assets/96958/ba2bfc43-525e-4a88-b37c-61842994d3bc After: https://github.com/flutter/engine/assets/96958/8b5d393e-3031-46b1-b3f0-cb7f63f8d960 *If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].* ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I signed the [CLA]. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview [Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene [test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests [Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo [C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style [testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/wiki/Chat
|
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of |
Is there an existing issue for this?
Steps to reproduce
Expected results
Actual results
I think it comes from ClipRRect() around the platform view, without it all seems working well
Code sample
Code sample
source code
Screenshots or Video
Screenshots / Video demonstration
Logs
Logs
too many logs, quarantine..
Flutter Doctor output
Doctor output
The text was updated successfully, but these errors were encountered: