Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interactable ScrollView content when settling a scroll activity #145848

Open
wants to merge 31 commits into
base: master
Choose a base branch
from

Conversation

Michal-MK
Copy link
Contributor

@Michal-MK Michal-MK commented Mar 27, 2024

Currently when the user:

  • flings a scrollable
  • overscrolls, and the scrollable is trying to settle
  • applies DrivenScrollActivity
  • swipes from one tab to the next

All inputs are discarded or forwarded directly to the Scrollable widget (scroll activity).

This leads to situations like these:

2024-03-27.19-45-59.mp4
2024-03-27.19-46-19.mp4
2024-03-27.19-46-46.mp4

Which leads to poor experience on iOS. The native behavior of iOS is to allow touches while a scrollable is settling:

native.mp4

This PR alters the shouldIgnoreTouches of BallisticScrollAvtivity and DrivenScrollActivity to not make the child of the scrollable ignore touches.

Fixes #145330

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
  • I signed the [CLA].
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is [test-exempt].
  • I followed the [breaking change policy] and added [Data Driven Fixes] where supported.
  • All existing and new tests are passing.

Currently tests that test tap to stop are not working as the taps now register when they should not. Because there is no distinction between flings inside and flings that go out of range.

@github-actions github-actions bot added a: text input Entering text in a text field or keyboard related problems framework flutter/packages/flutter repository. See also f: labels. f: scrolling Viewports, list views, slivers, etc. labels Mar 27, 2024
@Piinks Piinks self-requested a review March 29, 2024 19:07
Copy link
Contributor

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Michal-MK it looks like this is still a draft, so I'll wait for your say so when it's ready for review. :)

@Michal-MK
Copy link
Contributor Author

Michal-MK commented Mar 29, 2024

@Piinks Yeah, I am sort of stuck as I reverted all the meaningful changes and the tests are still failing (other than analyze, that one I expect to fail) looking at the output widget_tester_test is something I did not touch so I assume something else is going on? Or the:
E.g. PathNotFoundException: Cannot open file, path = 'C:\b\s\w\ir\x\w\recipe_cleanup\flutter_logs_dir/fake_result.json' (OS Error: The system cannot find the file specified.

I'll continue after Easter :)

EDIT: Ok seeing as other PRs have functioning tests I'll comment out everything and try that.

@Michal-MK
Copy link
Contributor Author

The three failing checks are in the FlutterGallery in the Style->Colors section. The issue there is the use of Scrollbar, widget. There are tabs, and as the test swipes through them it, at some point, it tries to render a second Scrollbar which then fails, as both are reading the primary scroll controller. I validated this on master using trackpad scrolling which is what this PR effectively brings to finger gestures and it fails on master as well. The test just does not pick it up as it is not using a trackpad to scroll (.fling/.trackpadFling and other alternatives). I do not see how to fix this without the TabBarView somehow providing a custom ScrollController to its children that is somehow connected to the primary one once the transition animation finishes. @Piinks

@Piinks
Copy link
Contributor

Piinks commented Apr 3, 2024

Interesting, I tried using a trackpad to reproduce this at https://flutter-gallery-archive.web.app/#/demo/colors but did not see an error from the scrollbar. Do you know how this change could have caused it?

I think this can be fixed by making the vertical scroll views in the tab bar view not use the primary scroll controller. This would be unrelated to TabBarView, since the list of colors is separate.

@Michal-MK
Copy link
Contributor Author

It happens on master without any changes, may be limited to MacOS then? Here is a video

Screen.Recording.2024-04-04.at.17.13.37.mp4

Copy link
Contributor

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Michal-MK
Copy link
Contributor Author

I'll try altering the Gallery then, I was not sure if that was allowed :) With that I'll see what else needs changing to make this a proper PR.

@Michal-MK
Copy link
Contributor Author

Alright, with the explicit ScrollController for the Scrollbar in the gallery we are all green.

So I'll do some cleanup and write tests tomorrow. Then I'll change it from a Draft to a normal PR.

@Michal-MK
Copy link
Contributor Author

@Piinks It seems I hit another block, this time customer tests are failing.

| 00:07 +201: /Volumes/Work/s/w/ir/x/t/flutter_customer_testing.super_editor.qUNvnh/tests/super_editor/test/super_editor/supereditor_scrolling_test.dart: SuperEditor scrolling stops momentum on tap down and doesn't place the caret (on Android)
| ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
| The following TestFailure was thrown running a test:
| Expected: <1194.3136212798695>
|   Actual: <1191.6569144503435>
|
| When the exception was thrown, this was the stack:
| #4      main.<anonymous closure>.<anonymous closure> (file:///Volumes/Work/s/w/ir/x/t/flutter_customer_testing.super_editor.qUNvnh/tests/super_editor/test/super_editor/supereditor_scrolling_test.dart:501:7)
| <asynchronous suspension>
| #5      testWidgetsOnAndroid.<anonymous closure> (package:flutter_test_runners/src/platform_runners.dart:235:7)
| <asynchronous suspension>
| #6      testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:183:15)
| <asynchronous suspension>
| #7      TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)
| <asynchronous suspension>
| <asynchronous suspension>
| (elided one frame from package:stack_trace)
|
| This was caught by the test expectation on the following line:
|   file:///Volumes/Work/s/w/ir/x/t/flutter_customer_testing.super_editor.qUNvnh/tests/super_editor/test/super_editor/supereditor_scrolling_test.dart line 501
| The test description was:
|   stops momentum on tap down and doesn't place the caret (on Android)
| ════════════════════════════════════════════════════════════════════════════════════════════════════

Those are not part of the repository as far as I know.

Copy link
Contributor

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a legitimate test failure from the customer.
It lives here:
https://github.com/superlistapp/super_editor/blob/661ecb84b4f79bd3f81c1bfd148b36ab7be6993b/super_editor/test/super_editor/supereditor_scrolling_test.dart#L473

When the user taps on a scrolling viewport, the scrolling should halt. This change in behavior would be a breaking change.

I wonder if that is why some of the tests here have changed warnIfMissed? That's also something to look at. Why did those tests change?

There are some options here, either

  • the PR can be changed to fix a bug it may have introduced here
  • or the PR can be changed to make this behavior an opt-in, so folks relying on the prior behavior can continue to work
  • or if the customer test is capturing a bug, that this PR is fixing, we can work with the customer to transition them to the solution without breaking them. (This PR would land after the customer has been fixed)

@@ -78,7 +78,7 @@ class ColorItem extends StatelessWidget {
}
}

class PaletteTabView extends StatelessWidget {
class PaletteTabView extends StatefulWidget {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is currently broken at head, and not related to this change, this should probably be broken out into a separate PR and validated with a test so it does regress in the future.

@@ -2424,7 +2424,8 @@ void main() {
expect(tester.getCenter(find.text('Item 49')).dy, equals(585.0));

// Overscroll, dragging like this will release with 0 velocity.
await tester.drag(find.text('Item 49'), const Offset(0.0, -50.0));
// , warnIfMissed: false
await tester.drag(find.text('Item 49'), const Offset(0.0, -50.0), warnIfMissed: false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these tests broken?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am working on these, even if the drag "misses", the scrollable is dragged as expected which is validated by the expect below. I added the warnIfMissed early in the PR, so it may not be necessary now.

@@ -323,6 +323,10 @@ class _PagePosition extends ScrollPositionWithSingleContext implements PageMetri
// for use when resizing the viewport to non-zero next time.
double? _cachedPage;

/// When page view goes ballistic we want to allow interaction with the page
@override
bool? get preferredBallisticIgnorePointer => false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only PageView?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The property preferredBallisticIgnorePointer is still a proof of concept, right now I am trying to make all tests pass and achieve the behavior I want. Then I'll add tests and refactor it. Here the false essentially mimics trackpad scrolling as that evaluates to false all the time. So for now I just slapped in a false as the preference which makes it behave like a trackpad. But with touch gestures.

/// Some ballistic activities may want to allow input on child widgets.
/// This flag takes precedence over [ScrollActivity.shouldIgnorePointer] when
/// starting a ballistic activity.
bool? get preferredBallisticIgnorePointer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean when this is null? It looks like some scroll positions have an opinion for this and some don't. Is this only relevant with PageView? If so, why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it is only set in PageView because I have not looked at other scrollables. That is something I want to address once the core is working. I can easily test it with PageView for now :)

@Michal-MK
Copy link
Contributor Author

Thanks for the link, I looked at it and from local testing it seems that my current code pumps one extra frame when tapping on a scrollable. In my sample app tap to stop works so I will have to figure out what it up with that test and why my changes break it.

@Michal-MK
Copy link
Contributor Author

Michal-MK commented Apr 13, 2024

@Piinks I got the customer tests working somehow :D

And I managed to remove the changes to the nested_scroll_view and remove one extra warnIfMissed below, as now it will correctly detect hits when the scrollable is over-scrolled.

With that there are a couple of things that need doing:

  1. I do not like the type cast in BallisticScrollActivity to the the ScrollPosition from the delegate, but if I introduce it to the delegate directly, it would mean editing(polluting) a lot of classess again, the outOfRange was fine as most of the implementations already defined the field/getter; but nobody defines updateIgnorePointer currently. Thoughts on this?

  2. There is still a difference between native. That is long presses. Native iOS does not allow long press on over-scrolled views. That would require setting a timeout and after that making the shouldIgnorePointer reset to true. But by then the interactable widget (e.g. Button) would already be highlighted? I have not touched and I have 0 experience with input handling. But I have a feeling that will not be as simple to implement (timeout sync/cancelling etc...). Should that be part of this PR?

  3. Other scrollables besides PageView. Here I implemented it because it was annoying as shown in the videos at the top, but I did not actually test the native behavior of either platform. Maybe it "does not work" there either so it is expected. Because if that was the case, the whole preferredBallisticIgnorePointer would go away, simplifying this PR quite a bit. So I'll need to hack together a native app and test this still.

@Michal-MK
Copy link
Contributor Author

I hacked together a simple page view using Chat GPT in Kotlin and the same janky behavior is present there as well. So... what now? From a UX point of view it is horrible, but it is consistent with the native behavior.

@Michal-MK Michal-MK marked this pull request as ready for review April 17, 2024 15:46
Copy link
Contributor

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hacked together a simple page view using Chat GPT in Kotlin and the same janky behavior is present there as well. So... what now? From a UX point of view it is horrible, but it is consistent with the native behavior.

I am not sure I understand, what is the question? :)

@Michal-MK
Copy link
Contributor Author

2024-04-18.16-15-00.mp4

That the native world has the same issue, so is it then an issue? Should I bother fixing it here if it may be intended?

@Michal-MK
Copy link
Contributor Author

@Piinks How about I remove the code related to the PageView functionality for now, that way we do not need to care about the Colors Gallery issue and at least get the out of bounds scrolling interactivity merged, since that is what I wrote tests for. I think that this functionality is ready. I will create a separate issue with the PageView with the videos from here and another one for the GalleryApp Colors trackpad bug. So we can track it separately and not clutter this one?

@Piinks
Copy link
Contributor

Piinks commented Apr 18, 2024

How about I remove the code related to the PageView functionality for now, that way we do not need to care about the Colors Gallery issue and at least get the out of bounds scrolling interactivity merged, since that is what I wrote tests for. I think that this functionality is ready. I will create a separate issue with the PageView with the videos from here and another one for the GalleryApp Colors trackpad bug. So we can track it separately and not clutter this one?

I am not sure what that would look like? I have not done a thorough review as you've told me a couple times it is not ready or just a proof of concept. The gallery issue is not specific to this change and needs to be fixed anyways. As far as the changes this PR is making, I would rather we sort out the intended behavior before merging any partial fix. We typically do not land partial fixes, or changes where we don't quite know what the behavior should be.

@Michal-MK
Copy link
Contributor Author

@Piinks
"The gallery issue is not specific to this change and needs to be fixed anyways."
Yes that is what we are both saying :D Since you said they are unrelated and should be fixed in a separate PR that is what I repeated.

"As far as the changes this PR is making, I would rather we sort out the intended behavior before merging any partial fix. We typically do not land partial fixes, or changes where we don't quite know what the behavior should be."
I agree with that, that is again why I suggested removing the PageView "fix", since native behaves in the same
"broken" way, or moving it into a separate issue where we can "figure out what the behavior should be". As for the interactivity when a scrollable is over scrolled, that is working and is consistent with the native behavior on iOS/MacOS. The attached videos show this. So I am not sure if by "partial fix" you meant the PageView, the long press or this PR as a whole?

@Piinks
Copy link
Contributor

Piinks commented Apr 26, 2024

Hi @Michal-MK, I've reread the comments above a couple of times, and I apologize I do not understand the state of the proposal. It sounds like some of it works, some of it does not and some of it is matching native behavior. Can you help me understand, from your last comment, if you remove the PageView changes, what problem is this PR solving? What I meant by partial fix was that it sounded like this would only apply to some scrollable classes rather than all of them.

@Michal-MK
Copy link
Contributor Author

@Piinks Sure, when we remove the PageView change (which currently behaves the way native tabs do in Android) we are left with the change to BouncingScrollPhysics not allowing interaction with the contents of the scrollable. When a scrollable with bouncing scroll physics is out of bounds, native iOS allows taps on the scrollable content, however Flutter currently forwards the input to the scrollable and children are "hit test invisible", so now you can only resume the scroll gesture, but not perform a tap on a button in the scrollable.

2024-03-27.19-45-59.mp4
2024-03-27.19-46-46.mp4
native.mp4

These are the relevant videos.

The changes are only in scroll_activty.dart, scroll_position.dart and scroll_position_with_single_context.dart. Then classes that extend/implement these need to now have outOfRange boolean getter, and most do already, so I assume most of the scrollable implementations in Flutter should support it without changes to the source code in them directly (I had to modify only nested_scroll_view.dart to make it compile). I will have to check 2D scrollables as I have not used them yet (even in my apps). The other thing that needs to be looked at, is that long press gestures should not happen on overscrolled scrollables, with this change they do. I asked if someone who has more experience in input handling would want to look at this, but I guess I will have to dig in myself and somehow make the long press gesture bail out of the arena when we are overscrolled. Before I start looking at it, i want to reduce the scope of this PR as now it is addressing 3 unrelated issues at once.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: text input Entering text in a text field or keyboard related problems f: scrolling Viewports, list views, slivers, etc. framework flutter/packages/flutter repository. See also f: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BouncingScrollPhysics Capturing all input until settled
2 participants