这是indexloc提供的服务,不要输入任何密码
Skip to content

feature: make the text input plugin use the correct view on the Windows platform #163847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 4, 2025

Conversation

mattkae
Copy link
Contributor

@mattkae mattkae commented Feb 21, 2025

What's new?

  • Updates the TextInput.setClient method to expect a view ID, which is already being sent up by clients. This makes it so that the IME info shows on the text input correctly across views 🎉. Also - text input on windows works properly across views (although it was working before too)
  • Using the view ID in TextInputPlugin::HandleMethodCall to resolve the view
  • Update tests to no longer assume that we are using the implicit view id
  • Add two tests to ensure that the view id is set

What's fixed?

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

@github-actions github-actions bot added a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. platform-windows Building on or for Windows specifically a: desktop Running on desktop labels Feb 21, 2025
@mattkae mattkae force-pushed the text-input-plugin-multi-view branch from ebf2e3f to a40000a Compare February 21, 2025 15:11
@mattkae mattkae marked this pull request as ready for review February 21, 2025 15:19
@mattkae mattkae changed the title feature: migrate the text input plugin for the Windows platform for multi view feature: make the text input plugin use the correct view on the Windows platform Feb 21, 2025
@mattkae mattkae force-pushed the text-input-plugin-multi-view branch from a40000a to 95b35a0 Compare February 21, 2025 16:36
@hbatagelo
Copy link
Contributor

I've noticed some errors that may be related to the text input plugin for multi view. After applying these changes to the canonical:foundation branch and adding a TextField widget to both the main window and the regular windows of the reference app, if I a focus the text field widget and then close the window (using Alt+F4 or clicking on the close button), sometimes I get the following error at message_codecs.dart:168:

Exception has occurred.
PlatformException (PlatformException(Internal Consistency Error, Text input is not available in Windows headless mode, null, null))

This issue wasn't introduced by this PR, as similar (though not identical) behavior can be observed in canonical:foundation.

Files changed in the reference app: ref_app_lib.zip

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from a text input standpoint. We already send the viewId as part of the TextInputConfiguration so this should just wire up. Is there an example app somewhere that I can use to test this out locally?

@mattkae
Copy link
Contributor Author

mattkae commented Feb 24, 2025

I've noticed some errors that may be related to the text input plugin for multi view. After applying these changes to the canonical:foundation branch and adding a TextField widget to both the main window and the regular windows of the reference app, if I a focus the text field widget and then close the window (using Alt+F4 or clicking on the close button), sometimes I get the following error at message_codecs.dart:168:

Exception has occurred.
PlatformException (PlatformException(Internal Consistency Error, Text input is not available in Windows headless mode, null, null))

This issue wasn't introduced by this PR, as similar (though not identical) behavior can be observed in canonical:foundation.

Files changed in the reference app: ref_app_lib.zip

So two things appear to be happening here:

  1. This is a bad error message. The real issue is that we are unable to find the view. I will update the logging to reflect that and write a test for this case.
  2. We're experiencing some sort of race. The framework asks the engine to TextInput.setClient. In the meantime, the window (and thus the view) gets destroyed. Hence, when we go to resolve the view later on from the provided ID, it is already destroyed. I'll see if there is anything that we can do about this, but it might just be an "expected" error.

@mattkae
Copy link
Contributor Author

mattkae commented Feb 24, 2025

LGTM from a text input standpoint. We already send the viewId as part of the TextInputConfiguration so this should just wire up. Is there an example app somewhere that I can use to test this out locally?

I have an example app that works with the multi window pull request. If you merge up this branch into foundation in canonical/flutter, then you can run the provided app like so:

flutter run --debug --local-engine-src-path C:/dev/flutter/engine/src/ --local-engine host_debug_unopt --local-engine-host host_debug_unopt .\lib\services\text_input\test.dart --enable-multi-window

(replacing my paths with your paths, and I added test.dart to the api example app for ease of use)

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';

void main() {
  final RegularWindowController controller1 = RegularWindowController(size: const Size(640, 480));
  final RegularWindowController controller2 = RegularWindowController(size: const Size(640, 480));
  runWidget(ViewCollection(
    views: [
      RegularWindow(controller: controller1, child: MyApp()),
      RegularWindow(controller: controller2, child: MyApp()),
    ]
  ));
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Name Prompt',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: NameInputScreen(),
    );
  }
}

class NameInputScreen extends StatefulWidget {
  @override
  _NameInputScreenState createState() => _NameInputScreenState();
}

class _NameInputScreenState extends State<NameInputScreen> {
  final TextEditingController _controller = TextEditingController();
  String? _name;

  void _submitName() {
    setState(() {
      _name = _controller.text;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Enter Your Name')),
      body: Padding(
        padding: const EdgeInsets.all(16.0),
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            TextField(
              controller: _controller,
              decoration: InputDecoration(labelText: 'Name'),
            ),
            SizedBox(height: 16),
            ElevatedButton(
              onPressed: _submitName,
              child: Text('Submit'),
            ),
            if (_name != null) ...[
              SizedBox(height: 20),
              Text(
                'Hello, $_name!',
                style: TextStyle(fontSize: 20, fontWeight: FontWeight.bold),
              ),
            ],
          ],
        ),
      ),
    );
  }
}

@mattkae mattkae requested a review from loic-sharma February 24, 2025 15:52
@mattkae
Copy link
Contributor Author

mattkae commented Feb 24, 2025

@loic-sharma What do you think about the issue brought up by Harlen #163847 (comment)?

I might be mistaken, but since the Window (and thus the view) is being destroyed in another thread, we might encounter an invalid view ID in this thread by the time that we try to resolve the view. What do you think is appropriate here?

@loic-sharma
Copy link
Member

@loic-sharma What do you think about the issue brought up by Harlen #163847 (comment)?

I might be mistaken, but since the Window (and thus the view) is being destroyed in another thread, we might encounter an invalid view ID in this thread by the time that we try to resolve the view. What do you think is appropriate here?

Key events and text input requires multiple asynchronous roundtrips. I suspect synchronizing a view destruction with pending key/text events is hard when we have split UI/platform threads.

@mattkae Can you reproduce this bug when Flutter Windows is in the merged thread mode that Matt Knopp added recently?

@justinmc For the split UI/platform thread mode, can the framework handle this scenario (likely by warning and ignoring the exception) if the Windows embedder produces a better exception?

@mattkae
Copy link
Contributor Author

mattkae commented Feb 25, 2025

Key events and text input requires multiple asynchronous roundtrips. I suspect synchronizing a view destruction with pending key/text events is hard when we have split UI/platform threads.

That's what appears to be happening.

@mattkae Can you reproduce this bug when Flutter Windows is in the merged thread mode that Matt Knopp added recently?

It still happens in this cases as far as I can tell.

@loic-sharma
Copy link
Member

@mattkae Can you reproduce this bug when Flutter Windows is in the merged thread mode that Matt Knopp added recently?

It still happens in this cases as far as I can tell.

That's unfortunate. Why does that happen? Can we fix it?

@justinmc
Copy link
Contributor

I would expect the error to still occur in merged thread mode because the method channels (like TextInput.setClient) are still happening asynchronously, even though the sender and recipient are now on the same thread. It's a longer term refactor for us to move text input to FFI or something in order to take advantage of the single thread.

@mattkae
Copy link
Contributor Author

mattkae commented Feb 26, 2025

I would expect the error to still occur in merged thread mode because the method channels (like TextInput.setClient) are still happening asynchronously, even though the sender and recipient are now on the same thread. It's a longer term refactor for us to move text input to FFI or something in order to take advantage of the single thread.

@loic-sharma

@hbatagelo also found the following:

"When we close the window with a focused text field, the focus manager notifies the listeners that the focus has changed, and TextInput.clearClient is called as expected. However, the focus manager notifies the listeners a second time (no idea why), causing the text field to get focus again (TextInput.setClientis called, starting a new transaction). Then the view is destroyed, and focus changes once again, but now TextInput.clearClient is called for the destroyed view, leading to the exception"

Seems like a bug (but perhaps an unrelated/deeper one with the focus manager that we should solve separately)

@mattkae mattkae requested a review from loic-sharma February 27, 2025 18:59
@justinmc
Copy link
Contributor

justinmc commented Feb 27, 2025

@mattkae I finally was able to get set up on a Windows machine to try this (per #163847 (comment)), but what is .\lib\services\text_input\test.dart? If I delete that in the command you gave then I get a single visible window and two windows that show up in the task bar but otherwise aren't visible.

I'm hoping to investigate what's going on with the second notification mentioned in #163847 (comment).

Copy link
Member

@loic-sharma loic-sharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks!

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 4, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 4, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 4, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Mar 5, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: desktop Running on desktop a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. platform-windows Building on or for Windows specifically
Projects
None yet
Development

Successfully merging this pull request may close these issues.

☂️ Multi View for Windows/MacOS
4 participants