Closed Bug 2048545 Opened 3 months ago Closed 1 month ago

Implement Async Clipboard API web custom format support on Android

Categories

(Core :: DOM: Copy & Paste and Drag & Drop, enhancement, P2)

enhancement

Tracking

()

RESOLVED FIXED
156 Branch
Tracking Status
firefox156 --- fixed

People

(Reporter: edenchuang, Assigned: edenchuang)

References

(Depends on 1 open bug, Blocks 1 open bug)

Details

(Keywords: dev-doc-complete)

Attachments

(3 files)

Bug 1956304 adds Async Clipboard API web custom format support (clipboard.write/read with "web foo/bar" flavors, ClipboardItem custom formats, and the clipboard.readCustomFormatsFromClipboard.enabled pref) on macOS, Linux, and Windows desktop.

Android is not yet implemented — the platform clipboard layer doesn't write or read the W3C-spec per-target atoms or the application/web;type="custom/formatmap" JSON map.

Several web-platform tests are marked expected: FAIL if os == "android" in the meta files:

  • testing/web-platform/meta/clipboard-apis/async-custom-formats-write-read.tentative.https.html.ini
  • testing/web-platform/meta/clipboard-apis/async-custom-formats-write-read-web-prefix.tentative.https.html.ini
  • testing/web-platform/meta/clipboard-apis/async-custom-formats-write-read-without-web-prefix.tentative.https.html.ini
  • testing/web-platform/meta/clipboard-apis/async-unsanitized-plaintext-formats-write-read.tentative.https.html.ini

And the in-tree mochitests skip their web custom format sub-tasks when navigator.userAgent.includes("Android"):

  • widget/tests/file_test_clipboard.js (test_web_custom_format, test_read_custom_formats_from_clipboard_pref)
  • widget/tests/file_test_clipboard_getDataSnapshotSync.js (test_web_custom_format_data)

Once Android implements the platform side, those annotations can be removed.

This is the Android-side follow-up Edgar requested in https://phabricator.services.mozilla.com/D271555.

Assignee: nobody → echuang
Severity: -- → S3
Priority: -- → P2

Currently, GeckoView doesn't have a content provider, so GV restricts some MIME type. I guess that we have to add GeckoRuntime.ContentProviderDelegate to support more type such as image/*. It has same things for drag and drop and Web Share API.

If custom MIME type is used on Gecko internal only, it creates something internal MIME Type such as application/x-moz-draganddrop for drag and drop.

Hi Makoto, thank you for the reminders.

Yes, I know that if the mimeType needs to be shared between applications, it needs a content provider.
And unfortunately, this is not only Gecko. Web custom format could be shared with other app.
So I am working on a content provider implementation. This is also why I split Android support from bug 1956304.

WIP/prototype. Mirrors the macOS/Linux/Windows implementations from Bug
1956304 for Android (GeckoView).

The W3C clipboard-apis spec, Appendix A treats Linux/ChromeOS/Android the
same: web custom format payloads are exposed via
"application/web;type="custom/formatmap"" (JSON map) and
"application/web;type="custom/formatN"" (per-essence bytes).

Java side
WebCustomFormatProvider is a new ContentProvider that serves the bytes
for Firefox-authored web custom format clips via content:// URIs of the
shape content://AUTHORITY/<sessionId>/<itemIndex>. It uses a Pipe so
bytes never hit disk.

Clipboard.java gets a session-based write API
(openWebCustomFormatWriteSession / addWebCustomFormatPayload /
commitWebCustomFormatWriteSession) so the C++ side can push payloads
one at a time without a byte[][] JNI hop. A new read API
(getWebCustomFormatMapJson / getWebCustomFormatPayload /
hasWebCustomFormatMap) walks the primary clip's items and resolves
each Uri's MIME via ContentResolver.getStreamTypes().

AndroidManifest_overlay.jinja registers WebCustomFormatProvider with
authority "org.mozilla.gecko.webcustomformat" and grantUriPermissions
so other apps can resolve the URIs cross-app.

C++ side
widget/android/nsClipboard.cpp handles "web foo/bar" flavors in
SetNativeClipboardData by building a WebCustomFormatMap, serialising
to JSON, and pushing through the new session API.

GetNativeClipboardData handles kWebCustomFormatMapType (fetch JSON,
build nsIMutableArray) and "web foo/bar" (fetch JSON, look up
per-essence MIME, fetch bytes).

HasNativeClipboardDataMatchingFlavors recognises both shapes and
lazy-loads the map JSON on the first "web " query.

Depends on Bug 1956304.

Attachment #9599205 - Attachment description: WIP: Bug 2048545 - Support Clipboard web custom format on Android. r?#geckoview-reviewers,#dom-core → WIP: Bug 2048545 - P1 Android web custom format ContentProvider plumbing. r?#geckoview-reviewers,edgar

Gecko (C++) side of Bug 2048545. Wires widget/android/nsClipboard.cpp
into the Java session/read APIs added in P1.

SetNativeClipboardData handles "web foo/bar" flavors by building a
WebCustomFormatMap, serialising to JSON, and pushing the payloads
through Clipboard.OpenWebCustomFormatWriteSession /
AddWebCustomFormatPayload / CommitWebCustomFormatWriteSession. The
standard text/HTML flavors in the same transferable are co-published
on the same ClipData so callers asking for text/plain after a mixed
write still find them.

GetNativeClipboardData adds two branches before the legacy text/raw
path:

  • kWebCustomFormatMapType fetches the map JSON from the primary clip,
    parses it, and returns the per-essence "web foo/bar" flavors as an
    nsIMutableArray.
  • "web foo/bar" fetches the map, looks up the matching per-essence
    MIME, and reads its bytes via the ContentProvider URI.

HasNativeClipboardDataMatchingFlavors recognises both shapes and
lazy-loads the map JSON on the first "web " query.

Depends on Bug 1956304.

Now that Bug 2048545 P1/P2 land the Android-side support for the W3C
Async Clipboard API web custom formats, lift the Android skips and
expected-failure annotations:

  • Drop the Android FAIL/CRASH expectations from the four
    clipboard-apis web-platform test .ini files.
  • Remove the navigator.userAgent.includes("Android") early-returns
    from widget mochitest tasks that exercise web custom formats
    (test_web_custom_format, test_read_custom_formats_from_clipboard_pref,
    test_web_custom_format_data).

Depends on Bug 1956304.

No longer blocks: 1956304
Depends on: 1956304
Attachment #9599205 - Attachment description: WIP: Bug 2048545 - P1 Android web custom format ContentProvider plumbing. r?#geckoview-reviewers,edgar → Bug 2048545 - P1 Android web custom format ContentProvider plumbing. r?#geckoview-reviewers,edgar
Attachment #9599305 - Attachment description: WIP: Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?#geckoview-reviewers,edgar → Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?#geckoview-reviewers,edgar
Attachment #9599306 - Attachment description: WIP: Bug 2048545 - P3 Enable web custom format tests on Android. r?edgar → Bug 2048545 - P3 Enable web custom format tests on Android. r?edgar
Attachment #9599305 - Attachment description: Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?#geckoview-reviewers,edgar → Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?edgar
Attachment #9599205 - Attachment description: Bug 2048545 - P1 Android web custom format ContentProvider plumbing. r?#geckoview-reviewers,edgar → Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar
Attachment #9599305 - Attachment description: Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?edgar → Bug 2048545 - P2 Android Gecko clipboard support for web custom format. r?#geckoview-reviewers,edgar
Attachment #9599205 - Attachment description: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar → WIP: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar
Attachment #9599205 - Attachment description: WIP: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar → Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar
Attachment #9599205 - Attachment description: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar → WIP: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar
Attachment #9599205 - Attachment description: WIP: Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar → Bug 2048545 - P1 Android clipboard ContentProvider plumbing. r?#geckoview-reviewers,edgar
Keywords: dev-doc-needed
QA Whiteboard: [qa-triage-done-c157/b156]

FF156 MDN docs work for this can be tracked in https://github.com/mdn/content/issues/45430

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: