diff --git a/README.md b/README.md index da27d39..3475603 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,7 @@ tools/ .gitea/workflows/ CI that builds, verifies and publishes the .plugin to Releases scripts/ ADB debug-loop automation and the git hooks (see below) docs/sdk-notes.md condensed SDK API surface gathered from the official docs +docs/media-viewer-notes.md app internals behind the Forward button (page-block adapters, rich messages) ``` ## The three commands that matter diff --git a/docs/media-viewer-notes.md b/docs/media-viewer-notes.md new file mode 100644 index 0000000..ba15006 --- /dev/null +++ b/docs/media-viewer-notes.md @@ -0,0 +1,100 @@ +# Media viewer internals — findings + +Notes on how exteraGram opens page-block media, gathered by decompiling the +12.10.6 APK (versionCode 71129) with jadx and testing on a device. Class and +field names are from that build. They are app internals, not SDK API, so check +them again after app updates. + +## Two adapters, one PhotoViewer + +Page-block media never opens with a `MessageObject` list. PhotoViewer runs on a +`PhotoViewer.PageBlocksAdapter` instead, and `onPhotoShow` fires the same way +for both callers below, so one hook covers them. + +| Opened from | Adapter | `getParentObject()` | +|---|---|---| +| Instant View article (`ArticleViewer.openPhoto`) | `ArticleViewer.RealPageBlocksAdapter` | `TLRPC.WebPage` | +| Rich message in a chat (`ChatActivity.openArticlePhoto`) | `ChatActivity.ChatArticlePageBlocksAdapter` | the message's `MessageObject`, or the bare `TL_iv.RichMessage` when constructed without one | + +Both adapters expose the same `getMedia(i)` / `getFile(i)` / `isVideo(i)`. Those +methods resolve through `ArticleViewer.WebPageUtils`, which has overloads for +`WebPage`, `TL_iv.Page` and `TL_iv.RichMessage`. That is why `ivmedia.media_at` +works unchanged for both. Only the parent object tells them apart. + +A rich message is recognised by `MessageObject.messageOwner.rich_message != null`. + +## Carousels (slideshows) + +- **Articles:** `WebpageAdapter.addAllMediaFromBlock` flattens every + `pageBlockSlideshow` / `pageBlockCollage` item into `photoBlocks`. Tapping a + slideshow item therefore goes through `RealPageBlocksAdapter` like any other + article photo, and needed no special handling. +- **Rich messages:** `RichMessageLayout.collectMediaBlocks` flattens single + media and slideshow items into one list for `ChatArticlePageBlocksAdapter`. + Swiping in the viewer walks through all of them, and + `updateSlideshowCell` keeps the in-chat carousel in sync. + +## telegra.ph cannot produce a carousel + +Telegraph has no gallery/slideshow node. The editor has no such block, and +`createPage` keeps only the first `` of a `
` holding several, so +consecutive images render as separate `pageBlockPhoto`s. Carousels only reach +the viewer from other Instant View templates (sites other than telegra.ph, +covered by the `any_article` setting) and from rich messages. + +`telegra.ph/upload` no longer accepts files ("Unknown error"). Test pages have +to reference external image URLs, e.g. on `upload.wikimedia.org`. + +## Rich messages need Premium to send media + +The rich composer is the attach menu's "Article" tab (`ChatAttachAlertRichLayout`). +Sending is locked unless `MessagesController.richEditorAllowed()` or +`UserConfig.isPremium()`, whenever the content is "lossy" +(`RichMessageConvert.isLossy`). Lossy means anything a plain message can't +represent, and every media block is lossy. A non-Premium account therefore +cannot send a test rich message with media. + +To test without one, build the message locally and open it the same way +`ChatActivity.openArticlePhoto` does: + +```python +rm = TL_iv.RichMessage() # fill rm.photos and rm.blocks +msg = TLRPC.TL_message() # peer_id/id/dialog_id of any chat +msg.rich_message = rm +mo = MessageObject(account, msg, False, False) +adapter = ChatActivity.ChatArticlePageBlocksAdapter(rm, flat_blocks, mo) +viewer = PhotoViewer.getInstance() +viewer.setParentActivity(chat_fragment) +viewer.openPhoto(index, adapter, PhotoViewer.EmptyPhotoViewerProvider()) +``` + +Real `TLRPC.Photo` objects with server locations can be borrowed from the +`cached_page.photos` of any loaded article. Pointing `peer_id` at a channel +exercises the message-link caption. + +`ArticleViewer.open(TL_webPage, anchor)` re-requests `messages.getWebPage` and +replaces the page with the server copy. Copy the original `hash` onto the +injected `TL_webPage` to keep a locally modified `cached_page`. + +## Message links + +Only channels and supergroups (`ChatObject.isChannel`) have message links: +`https://t.me//` when public, `https://t.me/c//` +otherwise. On Android the dialog id of a channel is `-chat.id`, with no `-100` +prefix. DMs, basic groups and Saved Messages have no link, so media forwarded +from rich messages there is sent without a caption. + +## File references + +Pass the adapter's parent object (WebPage or MessageObject) as `parentObject` +to `FileLoader.loadFile`. `FileRefController` uses it to refresh an expired +file reference, and it handles rich messages explicitly. + +## Device debugging + +Logcat can be completely silent when the device's log buffer is disabled in +developer options, and `scripts/logs.sh` then shows nothing. A plugin can +still write diagnostics to +`ApplicationLoader.applicationContext.getExternalFilesDir(None)`, which maps +to `/sdcard/Android/data/com.exteragram.messenger/files/` and is readable with +`adb shell cat` without `run-as`. diff --git a/plugin.json b/plugin.json index cfa5be4..d4fddae 100644 --- a/plugin.json +++ b/plugin.json @@ -1,9 +1,9 @@ { - "id": "telegraph_media_forward", - "name": "Telegraph Media Forward", - "description": "Adds a Forward action to media opened from telegra.ph pages: downloads the media and re-sends it from your account with a link back to the article.", + "id": "extended_media_forwards", + "name": "Extended Media Forwards", + "description": "Adds a Forward action to media opened from telegra.ph pages and rich messages (single media and carousels): downloads the media and re-sends it from your account with a link back to its source.", "author": "@lyuksovannyy", - "version": "1.0.0", + "version": "1.1.1", "icon": "exteraPlugins/1", "app_version": ">=12.5.1", "sdk_version": ">=1.4.4.3", diff --git a/src/icons.py b/src/icons.py index 0cd2081..ccf3eab 100644 --- a/src/icons.py +++ b/src/icons.py @@ -49,11 +49,11 @@ def drawable(context, res_id): if packed is not None: return packed except Exception as exc: - log("[telegraph_media_forward] icon pack lookup failed: %s" % exc) + log("[extended_media_forwards] icon pack lookup failed: %s" % exc) try: res_id = manager.getIcon(res_id) except Exception as exc: - log("[telegraph_media_forward] icon remap failed: %s" % exc) + log("[extended_media_forwards] icon remap failed: %s" % exc) resources = context.getResources() if density: return resources.getDrawableForDensity(res_id, density, theme) diff --git a/src/ivmedia.py b/src/ivmedia.py index e9afcf4..e429fad 100644 --- a/src/ivmedia.py +++ b/src/ivmedia.py @@ -1,9 +1,13 @@ -"""Reads media out of the PhotoViewer instance that Instant View opened. +"""Reads media out of a PhotoViewer that runs on a `PageBlocksAdapter`. When ArticleViewer opens a telegra.ph photo/video, PhotoViewer runs with `currentMessageObject == null` and a `PageBlocksAdapter` instead. That adapter is the only handle to the underlying `TLRPC.Photo` / `TLRPC.Document` and to the `TLRPC.WebPage` the article came from, so everything here goes through it. + +Rich messages (page blocks sent as a chat message) open their single media and +carousels the same way, through ChatActivity's `ChatArticlePageBlocksAdapter`, +whose parent object is the message's `MessageObject` instead of a WebPage. """ from hook_utils import get_private_field @@ -81,6 +85,52 @@ def is_telegraph(adapter): return any(host == h or host.endswith("." + h) for h in TELEGRAPH_HOSTS) +def rich_parent(adapter): + """For media opened from a rich message: the `MessageObject` carrying it, + or the bare `TL_iv.RichMessage` when the adapter has no message. None for + anything else, articles included.""" + from org.telegram.messenger import MessageObject + from org.telegram.tgnet.tl import TL_iv + + parent = adapter.getParentObject() + if isinstance(parent, MessageObject): + owner = parent.messageOwner + if owner is not None and owner.rich_message is not None: + return parent + return None + if isinstance(parent, TL_iv.RichMessage): + return parent + return None + + +def is_rich_message(adapter): + return rich_parent(adapter) is not None + + +def message_link(account, message): + """t.me link to `message`, or None when it has none. + + Only channels and supergroups have message links: public ones by username, + private ones through t.me/c/. DMs, basic groups and unsent local messages + cannot be linked to. + """ + from org.telegram.messenger import ChatObject, MessageObject, MessagesController + + if not isinstance(message, MessageObject): + return None + msg_id = message.getId() + dialog_id = message.getDialogId() + if msg_id <= 0 or dialog_id >= 0: + return None + chat = MessagesController.getInstance(account).getChat(-dialog_id) + if chat is None or not ChatObject.isChannel(chat): + return None + username = ChatObject.getPublicUsername(chat) + if username: + return "https://t.me/%s/%d" % (username, msg_id) + return "https://t.me/c/%d/%d" % (chat.id, msg_id) + + def media_at(adapter, index): """`(media, existing_file_path_or_None, is_video)` for the shown block.""" media = adapter.getMedia(index) diff --git a/src/main.py b/src/main.py index 7d11a0b..a1aa4e4 100644 --- a/src/main.py +++ b/src/main.py @@ -17,7 +17,7 @@ _BUTTON_SIZE_DP = 48.0 _LOAD_TOKEN = "tmf-%d" % id(object()) -class TelegraphMediaForwardPlugin(BasePlugin): +class ExtendedMediaForwardsPlugin(BasePlugin): def on_plugin_load(self): viewer = find_class("org.telegram.ui.PhotoViewer") if viewer is None: @@ -31,7 +31,7 @@ class TelegraphMediaForwardPlugin(BasePlugin): def create_settings(self): return [ - Header(text="Telegraph Media Forward"), + Header(text="Extended Media Forwards"), Switch( key="any_article", text="Allow on any Instant View article", @@ -39,18 +39,27 @@ class TelegraphMediaForwardPlugin(BasePlugin): subtext="Off: only telegra.ph and graph.org pages.", icon="msg_link", ), + Switch( + key="rich_messages", + text="Allow on rich messages", + default=True, + subtext="Single media and carousels inside rich (article) messages.", + icon="msg_media", + ), Text( text="How it works", subtext=( - "Open a photo or video from an article, tap the ⋮ menu and choose " + "Open a photo or video from an article or a rich message and tap " "Forward. The file is downloaded and sent from your account with " - "an Original link to the page." + "a link back to its source when it has one." ), icon="msg_info", ), ] def allows(self, adapter): + if ivmedia.is_rich_message(adapter): + return self.get_setting("rich_messages", True) if self.get_setting("any_article", False): return ivmedia.page_url(adapter) is not None return ivmedia.is_telegraph(adapter) @@ -60,7 +69,6 @@ class TelegraphMediaForwardPlugin(BasePlugin): adapter = ivmedia.adapter_of(viewer) if adapter is None: return - url = ivmedia.page_url(adapter) index = ivmedia.current_index(viewer) try: media, path, is_video = ivmedia.media_at(adapter, index) @@ -68,6 +76,8 @@ class TelegraphMediaForwardPlugin(BasePlugin): sender.notify_error(str(exc)) return account = get_selected_account() + caption = _caption_for(account, adapter) + parent = adapter.getParentObject() context = ivmedia.parent_activity(viewer) if context is None: return @@ -78,17 +88,27 @@ class TelegraphMediaForwardPlugin(BasePlugin): # The sheet sits above the viewer's window; drop it so the sent # message is what the user lands on. viewer.closePhoto(False, False) - sender.send_media(account, media, path, is_video, url, peers, comment) + sender.send_media(account, media, path, is_video, caption, peers, comment, parent) try: # ShareAlert owns its own window, so it shows over PhotoViewer just # like the stock share button does - no need to close the viewer. picker.pick_chats(context, picked, ivmedia.resources_provider(viewer)) except Exception as exc: - log("[telegraph_media_forward] picker failed: %s" % exc) + log("[extended_media_forwards] picker failed: %s" % exc) sender.notify_error(str(exc)) +def _caption_for(account, adapter): + """`(text, url)` linking back to where the media came from, or None.""" + rich = ivmedia.rich_parent(adapter) + if rich is not None: + url = ivmedia.message_link(account, rich) + return (sender.RICH_CAPTION_TEXT, url) if url else None + url = ivmedia.page_url(adapter) + return (sender.CAPTION_TEXT, url) if url else None + + class _OnPhotoShow(MethodHook): """Adds (or re-shows) the Forward item every time the viewer opens media.""" @@ -122,7 +142,7 @@ def _sync_menu(plugin, viewer): button = _add_button(plugin, menu, viewer) button.setVisibility(View.VISIBLE if visible else View.GONE) except Exception as exc: - log("[telegraph_media_forward] menu sync failed: %s" % exc) + log("[extended_media_forwards] menu sync failed: %s" % exc) def _find_button(menu): diff --git a/src/picker.py b/src/picker.py index 8b6f95e..e58f272 100644 --- a/src/picker.py +++ b/src/picker.py @@ -89,4 +89,4 @@ def _enable_focus(alert): if alert.getWindow() is not None: alert.setFocusable(True) except Exception as exc: - log("[telegraph_media_forward] focus handoff failed: %s" % exc) + log("[extended_media_forwards] focus handoff failed: %s" % exc) diff --git a/src/sender.py b/src/sender.py index 813689f..3df7e06 100644 --- a/src/sender.py +++ b/src/sender.py @@ -1,9 +1,11 @@ -"""Downloads telegra.ph media and re-sends it from the user's own account. +"""Downloads page-block media and re-sends it from the user's own account. Instant View media lives on the article's WebPage, not in any message, so it -cannot be forwarded. Instead the file is fetched through FileLoader (or reused -if already cached) and uploaded as a fresh photo/document, exactly like picking -the file from the gallery. Caption is a single markdown link to the article. +cannot be forwarded. Rich-message media is part of a message, but only as one +block of it, so forwarding would carry the whole message along. Instead the +file is fetched through FileLoader (or reused if already cached) and uploaded +as a fresh photo/document, exactly like picking the file from the gallery. The +caption, when there is a source to link to, is a single link back to it. """ import threading @@ -33,25 +35,31 @@ def notify_success(text): CAPTION_TEXT = "telegra.ph source" +RICH_CAPTION_TEXT = "message source" -def send_media(account, media, cached_path, is_video, page_url, peers, comment): - """Starts the download+send in the background. Safe to call on the UI thread.""" +def send_media(account, media, cached_path, is_video, caption, peers, comment, parent=None): + """Starts the download+send in the background. Safe to call on the UI thread. + + `caption` is `(text, url)` for a linked caption or None for none. `parent` + is the object the media belongs to (WebPage or MessageObject); FileLoader + uses it to refresh an expired file reference. + """ def worker(): try: - path = cached_path or _download(account, media, is_video) + path = cached_path or _download(account, media, is_video, parent) for peer in peers: if comment: _send(account, _text_params(comment, peer)) - _send(account, _media_params(account, media, path, is_video, page_url, peer)) + _send(account, _media_params(account, media, path, is_video, caption, peer)) except Exception as exc: - log("[telegraph_media_forward] send failed: %s" % exc) + log("[extended_media_forwards] send failed: %s" % exc) notify_error(str(exc)) return notify_success("Sent") - threading.Thread(target=worker, name="telegraph-media-forward", daemon=True).start() + threading.Thread(target=worker, name="extended-media-forwards", daemon=True).start() def _text_params(text, peer): @@ -60,12 +68,14 @@ def _text_params(text, peer): return SendMessagesHelper.SendMessageParams.of(text, peer) -def _media_params(account, media, path, is_video, page_url, peer): +def _media_params(account, media, path, is_video, caption, peer): from org.telegram.messenger import SendMessagesHelper params = SendMessagesHelper.SendMessageParams.of(None, peer) - params.caption = CAPTION_TEXT - params.entities = _link_entities(page_url) + if caption is not None: + text, url = caption + params.caption = text + params.entities = _link_entities(text, url) params.path = path if is_video: params.document = _fresh_document(account, media, path) @@ -77,14 +87,14 @@ def _media_params(account, media, path, is_video, page_url, peer): return params -def _link_entities(url): +def _link_entities(text, url): """Whole caption as one text_url entity, so it renders as a link.""" from java.util import ArrayList from org.telegram.tgnet import TLRPC entity = TLRPC.TL_messageEntityTextUrl() entity.offset = 0 - entity.length = len(CAPTION_TEXT) + entity.length = len(text) entity.url = url entities = ArrayList() entities.add(entity) @@ -119,7 +129,7 @@ def _send(account, params): try: get_send_messages_helper(account).sendMessage(params) except Exception as exc: - log("[telegraph_media_forward] sendMessage failed: %s" % exc) + log("[extended_media_forwards] sendMessage failed: %s" % exc) finally: done.set() @@ -128,7 +138,7 @@ def _send(account, params): done.wait(30) -def _download(account, media, is_video): +def _download(account, media, is_video, parent=None): """Loads the file through FileLoader and returns its on-disk path.""" from org.telegram.messenger import FileLoader, ImageLocation from org.telegram.tgnet import TLRPC @@ -136,11 +146,11 @@ def _download(account, media, is_video): loader = FileLoader.getInstance(account) if is_video or isinstance(media, TLRPC.Document): target = media - loader.loadFile(media, None, FileLoader.PRIORITY_HIGH, 0) + loader.loadFile(media, parent, FileLoader.PRIORITY_HIGH, 0) else: target = _best_size(media) loader.loadFile( - ImageLocation.getForPhoto(target, media), None, "jpg", FileLoader.PRIORITY_HIGH, 0 + ImageLocation.getForPhoto(target, media), parent, "jpg", FileLoader.PRIORITY_HIGH, 0 ) candidates = [loader.getPathToAttach(target, True), loader.getPathToAttach(target, False)] return _wait_for_file(candidates) diff --git a/tools/verify_bundle.py b/tools/verify_bundle.py index 069174a..caed0dc 100755 --- a/tools/verify_bundle.py +++ b/tools/verify_bundle.py @@ -116,6 +116,42 @@ class FakeWebPage: self.display_url = display_url +class FakeRichMessage: + """Stand-in for TL_iv.RichMessage.""" + + +class FakeMessage: + def __init__(self, rich_message=None): + self.rich_message = rich_message + + +class FakeMessageObject: + def __init__(self, msg_id, dialog_id, rich_message=None): + self.messageOwner = FakeMessage(rich_message) + self._id = msg_id + self._dialog_id = dialog_id + + def getId(self): + return self._id + + def getDialogId(self): + return self._dialog_id + + +class FakeChat: + def __init__(self, chat_id, channel, username=None): + self.id = chat_id + self.channel = channel + self.username = username + + +FAKE_CHATS = { + 100: FakeChat(100, channel=True, username="publicchan"), + 200: FakeChat(200, channel=True), + 300: FakeChat(300, channel=False), +} + + def install_fakes() -> None: base_plugin_mod = types.ModuleType("base_plugin") base_plugin_mod.BasePlugin = BasePlugin @@ -158,6 +194,22 @@ def install_fakes() -> None: ), ) tgnet_mod.TLRPC = tlrpc + tgnet_mod.__path__ = [] + tgnet_tl_mod = types.ModuleType("org.telegram.tgnet.tl") + tgnet_tl_mod.TL_iv = types.SimpleNamespace(RichMessage=FakeRichMessage) + tgnet_mod.tl = tgnet_tl_mod + sys.modules["org.telegram.tgnet.tl"] = tgnet_tl_mod + + messenger_mod = types.ModuleType("org.telegram.messenger") + messenger_mod.MessageObject = FakeMessageObject + messenger_mod.ChatObject = types.SimpleNamespace( + isChannel=lambda chat: chat.channel, + getPublicUsername=lambda chat: chat.username, + ) + messenger_mod.MessagesController = types.SimpleNamespace( + getInstance=lambda account: types.SimpleNamespace(getChat=FAKE_CHATS.get) + ) + sys.modules["org.telegram.messenger"] = messenger_mod # `java.util.ArrayList` only ever gets add()ed to and iterated. java_mod = types.ModuleType("java") @@ -346,11 +398,32 @@ def main(argv=None) -> int: plugin.allows(other) and not plugin.allows(no_page), ) + rich_msg = FakeAdapter(FakeMessageObject(5, -1000000000200, FakeRichMessage())) + rich_bare = FakeAdapter(FakeRichMessage()) + plain_msg = FakeAdapter(FakeMessageObject(5, -1000000000200, None)) + check( + "settings expose the rich_messages switch", + any(getattr(row, "key", None) == "rich_messages" for row in rows), + ) + check( + "rich message media is accepted by default (with or without a message)", + plugin.allows(rich_msg) and plugin.allows(rich_bare), + ) + check( + "message parent without a rich message is rejected", + not plugin.allows(plain_msg), + ) + plugin.set_setting("rich_messages", False) + check( + "rich_messages setting off rejects rich message media", + not plugin.allows(rich_msg) and not plugin.allows(rich_bare), + ) + ivmedia = ns.get("ivmedia") sender = ns.get("sender") caption = getattr(sender, "CAPTION_TEXT", None) link_entities = getattr(sender, "_link_entities", None) - entities = link_entities("https://telegra.ph/x") if link_entities else [] + entities = link_entities(caption, "https://telegra.ph/x") if link_entities and caption else [] entity = entities[0] if entities else None check( "caption is a single text_url entity covering the whole caption", @@ -363,6 +436,33 @@ def main(argv=None) -> int: f"got caption={caption!r} entities={entities!r}", ) + message_link = getattr(ivmedia, "message_link", None) + if message_link is not None: + # Channel dialog ids are -(chat id) in this fake, like getDialogId(). + links = ( + message_link(0, FakeMessageObject(7, -100)), + message_link(0, FakeMessageObject(8, -200)), + message_link(0, FakeMessageObject(9, -300)), + message_link(0, FakeMessageObject(9, 12345)), + message_link(0, FakeMessageObject(-1, -100)), + message_link(0, FakeRichMessage()), + ) + else: + links = None + check( + "message_link: public/private channels linked; groups, DMs, local and bare rich messages not", + links + == ( + "https://t.me/publicchan/7", + "https://t.me/c/200/8", + None, + None, + None, + None, + ), + f"got {links!r}", + ) + media_at = getattr(ivmedia, "media_at", None) no_media = getattr(ivmedia, "NoMedia", None) raised = False