XMPP Message Attaching, Fastening, References
Services like Twitter and Slack have functionality that attempts to interpret parts of the plain text of tweets or message as entered by the user. Pieces of the text that look like links, mentions of another user, hash tags, or stock symbols, cause additional meta data to be added to the object representing the message, so that receiving clients can mark up those pieces of text in a special way. Twitter calls this meta data Tweet Entities and for each piece of interpreted text, it includes indices for the start and end of along with additional information depending on the type of entity. A client can then do in-line replacements at the exact character indices, e.g. by making it into a hyperlink. Twitter Entities served as inspiration for XEP-0372: References.
References can be used in two ways: including a
reference as a sibling to the body element of a
message. The begin and end attributes then point to the indices of
the plain text in the body. This would typically be used if the
interpretation of the message is done by the sending client.
Alternatively, a service (e.g. a MUC service) could parse incoming
messages and send a separate stanza to mark up the original stanza. In
this case you need a mechanism for pointing to that other message.
There have been two proposals for this, with slightly differing
approaches, and in the examples below, I'll use the proto-XEP Message Fastening. While pointing to the stanza ID of
the other message, it embeds a reference element in the
apply-to element.
Mentioning another user
Let's start out with the example of mentioning another user.
<message from="room@muc.this.example/Kev" type="groupchat">
<stanza-id id="2019-09-02-1" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<body>Some rubbish @ralphm</body>
</message>A client might render this as:
Some rubbish @ralphm
The MUC service then parses the plain-text message, and finds a reference to my nickname prefixed with an @-sign, and sends a stanza to the room that marks up the message Kev sent to me.
<message from="room@muc.this.example"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-2" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-1">
<reference begin="13" end="19" xmlns="urn:example:reference:0">
<mention jid="room@muc.this.example/ralphm"/>
</reference>
</apply-to>
</message>This stanza declares that it is attached to the previous message by the stanza ID that was included with the original stanza. In its payload, it includes a reference, referring to the characters 13 through 19. It has a mention child pointing to my occupant JID. Alternatively, the room might have linked to my real JID. A client can then alter the presentation of the original message to use the attached mention reference:
Some rubbish @ralphm
The characters referencing @ralphm are now highlighted, hovering the mention shows a tooltip with my full name, and clicking on it brings you to a page describing me. This information was not present in the stanza, but a client can use the XMPP URI as a key to present additional information. E.g. from the user's contact list, by doing a vCard lookup, etc.
Note:
The current specification for References does not have defined child elements, but instead uses a type attribute and URIs. However, Jonas Wielicki Schäfer provided some valuable feedback, suggesting this idea. By using a dedicated element for the target of the reference, each can have their own attributes, making it more explicit. Also, it is a natural extension point, by including a differently namespaced element instead.
Referring to previous messages
<message from="room@muc.this.example/Ge0rG"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-3" by="room@muc.this.example"/>
<reference begin="0" end="6" xmlns="urn:example:reference:0">
<mention jid="room@muc.this.example/ralphm"/>
</reference>
<reference begin="26" end="32" xmlns="urn:example:reference:0">
<message id="2019-09-02-1"/>
</reference>
<body>@ralphm did you see Kev's message earlier?</body>
</message>Unlike before, this example does not point to another stanza with apply-to. Instead, Ge0rG's client added references to go along with the plain-text body: one for the mention of me, and one for a reference to an earlier message.
Emoji Reactions
Instead of reacting with a full message, Slack, like online forum software much earlier, has the ability to attach emoji reactions to messages.
<message from="room@muc.this.example/Kev"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-4" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-3">
<reactions xmlns="urn:example:reactions:0">
<reaction label=":+1:">👍</reaction>
</reactions>
</apply-to>
</message><message from="room@muc.this.example/ralphm"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-6" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-3">
<reactions xmlns="urn:example:reactions:0">
<reaction label=":parrot:"
img="cid:b729aec3f521694a35c3fc94d7477b32bc6444ca@bob.xmpp.org"/>
</reactions>
</apply-to>
</message>These two examples show two separate instances of a person
reacting to the previous message by Ge0rG. It uses
the protocol from Message
Reactions, another Proto-XEP. However, I expanded on it by
introducing two new attributes. The label allows for a textual shorthand, that
might be typed by a user. Custom emoji can be represented with the
img attribute, that points to a XEP-0231: Bits of
Binary object.
The attached emoji are rendered below the original message, and hovering over them reveals who were the respondents. Here my own reaction is highlighted by a squircle border.
Including a link
<message from="room@muc.this.example/ralphm" type="groupchat">
<stanza-id id="2019-09-02-7" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<body>Have you seen https://ralphm.net/blog/2013/10/10/logitech_t630?</body>
</message><message from="room@muc.this.example"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-8" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-7">
<reference begin="14" end="61" xmlns="urn:example:reference:0">
<link url="https://ralphm.net/blog/2013/10/10/logitech_t630"/>
</reference>
</apply-to>
</message>Here the MUC service marks up the original messages with an explicit link reference. Possibly, the protocol might be extended so that a service can include shortened versions of the URL for display purposes.
Have you seen https://ralphm.net/blog/2013/10/10/logitech_t630?
Logitech input devices are my favorite. This tiny bluetooth mouse is a nice portable device for every day use or while traveling.
The client has used the markup to fetch meta data on the URL and presents a summary card below the original message. Alternatively, the MUC service could have done this using XEP-0385: Stateless Inline Media Sharing (SIMS):
<message from="room@muc.this.example"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-8" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-7">
<reference begin="14" end="61" xmlns="urn:example:reference:0">
<link url="https://ralphm.net/blog/2013/10/10/logitech_t630"/>
<card xmlns="urn:example:card:0">
<title>Logitech Ultrathin Touch Mouse</ulink></title>
<description>Logitech input devices are my favorite. This tiny bluetooth mouse is a nice portable device for every day use or while traveling.</description>
</card>
<media-sharing xmlns='urn:xmpp:sims:1'>
<file xmlns='urn:xmpp:jingle:apps:file-transfer:5'>
<media-type>image/jpeg</media-type>
<name>ultrathin-touch-mouse-t630.jpg</name>
<size>23458</size>
<hash xmlns='urn:xmpp:hashes:2' algo='sha3-256'>5TOeoNI9z6rN5f+cQagnCgxitQE0VUgzCMeQ9JqbhWJT/FzPpDTTFCbbo1jWwOsIoo9u0hQk6CPxH4t/dvTN0Q==</hash>
<thumbnail xmlns='urn:xmpp:thumbs:1'uri='cid:sha1+21ed723481c24efed81f256c8ed11854a8d47eff@bob.xmpp.org' media-type='image/jpeg' width='116' height='128'/>
</file>
<sources>
<reference xmlns='urn:xmpp:reference:0' type='data' uri='https://test.ralphm.net/images/blog/ultrathin-touch-mouse-t630.jpg' />
</sources>
</media-sharing>
</reference>
</apply-to>
</message>Editing a previous message
<message from="room@muc.this.example/ralphm" type="groupchat">
<stanza-id id="2019-09-02-9" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<body>Some thoughtful reply</body>
</message>Some thoughtful reply
After sending that message, I want to add a bit more information:
<message from="room@muc.this.example/ralphm" type="groupchat">
<stanza-id id="2019-09-02-10" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-9">
<external name='body'/>
<replace xmlns='urn:example:message-correct:1'/>
</apply-to>
<body>Some more thoughtful reply</body>
</message>Unlike XEP-0308: Last Message Correction, this example uses Fastening to refer to the original message. I would also lift the restriction on correcting just the last message, but allow any previous message to be edited.
Some more thoughtful reply
Upon receiving the correction, the client indicates that the message has been edited. Hovering over the marker reveals when the message was changed.
Editing a previous message that had fastened references
<message from="room@muc.this.example/Kev" type="groupchat">
<stanza-id id="2019-09-02-11" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<body>A witty response mentioning @ralphm</body>
</message><message from="room@muc.this.example"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-12" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-11">
<reference begin="28" end="34" xmlns="urn:example:reference:0">
<mention jid="room@muc.this.example/ralphm"/>
</reference>
</apply-to>
</message>A witty response mentioning @ralphm
After a bit of consideration, Kev edits his response:
<message from="room@muc.this.example/Kev" type="groupchat">
<stanza-id id="2019-09-02-13" by="room@muc.this.example"
xmlns="urn:xmpp:sid:0"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-11">
<external name='body'/>
<replace xmlns='urn:example:message-correct:1'/>
</apply-to>
<body>A slighty wittier response mentioning @ralphm</body>
</message>A slightly wittier response mentioning @ralphm
Upon receiving the correction, the client discards all fastened references. The body text was changed, so the reference indices are stale. The room can then send a new stanza marking up the new text:
<message from="room@muc.this.example"
type="groupchat">
<stanza-id xmlns="urn:xmpp:sid:0"
id="2019-09-02-14" by="room@muc.this.example"/>
<apply-to xmlns="urn:xmpp:fasten:0"
id="2019-09-02-11">
<reference begin="40" end="46" xmlns="urn:example:reference:0">
<mention jid="room@muc.this.example/ralphm"/>
</reference>
</apply-to>
</message>A slightly wittier response mentioning @ralphm
Closing notes
Fastening should also gain a way to unfasten explicitly. I think that should use the stanza ID of the stanza that included the earlier fastening. This allows for undoing individual emoji reactions.
- Unfastening should probably not use the proto-XEP on Message Retraction. That is for retracting the entire original message plus all its fastened items, and invalidating all message references pointing to it.
It might make sense to have a separate document describing how to handle stanza IDs, so that all specifications could point to it instead of each having their own algorithm. In different contexts, different IDs might be used. The other proposal for attachments, XEP-0367: Message Attaching, has a section (4.1) on this that might be taken as a start.
-
In the discussion leading up to this post, a large part was about how to handle all these things attached/fastened to messages in message archives. This is not trivial, as you likely don't want to store a sequence of stanzas, but of (original) messages. Each of those message then might have one or more things fastened to it, and upon retrieval, you want these to come along when retrieving a message. Some of these might be collated, like edits. Some might cause summary counts (emoji, simple polls) with the message itself, and require an explicit retrieval of all the reactions, e.g. when hovering the reaction counts.
Details on message archive handling is food for a later post. I do think that having a single way of attaching/fastening things to messages makes it much easier to come up with a good solution for archive handling.
I didn't provide examples for stanza encryption, but discussions on this suggested that stanzas with fastened items would have an empty
apply-to, including theidattribute, so that message archives can do rudimentary grouping of fastened items with the original message.I didn't include examples on Chat Markers, as its current semantics are that a marker sent by a recipient applies to a message and all prior messages. This means the marker isn't really tied to a single message. I think this doesn't match the model for Message Fastening.