Introduction

This document outlines the set of changes to Tweets so that people can express even more in 140 characters: 1) to allow for richer public conversations that are easier to follow on Twitter and 2) to ensure people can attach media to Tweets without sacrificing the characters they have to express themselves.

These changes touch many aspects of the Twitter platform. To that end, we have prepared these technical materials to help you transition your products and applications to the new Tweet format. The following sections will step through the planned technical changes.

Recent revisions:

Overview

The following chart and sample JSON demonstrate the differences between Tweet JSON objects in the various API endpoints (REST, streaming and Gnip), once the API changes described above are active. (If the formatting is difficult to read, please see the documentation and samples on our corresponding Github repo). Additional samples are available on Github and also via the Gnip support documentation.

Mode Objective Availability Details Example Tweets (JSON)

Compatibility

Tweet payload will work with all existing integrations, regardless of content of Tweet

Default for all REST APIs

  • text field is truncated to 140, as needed
  • truncated field is true
  • Any entities only include those in the available 140 text range

Compatibility with additional extended_tweet in payload

Maintain backwards compatibility with a non-breaking, payload addition

Default for all Streaming and Gnip APIs

  • Includes new extended_tweet field in payload, containing:
    • full_text
    • display_text_range
    • entities
    • extended_entities

Extended

Tweet payload contains all information to render Tweets that contain more than 140 characters.

REST APIs only: add the below parameters to any endpoint:

tweet_mode=extended

  • full_text replaces text
  • truncated field is false
  • display_text_range delineates different sections of body for mentions/tweet/entities

What is changing?

We are simplifying the way that replies work on Twitter by moving some of the "scaffolding" of Tweets into display elements so that they no longer count towards the character limit within the Tweet.

This change will introduce new limits around the numbers of specific elements that may be included as part of a Tweet (specifically, mentions).

These changes are shipping in the coming months. Our goal is to give developers and partners this advance notice of changes to the format of Tweets so that they can prepare their products and applications appropriately.

Compatibility, and what this means for developers

Backward and forward compatibility for third party clients and other API users are our primary considerations.

There are a number of areas that will be impacted by the change:

Tweet object changes

The following things will change within Tweet payloads:

What does this look like?

This diagram shows the high-level change to Tweets, and the elements that will be hidden in the user interface.

When rendered in apps or on the web, the hidden @mentions shall appear outside of the visible Tweet body, in a format similar to below. When a Tweet is in reply to multiple people, the name of the person whom the author is directly replying to should be prioritized.

Terminology

Classic Tweet - A Tweet object where the total length of the text content does not exceed 140 characters. It may or may not contain leading and/or trailing text that shall be hidden by newer clients.

Extended Tweet - A Tweet object which includes hidden entities (e.g. leading @mentions and trailing attachment) and where the text content exceeds 140 characters in length. The display text region shall not exceed 140 characters.

Rendering modes

There will be two modes for rendering Tweet JSON objects to API clients: compatibility mode and extended mode. Compatibility mode is the default mode for the public REST and Streaming APIs and Gnip products, and is designed to not break existing clients.

REST API clients may opt into the extended mode via a request parameter.

In the future, an additional announcement will be made when the time is right to make a change for the rendering mode to default to extended mode.

Compatibility Mode JSON Rendering

In compatibility mode, Classic Tweets will be rendered exactly as today.

For Extended Tweets in compatibility mode, the following will be true:

  1. The existing text field will contain a truncated version of the Tweet text, followed by an ellipsis character, a space, and a shortened self-permalink URL. The total length of this text value shall not exceed 140 characters.
  2. The existing truncated field will be set to true.
  3. The existing entity fields (mentions, urls, media, etc.), will only contain entities that are fully contained within the text value. The from and to indices within each entity must be a valid code point index within the text value. The truncation point will avoid truncating mid-entity. A URL entity for the appended self-permalink will be appended to the list of entities.
  4. The payload may contain a new dictionary field named extended_tweet (this is specific to the Streaming and Gnip APIs). This will contain the following sub-fields:
  5. If the Tweet contains a Quote Tweet permalink URL, then the resulting embedded Quoted Tweet, if any, will still be included even if the permalink URL is not included in the truncated text.
  6. If the Tweet contains a URL entity that results in an attached card, then the card will still be included even if the original URL entity is not included in the truncated text.
  7. Since native media is only represented via entities, those will be missing from the truncated list of entities, but will be in extended_tweet.entities.

Extended Mode JSON Rendering

In extended mode, the following will be true both for Classic Tweets and Extended Tweets:

  1. The text field is no longer included; instead, the payload will contain a field named full_text, which contains the entire untruncated Tweet text.
  2. The payload shall contain a field named display_text_range, which is an array of two unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the tweet.
  3. The truncated field will be set to false.
  4. The entity fields will contain all entities, both hidden and displayable.

Limits

There will be restrictions placed on the content of the text. This is to improve the end-user experience, and to encourage high quality content. Tweets will be rejected at creation time if they exceed the new entity limits, via new API error codes. These restrictions will be enforced on all Tweets, regardless of overall character count (this represents a change to the existing methods that support creating new Tweets).

The numbers listed below are intended as initial guidelines.

API changes

Public REST API endpoints

Composition

The REST API endpoints that create new Tweets (statuses/update) will accept a new boolean parameter when a Tweet is sent as a reply to a conversation: auto_populate_reply_metadata (true to enable, false to disable, false being the default). The existing in_reply_to_status_id must also be set. The leading @mentions will subsequently be looked up from the original Tweet, and added to the new Tweet from there. In cases where the original Tweet has been deleted, the reply will fail. This is a change to existing behaviour, where it has been possible to reply to a deleted Tweet ID.

For older clients that are not updated for the auto_populate_reply_metadata option, mentions will continue to be included in the body of the Tweet and the server will decide on how to render the new Tweet.

The auto_populate_reply_metadata option will append @mentions into the metadata as a reply chain grows, until the limit on @mentions is reached. In order to edit down the list of handles, an additional option, exclude_reply_user_ids, will enable specific IDs (apart from the leading one) to be excluded from a reply. This parameter is an optional, comma-separated list of user ids which will be removed from the server-generated @-mentions prefix.

Note that the leading @mention cannot be removed as it would break the in-reply-to-tweet-id semantics. Attempting to remove it will be silently ignored.

In order for a URL to not be counted in the body of the Tweet, a new attachment_url parameter will be available on statuses/update to allow a client to attach it to the Tweet without explicitly adding it to the Tweet text. This URL must be a Tweet permalink, or DM deep link. Arbitrary, non-Twitter URLs should remain in the Tweet text and will count against the 140 character limit. URLs passed to the attachment_url parameter not matching either a Tweet permalink or DM deep link will fail at Tweet creation and cause an exception.

Consumption

Any REST API endpoints that return Tweets will accept a new tweet_mode request parameter.

Valid request values are compat and extended, which give compatibility mode and extended mode, respectively.

The default mode (if no parameter is provided) is compatibility mode, to support older clients and display methods.

Tweets rendered in compatibility mode via the public REST API will not contain the extended_tweet field. REST API clients that wish to get the full text can instead opt into extended mode.

Error codes

Due to the limitations listed above, new API response and error codes will be introduced. These reflect the new content requirements listed in the Limits section above.

44 attachment_url parameter is invalid. Corresponds with HTTP 400. The URL value provided is not a URL that can be attached to this Tweet.
385 You attempted to reply to a tweet that is deleted or not visible to you. Corresponds with HTTP 403. A reply can only be sent with reference to an existing public Tweet.
386 The Tweet exceeds the number of allowed attachment types. Corresponds with HTTP 403. A Tweet is limited to a single attachment resource (media, Quote Tweet, etc.)

Public Streams

The Streaming API does not provide the same ability to provide query parameters to configure request options. Therefore, the Streaming API will render all Tweets in compatibility mode at this time.

Tweets rendered in compatibility mode for the streaming APIs, unlike for the REST APIs, will include the extended_tweet field for any extended tweet. This is necessary to avoid breaking existing clients by sending text that is longer than they expect in the existing text field, and also to provide the entirety of the data in a single stream. If there is an extended_tweet field, it will also include the ranges described above.

Streaming API consumers should update their code to first check for the presence of the extended_tweet dictionary, and use that in preference to the truncated data as is applicable for their use case. When extended_tweet is not present, they must fall back to using the existing fields.

In the future, a date for a switchover to extended mode will be announced, after which time apps should be able to process the newer Tweet payloads.

Gnip (REST and Streaming APIs)

In the case of Data products, both the REST and Streaming endpoints will follow a similar pattern to the public Streaming API, and the current versions of the data products APIs will render Tweets in compatibility mode, with the extended_tweet field.

The impact is intended to be a minimal, additive and opt-in, non-breaking change. Gnip customers will have to make a code change to "opt-in" to utilize the new additive fields when present. They may also want to prepare for the impacts of increased payload sizes, including storage and bandwidth implications.

In addition to payload changes, upon release of new Tweet payloads, Gnip operators and enrichments will begin to analyze the longer text and entities as opposed to the truncated version.

Tweet display on Web

Web

Twitter's web embed products are powered by our widgets.js JavaScript library which will be automatically updated to support new Tweet display formats without additional configuration needed from publishers.

Tweet Web Intent

We currently have no planned changes to the Tweet web intent.