Asana is moving to string IDs [Updated with revised timeline]

[Joe_Trollo]

Hi everyone! Unfortunately this is one of my announcements that doesn’t include a new feature launch, sorry. However, this one is much more important to read, as it is guaranteed to cause your code to break.

Asana currently uses numerical IDs for all references, but we are running out of them. Once we surpass 2⁵³, JavaScript loses precision and is no longer able to accurately represent every ID. You can observe this by opening up a JavaScript console and seeing that 2**53 + 1 == 2**53 evaluates to true. Because of this, we must move away from numerical IDs before we reach that point, and by consequence that means our developers also need to move away from numerical IDs.

The API team is actively working on this migration both within our internal code and in the public API’s interface. Because of the scope and complexity of this change internally, you will start to see some of these changes leak (in backwards compatible ways) before we’re ready for you to move to string IDs yourself. However, we want to share the plan with you to provide clarity and minimize confusion during this change.

The preparation

Starting as early as this week, you will start to see objects containing two ID fields: the current, numerical id field and a new, string gid field. Additionally, places where the API accepts IDs will begin to accept both numbers and strings.

If you store Asana IDs in a service outside Asana (such as an SQL table), please start planning how you will migrate IDs in these services to become strings. For example, if you are using an SQL table, you will want to change from a type like BIGINT to a type like VARCHAR.

The change

Once we have comprehensively confirmed that the API is fully able to support string IDs, we will be using our deprecations framework for the migration. You’ll be able to send a flag in your request headers that enables the new behavior. This consists of two aspects:

1. When enabled, objects will not contain an id field. They will contain only the gid field as their ID.
2. When enabled, the API will reject API requests that use numbers instead of strings to reference objects.

After we have opened this up with the deprecations framework, the old behavior will be the default for about six months. Then, the new behavior will become the default and there will be another six months during which you can select the old behavior. After that, the API will operate with the new behavior fully and the old behavior will not be available. (Please refer to our deprecations documentation for more information about this process.) At this time, we cannot commit to a particular set of dates.

The future

After the migration is complete, IDs will only ever be strings. Additionally, that is the only guarantee that Asana will make about IDs. We make no guarantee that IDs will simply be the string representation of integers—newly issued IDs may have letters in them as well. Therefore, you must transition your format to handle strings.

Please let us know if you have any questions about string IDs, and we’ll answer them in this thread. We will be sending out more communication over the coming months as we get closer to completion of the work on our end and the finalization of a timeline.

FAQ:

What does gid mean?

This stands for “global ID.” In Asana, IDs are globally unique, across all workspaces and even across all data types. gid feels like a logical alternative to id and also conveys more information about Asana’s schema.

My code was already using strings. Why not just make id a string instead of removing it?

While we will do our best to make sure that all developers hear about this change in some form, the truth is that some will still fall through the cracks. There are also several libraries and frameworks out there that will silently coerce strings to numbers where asked, and if we left in the id field these applications might never notice the change.

If we remove this field, we force every developer to react during this migration period, during which they still have a chance to choose between old and new behavior and can quickly recover if their app breaks. If we don’t remove this field, it’s possible for a developer not to notice the change during the migration period. Then, after the migration period is over and the new behavior is cemented, we could send an ID with letters in it that would break their app and there would be no safe recovery.

What about existing IDs?

Any ID that was already issued by Asana will continue to be the only ID associated with that object and will not change with this migration. If you have an ID, the only transformation you need to apply is turning it into a string.

How long will these new IDs be?

Asana’s new string IDs will not exceed 31 characters in length, If you are using an SQL table, this means that you can use VARCHAR(31) as the column type for IDs.

[Diakoptis]

Hey joe,

If we already handle the ids like string is necessary to switch on getting the gid field or we can keep getting the id field as we already do?

[Joe_Trollo]

We will be removing the id field so that, if a developer has missed communication and not migrated their app, it will break during the migration window when we can support both old and new behavior. If we did not remove this field as part of the migration, some apps might not break until they start seeing letters in the IDs, at which point we have already fully committed to the new behavior. If that happens, developers won’t have the benefit of our deprecations framework to return to the old behavior while they migrate their app.

[Bastien_Siebman]

Thanks for the detailed explanation. That is going to break a lot of apps not reading official communications

[carolina]

Hello,

When is this going to happen exactly?

[Joe_Trollo]

As stated in the original post, we are not committing to a set of dates for this migration yet. We will send out more communication when we have a final timeline. As an estimate, we are working on this now and hope to be ready to open the migration window within three months.

[gregg]

I find it hard to believe that 9 007 199 254 740 991 is not enough primary keys.

I agree that GUIDs are superior because of the ability to merge data from different sources and not have to worry about updating keys but 9,007 trillion records ought to be enough!

[Joe_Trollo]

@gregg, yes, it is an astonishingly large number of IDs to burn through! There are three main factors for this:

• IDs in Asana are globally unique, across all workspaces and data types. If a task has ID 1, then there cannot be a project with ID 1, or a team with ID 1, or even a task in a different workspace with ID 1.
• Rather than have every ID assigned to an object upon creation, the web app running in your browser will reserve a block of IDs to use during your session. Those IDs are permanently reserved, however, so they are not released for other clients to use when you close Asana. This means that a large percentage of our IDs go unused, drastically increasing the rate at which we consume them.
• When looking at the total number of objects in Asana, we’re observing consistent exponential growth, which makes it surprisingly easy to reach 2⁵³.

[gregg]

Wow, so interesting! Thank you for the explanation.

[Joe_Trollo]

Hi all, we have a new update to share. Our above plan is still accurate, and we’re aiming to let developers start migrating in early November. Notably, though, the plan is missing one crucial component: webhooks.

Our deprecations framework is built into our framework for handling API requests. Webhooks, however, originate from within Asana’s API and are not made in response to any request. Client don’t send any headers, and so there’s no way to use those headers to define behavior. Because of this, we’re adding new settings to webhooks and OAuth apps/PATs directly. Here’s how these settings will work:

To test out string IDs in a particular webhook, you will be able to set a boolean flag on the webhook itself, either when it’s created or in a follow-up request to PUT /webhooks/<webhook-id>. When this flag is set to true, the webhook bodies will be sent with string IDs. When it’s false, it falls back to the settings on the OAuth app/PAT.

Once you’re confident that your app can handle webhooks with string IDs, you can toggle an app-level option to decide the global behavior for all webhooks (so that you don’t need to manually update each and every one). This option will have three states: “default,” “disabled,” and “enabled,” which roughly correspond to “not setting any header,” “setting an Asana-Disable header” and “setting an Asana-Enable header” respectively.

To clarify the various configurations, here’s a table that enumerates them all:

App setting	Webhook setting	Behavior
Default	False	numeric IDs before activation, string IDs after activation
Default	True	string IDs
Disabled	False	numeric IDs
Disabled	True	string IDs
Enabled	False	string IDs
Enabled	True	string IDs

Now, the current webhook event structures are kinda weird and don’t fit with the rest of the API. In particular, there are no id fields that we could replace with gid. Instead, they look like this:

{
  "resource": 1337,
  "parent": null,
  "created_at": "2013-08-21T18:20:37.972Z",
  "user": 1123,
  "action": "changed",
  "type": "task"
}

We’re going to take advantage of this deprecation to make them match the rest of the API. When string IDs are enabled, these events will match our events API and look like this:

{
  "resource": {
    "gid": "1337",
    "resource_type": "task"
  },
  "parent": null,
  "created_at": "2013-08-21T18:20:37.972Z",
  "user": {
    "gid": "1123",
    "resource_type": "user"
  },
  "action": "changed"
}

While seemingly benign, this structural change actually opens the door for a massive improvement to our API that I’ll talk about in a later announcement, so stay tuned! And as always, please ask any questions you have in this thread so that we can ensure this process is clear for all developers. Thank you for your patience!

[Phil_Seeman]

I’m not clear on what the flag should be called that we send. Maybe you can show an example of a webhook call that includes it?

[Joe_Trollo]

The flag will be called always_use_string_ids so you’d update an existing webhook with something like:

curl -X PUT https://app.asana.com/api/1.0/webhooks/<webhook-id> \
  -d "always_use_string_ids=true"

[Pradyumna_Dhakal]

Hello, I’ve been using the Python API client library and I’m starting to plan for this change. I have made a few changes to my code in anticipation. However, I don’t seem to find a way to set the Default/Enabled/Disabled flags using the Python library. Can someone please point me in the right direction?

[Bastien_Siebman]

Did you try to deep dive into the library code already?

[Pradyumna_Dhakal]

Yes, since I’d like to set it at global level, I believe it goes in one of the several options with the Client class (https://github.com/Asana/python-asana/blob/master/asana/client.py)? But I’m not 100% where exactly and what the option name and parameter should look like. Perhaps an example that shows how to turn the integer IDs on and off will help other forum members as well.

[Bastien_Siebman]

I don’t have any example, you should inspect somehow the requests that goes out and see if the header you added matches what the documentation says. @Diakoptis maybe you can help?

[Diakoptis]

Hey @Pradyumna_Dhakal,

I don’t write in python but at the client at official library you can set headers for every request type and add your options to disable or enable things in Asana api.

def post(self, path, data, **options):
    """Parses POST request options and dispatches a request."""
    parameter_options = self._parse_parameter_options(options)
    body = {
        # values in the data body takes precendence
        'data': _merge(parameter_options, data),
        'options': self._parse_api_options(options)
    }
    headers = _merge(
        {'content-type': 'application/json'},
        options.pop('headers', {})
    )
    return self.request('post', path, data=body, headers=headers, **options

Sorry for the short response but I’m on mobile.

[Joe_Trollo]

Hi @Pradyumna_Dhakal,

You can see examples of how to set headers in the client libraries in the tests for the client. You can either set them using client.headers["key"] = "value" to send them with every request, or pass headers={"key": "value"} to method calls to send them on an individual request.

For string IDs, you would set client.headers["Asana-Enable"] = "string_ids", however there is still one gap in our implementation and we are not 100% ready to begin this migration.