Asana is moving to string IDs [Updated with revised timeline]

Hello,

When is this going to happen exactly?

As stated in the original post, we are not committing to a set of dates for this migration yet. We will send out more communication when we have a final timeline. As an estimate, we are working on this now and hope to be ready to open the migration window within three months.

3 Likes

I find it hard to believe that 9 007 199 254 740 991 is not enough primary keys.

I agree that GUIDs are superior because of the ability to merge data from different sources and not have to worry about updating keys but 9,007 trillion records ought to be enough!

1 Like

@gregg, yes, it is an astonishingly large number of IDs to burn through! There are three main factors for this:

  • IDs in Asana are globally unique, across all workspaces and data types. If a task has ID 1, then there cannot be a project with ID 1, or a team with ID 1, or even a task in a different workspace with ID 1.
  • Rather than have every ID assigned to an object upon creation, the web app running in your browser will reserve a block of IDs to use during your session. Those IDs are permanently reserved, however, so they are not released for other clients to use when you close Asana. This means that a large percentage of our IDs go unused, drastically increasing the rate at which we consume them.
  • When looking at the total number of objects in Asana, we’re observing consistent exponential growth, which makes it surprisingly easy to reach 253.
2 Likes

Wow, so interesting! Thank you for the explanation.

2 Likes

Hi all, we have a new update to share. Our above plan is still accurate, and we’re aiming to let developers start migrating in early November. Notably, though, the plan is missing one crucial component: webhooks.

Our deprecations framework is built into our framework for handling API requests. Webhooks, however, originate from within Asana’s API and are not made in response to any request. Client don’t send any headers, and so there’s no way to use those headers to define behavior. Because of this, we’re adding new settings to webhooks and OAuth apps/PATs directly. Here’s how these settings will work:

To test out string IDs in a particular webhook, you will be able to set a boolean flag on the webhook itself, either when it’s created or in a follow-up request to PUT /webhooks/<webhook-id>. When this flag is set to true, the webhook bodies will be sent with string IDs. When it’s false, it falls back to the settings on the OAuth app/PAT.

Once you’re confident that your app can handle webhooks with string IDs, you can toggle an app-level option to decide the global behavior for all webhooks (so that you don’t need to manually update each and every one). This option will have three states: “default,” “disabled,” and “enabled,” which roughly correspond to “not setting any header,” “setting an Asana-Disable header” and “setting an Asana-Enable header” respectively.

To clarify the various configurations, here’s a table that enumerates them all:

App setting Webhook setting Behavior
Default False numeric IDs before activation,
string IDs after activation
Default True string IDs
Disabled False numeric IDs
Disabled True string IDs
Enabled False string IDs
Enabled True string IDs

Now, the current webhook event structures are kinda weird and don’t fit with the rest of the API. In particular, there are no id fields that we could replace with gid. Instead, they look like this:

{
  "resource": 1337,
  "parent": null,
  "created_at": "2013-08-21T18:20:37.972Z",
  "user": 1123,
  "action": "changed",
  "type": "task"
}

We’re going to take advantage of this deprecation to make them match the rest of the API. When string IDs are enabled, these events will match our events API and look like this:

{
  "resource": {
    "gid": "1337",
    "resource_type": "task"
  },
  "parent": null,
  "created_at": "2013-08-21T18:20:37.972Z",
  "user": {
    "gid": "1123",
    "resource_type": "user"
  },
  "action": "changed"
}

While seemingly benign, this structural change actually opens the door for a massive improvement to our API that I’ll talk about in a later announcement, so stay tuned! And as always, please ask any questions you have in this thread so that we can ensure this process is clear for all developers. Thank you for your patience!

I’m not clear on what the flag should be called that we send. Maybe you can show an example of a webhook call that includes it?

The flag will be called always_use_string_ids so you’d update an existing webhook with something like:

curl -X PUT https://app.asana.com/api/1.0/webhooks/<webhook-id> \
  -d "always_use_string_ids=true"
1 Like

Hello, I’ve been using the Python API client library and I’m starting to plan for this change. I have made a few changes to my code in anticipation. However, I don’t seem to find a way to set the Default/Enabled/Disabled flags using the Python library. Can someone please point me in the right direction?

Did you try to deep dive into the library code already?

Yes, since I’d like to set it at global level, I believe it goes in one of the several options with the Client class (https://github.com/Asana/python-asana/blob/master/asana/client.py)? But I’m not 100% where exactly and what the option name and parameter should look like. Perhaps an example that shows how to turn the integer IDs on and off will help other forum members as well.

I don’t have any example, you should inspect somehow the requests that goes out and see if the header you added matches what the documentation says. @Diakoptis maybe you can help?

Hey @Pradyumna_Dhakal,

I don’t write in python but at the client at official library you can set headers for every request type and add your options to disable or enable things in Asana api.

def post(self, path, data, **options):
    """Parses POST request options and dispatches a request."""
    parameter_options = self._parse_parameter_options(options)
    body = {
        # values in the data body takes precendence
        'data': _merge(parameter_options, data),
        'options': self._parse_api_options(options)
    }
    headers = _merge(
        {'content-type': 'application/json'},
        options.pop('headers', {})
    )
    return self.request('post', path, data=body, headers=headers, **options

Sorry for the short response but I’m on mobile.

Hi @Pradyumna_Dhakal,

You can see examples of how to set headers in the client libraries in the tests for the client. You can either set them using client.headers["key"] = "value" to send them with every request, or pass headers={"key": "value"} to method calls to send them on an individual request.

For string IDs, you would set client.headers["Asana-Enable"] = "string_ids", however there is still one gap in our implementation and we are not 100% ready to begin this migration.

Thanks all for your prompt responses!

@Joe_Trollo, that makes it very clear. I’m assuming the settings apply only for that session and for a new session, if I don’t send in the header, the behavior will fallback to default, i.e. receiving both integer and string IDs. Also, would you advice we wait till your implementation is 100% before performing any further tests?

If you set the headers on the client directly, they will apply to every request made with that client. If you pass them in the method call, they will apply only to that one request. Clearing the header from the client or not including them in the method call will then fall back to the default behavior, yes.

To be more explicit about the gap in our implementation: we have 100% coverage on read and write requests in the API (enabling this feature will hide numeric IDs on reads and reject numeric IDs on writes) but do not yet have support for requests that originate from within Asana (i.e., outgoing webhook payloads). If your app does not use webhooks, you can freely test the rest of the API, but we won’t be starting the clock for the deprecation until the entire API supports string IDs.

Got it. I’ve successfully tested switching the integer ids on an off - thanks!

Hi everyone! After a disappointingly large technical hurdle, we’re finally able to begin this deprecation! To recap:

  • This deprecation will operate under the name "string_ids" and can be enabled by sending an Asana-Enable: string_ids header in your requests
  • When this deprecation is enabled,
    • The API will not provide a numerical "id" field for any object in the response, and you will only receive the string "gid" field
    • The API will not accept numerical IDs in any part of the request
      • URLs and paths are already always strings and do not need to be changed
      • Form-encoded key-value pairs don’t have type information and are already always strings, so they do not need to be changed
      • JSON bodies do have type information, and you’ll need to change the fields in your request bodies from things like "projects": [123, 456] to "projects": ["123", "456"]
  • There are new settings to control the format of webhook bodies
    • The webhook objects themselves have an "always_use_string_ids" field on them which you can change to migrate individual webhooks into the new format
    • Each app in your developer console will have options for migrating the entire app as a whole (screenshots below)
    • Refer to the table in my previous post for specifics on how these two settings interact

Here’s what the app option will look like in the developer console:

On the matter of the timeline:

  • You can begin the deprecation starting today. The new options in the developer console will be gradually rolling out over the next week or so.
  • The “activation date” (the day on which string IDs become the default behavior) will be August 13th, 2019
  • The “end date” (the day on which string IDs become permanent) will be February 11th, 2020

As always, let us know if you have any questions about this deprecation. Because of the scope of this deprecation, we’ll be monitoring the situation closely as developers migrate their apps.

6 Likes