Upcoming webhook improvements [Updated: Launched!]

[Joe_Trollo]

Hello all! I have perhaps the most exciting news I’ve ever announced on this forum—the API team at Asana is now working on a number of webhook improvements! Ever since we launched the search API last year, webhooks has been the number one feature request/source of developer pain. (In fact, even before we launched the search API it was mostly a toss-up between the two.) After lots of investigation and design, we’re planning on three major improvements that will make Asana’s webhooks easier and more powerful.

A quick note: while we are actively working on these improvements and have an internal timeline, we don’t yet have a launch date for all these features. However, due to some technical constraints, you may observe these features (in a non-breaking way) before their official launch (such as new fields being included in webhook bodies). Now, to the good stuff!

• Richer webhook events — We’re going to include more information in webhook events so that clients listening for webhooks can have a much better understanding of what actually changed in webhooks. Specifically, events will indicate what field on the object changed. No more fetching all the stories on the task to see what changed most recently—the webhook will tell you up-front. This also applies to events fetched in event streams.
• Filters on webhook events — You’ll be able to define “filters” on your webhooks so that they only deliver the events you care about. No more dealing with a constant stream of noise when you only care about assignees and custom fields changing. Webhooks can have multiple filters to permit multiple kinds of events through a single subscription.
• Webhooks with larger scopes (and new events) — Currently you can only create webhooks on tasks, projects, and tags. We’re going to expand that to include teams and workspaces so your app can learn about changes at higher levels. As a consequence of this, we’ll be introducing new events like “project created” and “user added to team.” Webhooks at larger scopes will be required to have certain filters on them, though, so you won’t be able to make high-volume subscriptions like “every change on every task in this organization.”

We hope you’re as excited about these changes as we are! We’ll be sharing more information in the coming weeks, but feel free to leave comments and questions here. Please also let us know what you think in this poll!

On a scale from 1 to 5, how excited are you about these improvements?

• 1 - I strongly dislike these changes. This is bad for the API.
• 2 - I dislike these changes. I think there are different improvements you should be making.
• 3 - I’m indifferent to these changes.
• 4 - I like these changes. They’ll have direct, positive impact on my app.
• 5 - I love these changes! I can’t wait to start using them!

[Daniel_Santiago]

I’m really looking forward for this feature. Do you have a ETA?

[Seth]

@Joe_Trollo

However , due to some technical constraints, you may observe these features (in a non-breaking way) before their official launch

Looks like the new webhook changes are breaking our integration. We expected this format as used in the documentation:

{
“resource”: 1337,
“parent”: null,
“created_at”: “2013-08-21T18:20:37.972Z”,
“user”: 1123,
“action”: “changed”,
“type”: “task”
}

And, now we’re getting this:
[
{
“user”:
{“gid”:“123”,
“resource_type”:“user”
},
“created_at”:“2019-08-27T19:19:18.879Z”,
“action”:“changed”,
“parent”:null,
“resource”:
{“gid”:“247”,
“resource_type”:“project”
}

[Phil_Seeman]

Hi @Seth,

Actually what you’re seeing is completely unrelated to this upcoming change; what you’ve run into is this change to webhook structure:

Asana is moving to string IDs [Updated with revised timeline]

Hi all, we have a new update to share. Our above plan is still accurate, and we’re aiming to let developers start migrating in early November. Notably, though, the plan is missing one crucial component: webhooks. Our deprecations framework is built into our framework for handling API requests. Webhooks, however, originate from within Asana’s API and are not made in response to any request. Client don’t send any headers, and so there’s no way to use those headers to define behavior. Because of t…

That change was announced many months ago and has been slowly rolled out, but yesterday was made live for good; see:

Asana is moving to string IDs [Updated with revised timeline]

Hello all, we have an important update to share about the timeline of this deprecation. Because this deprecation affects literally every API request and webhook, there’s a risk that we will cause a large number of heavily-used apps to encounter errors for extended periods of time if we change the default behavior in an instant on the activation date. In fact, in the last month, nearly 7,000 apps with over 230,000 users have not opted in or out of the deprecation and will almost certainly break …

You’re right, though; looks like the documentation still needs to be updated. (@Joe_Trollo…)

[Seth]

Thanks Phil. Normally, my source of truth is the documentation - and it’s unfortunately wrong.

[Matt_Bramlage]

I have a PR open for this right now - it’s in my hands, @Phil_Seeman and @Seth.

@Phil_Seeman is right, this webhook change is for string IDs and not for the improvements @Joe_Trollo is mentioning above. Sorry for the delay on this, with our internal Roadmap Week it got put on the back burner but it’s high-priority for me at the moment. Expect updates soon!

[Joe_Trollo]

Hello all! I have some updates to share about pieces of this project that have now been shipped to the API.

First up, we’ve removed some noisy events from the API. There were a number of situations where we were emitting events for changes to properties that aren’t actually visible to users, or are side-effects of other events already being emitted. For example, there is a “last story time” value on tasks in our data model, but this is never exposed as a property of tasks in the web app or API. When users add tasks to projects, we add stories on the task describing those additions, but the task itself doesn’t change. The invisible internal field was updated, however, and so we were sending “task changed” events in addition to the “story added to task” and “task added to project” events. This (1) contributed to the “noisy” volume of events we send, and (2) was especially confusing when the task is added to a private project. In that case, the story and project are invisible to you, and so the “story added” and “task added” events wouldn’t fire, but the “task changed” event would.

As a result, adding or removing tasks to projects, moving tasks between columns, adding attachments on tasks, and a few other edge cases no longer emit “task changed” events. You will, however, still see the appropriate “task added to section/project” and “attachment added to task” events.

Secondly, we’ve added a field to event stream responses that tells you whether there are more events waiting on the server to be retrieved. This new field is called has_more and, if it’s true, your app should make another request right away to retrieve the additional events. Previously, there was no limit to the number of events returned in an event stream, but as we increase the amount of data in each event, this becomes more of a risk. To prevent issues in this area, we’ve decided to cap the number of events per event stream response to 100 (the same as the maximum number of events delivered in a single webhook body). Our metrics show that these responses were almost never over 100 events anyway, so there would be no present impact on apps already using event streams.

We’ll continue to provide updates as new features land in the API, and we are closing in on the final pieces for the full launch. Thank you all for your patience! As always, feel free to ask any questions you have and we’ll respond in this thread.

[Anton_Afanasev]

I would love to have a webhook for the whole workspace…
Because for now I have to update my created webhooks using cron every 10 minutes. With feature “webhook for workspace” I won’t need to use cron to ask asana about new projects or tasks that I should create webhook for, cause once I will have 1 webhook for every user - this user will get all events that will be vissible for this user.

[Phil_Seeman]

That’s coming soon, @Anton_Afanasev - from @Joe_Trollo’s post above:

@Joe_Trollo

Webhooks with larger scopes (and new events) — Currently you can only create webhooks on tasks, projects, and tags. We’re going to expand that to include teams and workspaces so your app can learn about changes at higher levels. As a consequence of this, we’ll be introducing new events like “project created” and “user added to team.” Webhooks at larger scopes will be required to have certain filters on them, though, so you won’t be able to make high-volume subscriptions like “every change on every task in this organization.”

[Anton_Afanasev]

I’ve seen this post…
But maybe my message will help developers to understand why this feature will be useful, especially for me

[Raul_Popadineti]

Is there an ETA for this yet?

[Joe_Trollo]

Hi @Raul_Popadineti, we’re in the final stages of implementation and will begin beta testing soon. We currently estimate that these new changes will be generally available before the end of February. Thanks for your patience!

[Joe_Trollo]

Hello all, I have some news: our webhook improvements are ready for testing! We’re looking for a couple of eager developers to kick the tires on these changes and help us find any lingering bugs. We’ve already found a few that we’re working on fixing, but with how complex some pieces of our API are, we could use your help to trigger weird edge cases that we haven’t thought of, and even just to let us know when something doesn’t match your expectations. If you’d like to have early access to these improvements, read on!

Some notes about what we’re looking for and what you should expect from testing:

• Testing will be scoped to individual workspaces. Our webhooks infrastructure is a pipeline from where users change data to where we send the events to your app. In order to provide new events, two separate flags must be enabled on each end of the pipeline. This means that your test app will only receive new events from individual workspaces, i.e., your apps will not receive new-style events from outside the workspace you choose.
• We recommend that you test in a workspace that is not connected to other webhooks-powered apps. This includes your own non-testing apps! This is to make sure that new events from your testing workspace do not end up hitting apps that aren’t expecting them, such as your production apps or apps owned by other developers with early access.
• We’re primarily looking for issues around richer webhook events. The mapping between our internal data model and the API’s data model is one of the most complex pieces, and we want to make sure that the new action: "changed" events are working as expected in all cases.
• You should be willing to submit reports about bugs and the overall experience with the improvements. We’re looking not just for issues like “this event is missing” or “this event has incorrect data,” but also “this event has an unclear meaning” and “this event has an unintuitive structure.”
• Logging events that your app sees will be extremely useful. It will be hard for us to act on bugs if we can’t see the raw JSON events that your app received, so be prepared to start logging and saving webhook bodies your app receives from Asana. We’ll want to see them in any bug reports you submit.

If these points haven’t deterred you, head over to this form to request early access to these improvements. Note that we may not be able to admit all applicants if the volume of requests is too high. Update: Now that we are rolling out, we are no longer accepting beta testers. If you want to submit a bug report or any feedback, please fill out this form.

[Joe_Trollo]

Hello again! I’m happy to announce that we’re ready to begin rolling out these improvements! Over the next week or so you’ll start seeing changes in webhooks, starting with richer events. After that we’ll be rolling out new event types, and then finally the ability to create webhooks on teams and workspaces. Please let us know if you encounter anything unexpected during this time, so that we can address it quickly!

Updates to our documentation will be following shortly.

[Nicolas_Benais1]

Hi Joe, do we have any documentation around this new webhook configuration ?
Thanks

[Joe_Trollo]

Hi @Nicolas_Benais1, our developer relations team is still working on the new documentation and should be publishing it soon.

[Jeff_Schneider]

Hi @Nicolas_Benais1, the webhooks docs were updated last week. Thank you for your patience!

[Gijs]

Hi! Thanks for the improvements. Is it supposed to work already? I can not get it to work. No matter what I try, I always get a succesfully created response, but without the filters.
example:
{
“data”: {
“resource”: <workspace_id>,
“target”: ,
“filters”: [
{
“resource_type”: “task”,
“resource_subtype”: “default_task”,
“action”: “changed”,
“fields”: [
“completed_at”
]
}
]
}
}

Am I missing something?

Thanks!