Azure Data Factory Asana Inline Link Service Issue

English Forum Developers & API
1

Hello,

I met an issue when I used Azure Data Factory Asana link service as source in data flow to get Project data, it can get project data with a predefined looks schema correctly if I put tasks data in UI, but I don’t know at sometime, it has below cast type error.
I have done some checks, it looks like this error happened when I created portfolio including the project in the UI, and the error continues even I deleted the portfolio.

Could you help check what caused the error, if it can be fixed or I can only use Rest API for getting project/task data accurately for now in ADF.

Thanks!

at Source ‘source1’: cannot resolve ‘Project_Custom_Fields’ due to data type mismatch: cannot cast array<struct<created_by:string,display_value:string,enabled:boolean,enum_options:array<structcolor:string,enabled:boolean,gid:string,name:string,resource_type:string>,enum_value:structcolor:string,enabled:boolean,gid:string,name:string,resource_type:string,gid:string,name:string,resource_subtype:string,resource_type:string,type:string,description:string,is_formula_field:string,is_value_read_only:string>> to array<struct<created_by:string,display_value:string,enabled:boolean,enum_options:array<structcolor:string,enabled:boolean,gid:bigint,name:string,resource_type:string>,enum_value:structcolor:string,enabled:boolean,gid:bigint,name:string,resource_type:string,gid:bigint,name:string,resource_subtype:string,resource_type:string,type:string>>;
'Project [Tags#240, Custom_Fields#276, Task_Assignee_Name#149, Task_Assignee_Id#312L, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#348L, Task_Section_Id#384L, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#420L, Project_Color#170, … 11 more fields]
± Project [Tags#240, Custom_Fields#276, Task_Assignee_Name#149, Task_Assignee_Id#312L, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#348L, Task_Section_Id#384L, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, cast(Project_Id#169 as bigint) AS Project_Id#420L, Project_Color#170, … 11 more fields]
± Project [Tags#240, Custom_Fields#276, Task_Assignee_Name#149, Task_Assignee_Id#312L, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#348L, cast(Task_Section_Id#159 as bigint) AS Task_Section_Id#384L, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#169, Project_Color#170, … 11 more fields]
± Project [Tags#240, Custom_Fields#276, Task_Assignee_Name#149, Task_Assignee_Id#312L, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, cast(Task_Id#158 as bigint) AS Task_Id#348L, Task_Section_Id#159, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#169, Project_Color#170, … 11 more fields]
± Project [Tags#240, Custom_Fields#276, Task_Assignee_Name#149, cast(Task_Assignee_Id#150 as bigint) AS Task_Assignee_Id#312L, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#158, Task_Section_Id#159, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#169, Project_Color#170, … 11 more fields]
± Project [Tags#240, cast(Custom_Fields#148 as array<struct<display_value:string,gid:bigint,name:string>>) AS Custom_Fields#276, Task_Assignee_Name#149, Task_Assignee_Id#150, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#158, Task_Section_Id#159, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#169, Project_Color#170, … 11 more fields]
± Project [cast(Tags#147 as array<structgid:bigint,name:string>) AS Tags#240, Custom_Fields#148, Task_Assignee_Name#149, Task_Assignee_Id#150, Task_Assignee_Email#151, Task_Completed#152, Task_Completed_At#153, Task_Created_At#154, Task_Due_At#155, Task_Start_On#156, Task_Type#157, Task_Id#158, Task_Section_Id#159, Task_Section_Name#160, Task_Modified_At#161, Task_Name#162, Task_Number_Of_Likes#163L, Project_Current_Status_Color#164, Project_Current_Status_Text#165, Project_Current_Posted_On#166, Project_Current_Posted_By#167, Project_Data_Archived#168, Project_Id#169, Project_Color#170, … 11 more fields]
± Project [tags#21 AS Tags#147, custom_fields#16 AS Custom_Fields#148, assignee#17.name AS Task_Assignee_Name#149, assignee#17.gid AS Task_Assignee_Id#150, assignee#17.email AS Task_Assignee_Email#151, completed#24 AS Task_Completed#152, completed_at#19 AS Task_Completed_At#153, created_at#27 AS Task_Created_At#154, due_on#26 AS Task_Due_At#155, start_on#20 AS Task_Start_On#156, resource_subtype#30 AS Task_Type#157, gid#18 AS Task_Id#158, memberships#112.section.gid AS Task_Section_Id#159, memberships#112.section.name AS Task_Section_Name#160, modified_at#25 AS Task_Modified_At#161, name#15 AS Task_Name#162, num_likes#22L AS Task_Number_Of_Likes#163L, projects#113.current_status.color AS Project_Current_Status_Color#164, projects#113.current_status.text AS Project_Current_Status_Text#165, projects#113.current_status.created_at AS Project_Current_Posted_On#166, projects#113.current_status.created_by.name AS Project_Current_Posted_By#167, projects#113.archived AS Project_Data_Archived#168, projects#113.gid AS Project_Id#169, projects#113.color AS Project_Color#170, … 11 more fields]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, resource_subtype#30, memberships#112, projects#113]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, resource_subtype#30, col#80, col#80.memberships AS memberships#112, col#80.projects AS projects#113]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, resource_subtype#30, col#80]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, resource_subtype#30, combined#47, col#80]
± Generate explode(combined#47), true, [col#80]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, resource_subtype#30, combined#47]
± Project [name#15, custom_fields#16, assignee#17, gid#18, completed_at#19, start_on#20, tags#21, num_likes#22L, workspace#23, completed#24, modified_at#25, due_on#26, created_at#27, projects#28, memberships#29, resource_subtype#30, arrays_zip(memberships#29, projects#28, memberships, projects) AS combined#47]
± Project [col#13.name AS name#15, col#13.custom_fields AS custom_fields#16, col#13.assignee AS assignee#17, col#13.gid AS gid#18, col#13.completed_at AS completed_at#19, col#13.start_on AS start_on#20, col#13.tags AS tags#21, col#13.num_likes AS num_likes#22L, col#13.workspace AS workspace#23, col#13.completed AS completed#24, col#13.modified_at AS modified_at#25, col#13.due_on AS due_on#26, col#13.created_at AS created_at#27, col#13.projects AS projects#28, col#13.memberships AS memberships#29, col#13.resource_subtype AS resource_subtype#30]
± Project [col#13]
± Generate explode(body#9.data), false, [col#13]
± Relation[headers#8,body#9] RestRelation

image

2

Hello, @Lei_Zhong

It seems you’re encountering a data type mismatch error in Azure Data Factory when using the Asana link service as a source in your data flow. The error message indicates that there’s an issue casting the Project_Custom_Fields from one complex data type to another, specifically with the gid field, which is expected to be a bigint but is provided as a string.

This type of error can occur when there’s a schema change in the source data or when the data flow expects a different schema than what is being provided. Here are a few steps you can take to troubleshoot and potentially resolve this issue:

Check the Schema: Verify that the schema defined in your data flow matches the actual schema of the data coming from Asana. Ensure that the gid fields are consistently defined as either string or bigint across your entire data flow.
Use Auto-Mapping: If you’re manually mapping fields, consider using auto-mapping to avoid type mismatches. Auto-mapping will automatically align the source and sink data types if they are compatible.
Modify the Data Flow: If certain fields are causing issues and are not essential for your data processing, you might consider excluding them from the data flow to see if that resolves the error.
Cast Transformation: Use the Cast transformation in your data flow to explicitly define the data types of individual columns. This can help prevent type mismatch errors by ensuring that each field is cast to the correct data type before further processing.
Derived Column Transformation: If the Cast transformation doesn’t support complex data type casting, you can use a Derived Column transformation instead. This allows you to handle type conversion errors explicitly and can be used to cast complex data types.

Best Regard,
ryan1969