Data Ingestion

This API is used to ingest data into PangoCDP

This API is used to ingest data into PangoCDP. The use of this API requires Authentication

API	https://svc.mydatalakes.com/dhub-i/api/v1.0/ingest
Method	POST
Request Parameters	initData: [true, false]: Is used to determine the data in init stage or not (default=false)
payload	{ "objectType":"OBJECT_TYPE", "source":"SOURCE_NAME", "jeConfigId":null, "jeEventKey":null, "entries":[ { ENTITY_OBJECT } ], "ignoredUpdateCfs":[ "customField01", "customFieldLong03" ] }
Headers	Content-type: application/json TOKEN: token API_KEY: api_key

Headers

Name	Type	Required	Valid value / Description
Content-type	String	Yes	application/json
TOKEN	String	Yes	The token is created by calling the Authentication API
API_KEY	string	Yes	Api Key belongs to username

Request

Name	Type	Required	Valid value / Description	Example
objectType	String	Yes	Object Type. Valid value: OBJECT_TYPE	shopping-cart
source	String		The source where your data comes from.	CRM
jeConfigId	String		Journey Event Configuration ID. Once you provide this value, Ingestion API will generate new Event data based on your given JE Id.
jeEventKey	String		Journey Event Configuration Key. Once you provide this value, Ingestion API will generate new Event data based on your given JE KEY.
entries	Array	Yes	List of Objects following Object Type. Max: 100 records / 1 request Limit: [1- 50] List of fields of each object type: ENTITY_OBJECT	[]
ignoredUpdateCfs	Array		List of custom fields that you don’t want to update to existing records. Only apply to existing records. This value is applied to ENTITY_OBJECT that has the tag: “SUPPORT_IGNORED_UPDATE_CFS”	[“customField01”]

Response

Type	Code	Example
Error	400	//ObjectType, Entity is missing { "success": false, "errorCode": "MISSING_INFO", "message": "Invalid value of: objectType" } //Entity is empty { "success": false, "requestId": "{REQUEST_ID}", "initData": false, "validRecords": 1, "invalidItems": 2, "errorCode": "MISSING_INFO", "invalidIds": [“Input_Id", “Input_Id2"], "message": "Invalid value of: entities, entity size [1-50]" } //Entity data is invalid { "entity0": "Invalid value of: orderId", "entity1": "Invalid value of: orderStatus", "success": false, "errorCode": "MISSING_INFO", "message": "Some entities are invalid." }
Error	401	{ "success": false, "errorCode": "UNAUTHENTICATED", "message": null }
Success	200	{ "success": true }

Note:

X-Rate-Limit-Retry-After-Seconds (in the response header): indicates the number of seconds you need to wait before sending another request after being rate limited. The specific limits are as follows:

• 1 second / 50 request

• 1 minute / 1000 request

• 5 minute / 5000 request

• 1 hour / 50000 request

• 24 hours / 400000 request

Payload Parameters:

Name	Type	Required	Valid value / Description	Example
objectType	String	Yes	Object Type value to determine Entity Object. OBJECT_TYPE below	order
source	String	Yes	Source name that owns your data.	POS, Ecommerce,...
entries	List	Yes	This is a List of Entities. Not null Size: 1-50 entities Entity Objects will be parsed based on objectType. Please Refer to ENTITY_OBJECT tables	[ .. ]

Example

Payload Sample

{
    "objectType":"order",
    "source":"VTA",
    "entries":[
        {
            "orderId":"CDP_TEST_000002",
            "orderStatus":"Created",
            "customerId":"CU_000001",
            "customerName":"Nguyen Van A",
            "customerEmail":"[email protected]",
            "customerPhone":"0123456789",
            "createdBy":"user1",
            "createdAt":1584407426000,
            "updatedBy":null,
            "updatedAt":0,
            "recordStatus":"Active",
            "items":[
                {
                    "productSku":"SKU_0001",
                    "productName":"Product 1",
                    "price":150000,
                    "imageLink":"https://dma.mydatalakes.com/static/media/LogoCDP.24bbda4b.png",
                    "quantity":3
                }
            ],
            "customFieldTexts":{
                "customField01":"Value 1",
                "customField12":"Value 12"
            },
            "customFieldLongs":{
                "customFieldLong01":12,
                "customFieldLong12":3
            },
            "customFieldBooleans":{
                "customFieldBoolean01":true,
                "customFieldBoolean12":false
            },
            "customFieldDoubles":{
                "customFieldDouble01":12,
                "customFieldDouble12":3.6
            },
            "customFieldTimestamps":{
                "customFieldTimestamp01":12122121212121,
                "customFieldTimestamp12":12122121212121
            },
            "customFieldListTexts":{
                "customFieldList01":[
                    "V1",
                    "V2"
                ],
                "customFieldList12":[
                    "V3"
                ]
            }
        }
    ]
}

Response

{
    "success": true
}

Instructions for creating jeConfigId and jeEventKey on the PangoCDP system

• Step 1: Access the Console Module, select the Custom Model that receives Data Ingestion.

• Step 2: Select the Data Manipulation feature, then choose the User Event tab

• Step 3: Use the + Add feature and fill in all required information (depending on usage needs) to create an Event under User Event.

Input Field	Description
Trigger when	Create a User Event when a Data in the Model is created or updated
Event Key	Unique identifier parameter of the Event. System-generated, cannot be entered or modified.
Event Name	The Event name created under User Event. Example: create_order, delivery_order
Event Time	The time when the Event is generated: + When data is created/updated in the Model + According to the time field of creation/update that is ingested into the Model
Channel	Data recording channel: information + None + Online + Offline
Category	Data category: select the appropriate information
Sub Category	Enter sub-category
Filter	Add data filtering conditions within the Model
Update Value	Assign the required values from the Model to the data of the Event generated in User Event.
Disable/Enable	Activation status

• Step 4: After completing the creation, make sure to Enable so the feature can operate.

• Step 5: Click the More Action iconselect View Detailto obtain the information.

  • jeConfigId: use the value of the ID field.

  Example: "id":"890e4b921b11ee870b36f6b36959decd-Created-EWal77RxKcLjsew"

  • jeEventKey: use the value of the eventNameKey field.

  Example: "eventNameKey":"CJE-MA9CNMC7ZA.KFUBAVXEGZ"

  
• Step 6: Use these two values ⇒ add them to the Payload when performing Data Ingestion into the PangoCDP system ⇒ when data is added to the Model, the configured User Event will be generated. A maximum of 200 events can be generated under User Event in a single Ingest Data request.