0

I'm trying to convert a CSV to XML using Logic App, which had always worked fine until I hit a size issue, don't even know what the size limit is. Anyone have any idea!?

enter image description here

tommyhmt
  • 145
  • 2
  • 11
  • Any reason you tagged this with BizTalk? – Dijkgraaf Sep 23 '21 at 02:50
  • Hmm, this doesn't really answer your question, https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-handle-large-messages – Dijkgraaf Sep 23 '21 at 02:54
  • @Dijkgraaf because I created the schema and mapping files with BizTalk, wasn't sure if there was a limit on that too And yes can see there is a 100mb limit but is there a way round it? – tommyhmt Sep 23 '21 at 08:16

2 Answers2

0

Based on the MS DOC :

Messages are "large" based on the service handling those messages. The exact size limit on large messages differs across Logic Apps and connectors. Both Logic Apps and connectors can't directly consume large messages, which must be chunked.
For each connector's message size limit, see the connector's specific technical details.

You can try to enable chunking and chunk the data to the logic app over the HTTP as below screenshot.

enter image description here

The details for the chunking process is available at Handle large messages with chunking in Azure Logic Apps

For more information please refer this SO Thread HTTP-413(request entity too large) error in logic app .

AjayKumarGhose
  • 4,257
  • 2
  • 4
  • 15
  • What if none of the steps I use contain the Allow chunking setting? Because the first trigger I use is "When a HTTP request is received" and not HTTP as that is how I'm calling the logic app to run via data factory. Apologies for my ignorance but would simply changing my trigger from "When a HTTP request is received" to "HTTP" do the trick, or would I need to place "HTTP" either before or after "When a HTTP request is received"? – tommyhmt Sep 23 '21 at 12:54
  • For connectors that support chunking, the underlying chunking protocol is invisible to end users. However, not all connectors support chunking, so these connectors generate runtime errors when incoming messages exceed the connectors' size limits. See this [MS DOC](https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-handle-large-messages#:~:text=For%20actions%20that%20support%20and%20are%20enabled%20for%20chunking,%20you%20can%27t%20use%20trigger%20bodies,%20variables,%20and%20expressions) – AjayKumarGhose Sep 23 '21 at 13:23
  • so what are my options here exactly I'm still not sure whether I can actually have a workaround solution. I've read somewhere that Azure functions could potentially be an option but I can't find any examples to do that with the already existing schema and mapping files I created using BizTalk. – tommyhmt Sep 23 '21 at 21:58
0

As far as I remember, the size limit for XML's that can be processed by an Integration Account is 50 MB.

In the past, I used to split huge XML's into smaller chunks using Azure Functions before transforming them one by one in the Integration Account, but that's not a universal solution.

You could also create an Azure Function to handle the transformation instead of using an Integration Account for this purpose.

Another hack you could try - make the XML tags smaller (1- or 2-character long) by renaming the nodes in your schema files. It could greatly reduce the size of your XML's. This isn't a panacea, but in some scenarios it could help.

10p
  • 5,488
  • 22
  • 30