I am using Delphi XE5 and I am trying to send Items to Walmart using the Bulk API Upload but I am struggling with making proper connection with the server of Walmart. I have already checked some similar problems here like Walmart API - payload mime type issue
The strangest thing of all is that the feed is actually accepted by Walmart (I see it in the list of feeds) but I receive Internal server error and then meaningless characters (probably type of encoding or just kind of error) like
(#$D#$A#$1F#$008B#8#0#0#0#0#0#0#3#$D'A±'#$D#$0080'0'#$C#4'AU~'#2'v ?d'#$0087#4'?'#1#$0091'OE'#$E#$008A'O'#$1E'®?O?'#$18#$0091#$A#$0091'M^'#$C'?;'#$1D'G'#$A'dR±U?oip?m'#$1A'AVS'#4#$16'R?iR'#$1B#$0095'RO?}u'#0'enIjM'#0#0#0)
So I use Indy IdHTTP and I have tried using TIdMultipartFormDataStream and also creating the payload on my own both with receiving the internal server error. I presume I am breaking their parser somehow because if I miss one CR LF (#13#10) at the beginning I get a specific error
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><ns2:errors xmlns:ns2="http://walmart.com/"><ns2:error><ns2:code>SYSTEM_ERROR.GMP_GATEWAY_API</ns2:code><ns2:field>PDR-0012</ns2:field><ns2:description></ns2:description><ns2:info>System encountered some internal error.</ns2:info><ns2:severity>ERROR</ns2:severity><ns2:category>SYSTEM</ns2:category><ns2:causes/><ns2:errorIdentifiers/></ns2:error></ns2:errors>
If I add an extra CRLF at the end then I get another error stating that there is more than one feed in the request. So I really presume that there is something which makes that server / parser return the Internal error but I don't get what and again I repeat that the feed actually gets accepted and then processed by the server in the case I receive this error.
Here is an example request with a payload created on my own:
Headers
POST /v3/feeds?feedType=item HTTP/1.1
Content-Type: multipart/form-data; boundary=qwerty
Content-Length: 5077
WM_SVC.NAME: Walmart Marketplace
WM_QOS.CORRELATION_ID: {67F0E2F9-5EAC-4E8C-9C90-650D8F7B3B7A}
WM_SEC.TIMESTAMP: 1528364885909
WM_SEC.AUTH_SIGNATURE: nSHgqzPOtzSR4wJU+U/vQJk+rk6Ke2QwodTHjzkjau2BonXZxiU9e+3NFPzaat2OUyc+vr0jqRk0H0QWTSC21PrI87mvqei5UJCJwNiIx0zVjAGpxsnIuvtIKkQsBpuUAa8C6SjTiTpDRsNt4IOxrk+tLWxlwQubWVCV+009a6o=
WM_CONSUMER.ID: 16248274-1f53-4e6d-880e-b5417698b878
WM_CONSUMER.CHANNEL.TYPE: 0f3e4dd4-0514-4346-b39d-af0e00ea066d
Host: marketplace.walmartapis.com
Accept: application/xml, */*
Accept-Encoding: identity
Accept-Language: en-US
User-Agent: Mozilla/3.0 (compatible; Indy Library)
Request
--qwerty
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<MPItemFeed xmlns:ns2="http://walmart.com/">
<MPItemFeedHeader>
<version>3.1</version>
<requestId>{694A5564-7E8E-46C7-B772-C944A8C9CF99}</requestId>
<mart>WALMART_US</mart>
</MPItemFeedHeader>
<MPItem>
<processMode>REPLACE_ALL</processMode>
<sku>w-cc-a1-4pk-white-l</sku>
<productIdentifiers>
<productIdentifier>
<productIdType>UPC</productIdType>
<productId>192082276845</productId>
</productIdentifier>
</productIdentifiers>
<MPProduct>
<SkuUpdate>Yes</SkuUpdate>
<productName>Caramel Cantina 4 Pack Sleeping Nursing Cross Front Maternity Bra (Large, White)</productName>
<ProductIdUpdate>Yes</ProductIdUpdate>
<category>
<ClothingCategory>
<Clothing>
<shortDescription>Stretchy comfort cross front sleep bra by Caramel Cantina. Whether for nursing / maternity or for wearing as a sleep bra (or both nursing and sleep bras) - you'll experience a wire free regular back strap sleep bra. This mid-weight fabric but light support wireless bra is great for all day wear and comfort. The straps are wider than a standard bra - offering a longer range of wear time. Great for wearing indoors (ie when you're kicking around the home or going to bed). These are padless - most moms use their own favorite nursing pads with these (though no pads are included). The front cups have two layers of fabric to add a bit more support (though we still call them our light support comfort bra). The criss cross front style allows for easy access while nursing your baby. 92% Nylon 8% Spandex fabric washes up nicely. Cool wash and cool dry. Hang dry for longer life. The straps narrowest point at the shoulders are 1.25-1.5 inches (depending on the size). The bras are super stretchy - if you're looking for something super supportive (ie - they pull you in really tight - more like a sports bra) these probably aren't what you're looking for. Not recommended for heavy sports activity. Best worn for activities such as walking, sleeping, lounging, light activity, running errands, etc.</shortDescription>
<keyFeatures>
<keyFeaturesValue>4 Pack Includes 4 Bras of the Same Design - Color Option Above</keyFeaturesValue>
<keyFeaturesValue>Soft Cup - Front Double Layered - No padding - Wireless</keyFeaturesValue>
<keyFeaturesValue>Super Soft Sleep In Style Bra - Great for Nursing or Sleeping</keyFeaturesValue>
<keyFeaturesValue>Criss Cross Front Design Gives Easy Nursing Access</keyFeaturesValue>
<keyFeaturesValue>Regular Straps - Scoop Back Design - Strap Width 1.25-1.5 Inches Depending on Size</keyFeaturesValue>
</keyFeatures>
<brand>Caramel Cantina</brand>
<mainImageUrl>https://www.dancecheerskate.com/_inv/D54173DB8F934DB2A4C68D24FAC087F5/wm-cc-a1-4pk-white.jpg</mainImageUrl>
<productSecondaryImageURL>
<productSecondaryImageURLValue>https://www.dancecheerskate.com/_inv/D54173DB8F934DB2A4C68D24FAC087F5/wm-cc-a1-wht-m1.jpg</productSecondaryImageURLValue>
</productSecondaryImageURL>
<color>White</color>
<clothingSize>Large</clothingSize>
<variantGroupId>cc-a1-4pk-parent</variantGroupId>
<variantAttributeNames>
<variantAttributeName>color</variantAttributeName>
<variantAttributeName>clothingSize</variantAttributeName>
</variantAttributeNames>
<countryOfOriginTextiles>Imported</countryOfOriginTextiles>
<fabricCareInstructions>
<fabricCareInstruction>Machine wash cold gentle cycle with like colors. Hang Dry. Do Not Bleach. Do Not Iron.</fabricCareInstruction>
</fabricCareInstructions>
<multipackQuantity>1</multipackQuantity>
<countPerPack>4</countPerPack>
<count>4</count>
<pattern>Solid</pattern>
<material>Nylon; Spandex</material>
<gender>Female</gender>
<ageGroup>
<ageGroupValue>Adult</ageGroupValue>
</ageGroup>
<clothingSizeGroup>Maternity</clothingSizeGroup>
<isSet>No</isSet>
<requiresTextileActLabeling>Yes</requiresTextileActLabeling>
<clothingTopStyle>Sleep Bra</clothingTopStyle>
<upperBodyStrapConfiguration>Regular</upperBodyStrapConfiguration>
<braStyle>Nursing</braStyle>
<clothingStyle>Maternity</clothingStyle>
<clothingFit>Maternity</clothingFit>
<isMaternity>Yes</isMaternity>
<swatchImages>
<swatchImage>
<swatchVariantAttribute>color</swatchVariantAttribute>
<swatchImageUrl>https://www.dancecheerskate.com/_inv/D54173DB8F934DB2A4C68D24FAC087F5/wm-cc-a1-4pk-white.jpg</swatchImageUrl>
</swatchImage>
</swatchImages>
</Clothing>
</ClothingCategory>
</category>
</MPProduct>
<MPOffer>
<price>25.00</price>
<ShippingWeight>
<measure>11.80</measure>
<unit>lb</unit>
</ShippingWeight>
<ProductTaxCode>2038895</ProductTaxCode>
</MPOffer>
</MPItem>
</MPItemFeed>
--qwerty--
The response which I get is as following
HTTP/1.1 500 Internal Server Error
Method: null
URI: null
WM_CONSUMER.ID: 16248274-1f53-4e6d-880e-b5417698b878
WM_CONSUMER.INTIMESTAMP:
WM_QOS.CORRELATION_ID: {67F0E2F9-5EAC-4E8C-9C90-650D8F7B3B7A}
WM_SEC.AUTH_TOKEN:
WM_SEC.REFRESH_AUTH_TOKEN:
WM_SVC.CLASS_NAME: com.walmart.services.impl.GMPGatewayService$$EnhancerBySpringCGLIB$$c88958a9
WM_SVC.ENV: prod
WM_SVC.INTIMESTAMP: 1528364905439
WM_SVC.METHOD_NAME: doPostMultiPart
WM_SVC.NAME: Walmart Marketplace
WM_SVC.OUTTIMESTAMP: 1528364905567
WM_SVC.SERVER_IP: 10.65.34.219
WM_SVC.SERVER_NAME: PartnerGMPService-5480119-4-96141283
WM_SVC.VERSION:
X-Powered-By: soari-interceptors-4.4.4
Content-Type: text/plain
Cteonnt-Length: 77
Server: web
Content-Encoding: gzip
Content-Length: 89
Expires: Thu, 07 Jun 2018 09:48:25 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 07 Jun 2018 09:48:25 GMT
Connection: close
And then those characters from above. Note the Cteonnt-Length: 77 header, it is not my TYPO it just comes in this way.
According to the support they have success using the following headers and ask me to use them, but I don't see any difference to what I make
WM_SVC.NAME:Walmart Marketplace
WM_QOS.CORRELATION_ID:123456abcdef
WM_SEC.AUTH_SIGNATURE:TnjevCf+voP9dmnafuCTFruwq6leBuAihSvag89WLieDRBsz7aULxgEqV71ZjIp572wVYPI07y6tdMutNLklDGwxvNdlJ2Q2xGvUIqjVPtqlhdcWvsmgqdpio7puQ4G03q1lReWzTquKecDEbB1ztH6ukj9F5rMe7d7PH8QkFsY=
WM_SEC.TIMESTAMP:1528152581896
WM_CONSUMER.ID:16248274-1f53-4e6d-880e-b5417698b878
WM_CONSUMER.CHANNEL.TYPE:0f3e4dd4-0514-4346-b39d-af0e00ea066d
Accept:application/xml
Host:marketplace.walmartapis.com
Content-Type:multipart/form-data
I already make successful communication with eBay, Amazon, PayPal, Magento and many more, but here obviously I am missing something and really don't know how to solve it.
Thank you for reading this, and hopefully someone can give a clue what is going on here.
EDIT I am adding some Delphi code as required
Version 1 without TIdMultipartFormDataStream
function TdmSyncWallmart.CreateItemFeed(aIDPlatforms: String; aLastChanged,
aCurrDate: TDateTime; var aReqID: String): TStringStream;
var
aXML: TNativeXML;
aNode: TXmlNode;
aItemNode: TXmlNode;
aCatNode: TXmlNode;
aTmpNode: TXmlNode;
aShortDesc: String;
i: Integer;
aIDItems: String;
begin
Result := nil;
dstListPlatformsSyncItemsWallmart.Close;
dstListPlatformsSyncItemsWallmart.Parameters.ParamByName('@IDPlatforms').Value := aIDPlatforms;
dstListPlatformsSyncItemsWallmart.Parameters.ParamByName('@LastChanged').Value := aLastChanged;
dstListPlatformsSyncItemsWallmart.Parameters.ParamByName('@CurrDate').Value := aCurrDate;
dstListPlatformsSyncItemsWallmart.Open;
if dstListPlatformsSyncItemsWallmart.IsEmpty then
Exit;
aReqID := GenGuid;
aXML := TNativeXml.CreateName('MPItemFeed');
try
aXML.Declaration.AttributeAdd('standalone', 'yes');
aXML.Root.AttributeAdd('xmlns:ns2', 'http://walmart.com/');
//Header
aNode := aXML.Root.NodeNew('MPItemFeedHeader');
with aNode.NodeNew('version') do
Value := '3.1';
with aNode.NodeNew('requestId') do
Value := aReqID;
with aNode.NodeNew('mart') do
Value := 'WALMART_US';
//... some code for generating the XML
Result := TStringStream.Create;
aXML.XmlFormat := xfReadable;
Result.WriteString('--qwerty'#13#10#13#10{'Content-Disposition: form-data; filename="wallmartreq.xml"'#13#10}+aXML.WriteToString +'--qwerty--'#13#10);
Result.Position := 0;
finally
aXML.Free;
end;
end;
function TdmSyncWallmart.CreateHTTP(aContentType: String): TIdHTTP;
begin
Result := TIdHTTP.Create(nil);
Result.ConnectTimeout:=5000;
Result.ReadTimeout:=20000;
Result.ProtocolVersion:=pv1_1;
Result.HTTPOptions := [hoForceEncodeParams, hoKeepOrigProtocol];
Result.HandleRedirects:=True;
Result.IOHandler := SSLHandler;
Result.Request.Accept:='application/xml, */*';
Result.Request.AcceptLanguage:='en-US';
Result.Request.ContentType:=aContentType;
// Result.Request.CharSet:='utf-8';
Result.Intercept := IdLogEvent1;
end;
procedure TdmSyncWallmart.AddCustomHeaders(aHTTP: TIdHTTP; aSign,
aTimeStamp: String);
begin
aHTTP.Request.CustomHeaders.AddValue('WM_SVC.NAME', 'Walmart Marketplace');
aHTTP.Request.CustomHeaders.AddValue('WM_QOS.CORRELATION_ID', GenGuid);
aHTTP.Request.CustomHeaders.AddValue('WM_SEC.TIMESTAMP', aTimeStamp);
aHTTP.Request.CustomHeaders.AddValue('WM_SEC.AUTH_SIGNATURE', aSign);
aHTTP.Request.CustomHeaders.AddValue('WM_CONSUMER.ID', dstPlatformsUserName.AsString);
aHTTP.Request.CustomHeaders.AddValue('WM_CONSUMER.CHANNEL.TYPE', dstPlatformsPassword.AsString);
end;
//some code for debugging purposes
procedure TdmSyncWallmart.IdLogEvent1Send(ASender: TIdConnectionIntercept;
var ABuffer: TIdBytes);
var
aList: TStringList;
begin
aList := TStringList.Create;
try
aList.Text := BytesToString(ABuffer);
aList.SaveToFile('e:\wm_req_txt.txt');
finally
aList.Free;
end;
end;
procedure TdmSyncWallmart.IdLogEvent1Receive(ASender: TIdConnectionIntercept;
var ABuffer: TIdBytes);
var
aList: TStringList;
begin
aList := TStringList.Create;
try
aList.Text := BytesToString(ABuffer);
aList.SaveToFile('e:\wm_Resp.txt');
finally
aList.Free;
end;
end;
Actual call
procedure TdmSyncWallmart.SendItems(aIDPlatforms: String; aLastChanged,
aCurrDate: TDateTime);
var
aSign, aTimeStamp: String;
aURL: String;
aHTTP: TIdHTTP;
aRes: String;
aStrm: TMemoryStream;
aReqID: String;
aXML: TNativeXml;
aNode: TXmlNode;
aErrMsg: String;
begin
aURL := dstPlatformsURLAddress.AsString+'/v3/feeds?feedType=item';
GenrateSignature(aSign, aTimeStamp, aURL, 'POST');
if aSign = '' then
Exit;
aHTTP := CreateHTTP('multipart/form-data;boundary=qwerty');
AddCustomHeaders(aHTTP, aSign, aTimeStamp);
try
aStrm := CreateItemFeed(aIDPlatforms, aLastChanged, aCurrDate, aReqID);
if aStrm <> nil then
begin
try
aRes := TIdURI.URLDecode(aHTTP.Post(aURL, aStrm));
aXML := TNativeXml.Create(nil);
try
aXML.ReadFromString(aRes);
aNode := aXML.Root.FindNode('feedId');
cmdUpdFeedID.Parameters.ParamByName('ID').Value := aRes;
cmdUpdFeedID.Parameters.ParamByName('FeedID').Value := aNode.Value;
cmdUpdFeedID.Execute;
finally
aXML.Free;
end;
except
on E:Exception do
begin
if E is EIdHTTPProtocolException then
aErrMsg := E.Message + #13#10 + (E as EIdHTTPProtocolException).ErrorMessage
else
aErrMsg := E.Message;
dmMain.InsLog(cLTError, aErrMsg, aIDPlatforms);
end;
end;
aStrm.Free;
end;
finally
aHTTP.Free;
end;
end;
EDIT2
procedure TdmSyncWallmart.SendItems(aIDPlatforms: String; aLastChanged,
aCurrDate: TDateTime);
var
aSign, aTimeStamp: String;
aURL: String;
aHTTP: TIdHTTP;
aRes: String;
aStrm: TMemoryStream;
aReqID: String;
aXML: TNativeXml;
aNode: TXmlNode;
aErrMsg: String;
aIdStream: TIdMultipartFormDataStream;
begin
aURL := dstPlatformsURLAddress.AsString+'/v3/feeds?feedType=item';
GenrateSignature(aSign, aTimeStamp, aURL, 'POST');
if aSign = '' then
Exit;
aHTTP := CreateHTTP('multipart/form-data');
AddCustomHeaders(aHTTP, aSign, aTimeStamp);
try
aStrm := TStringStream.Create();
begin
try
aIdStream := TIdMultiPartFormDataStream.Create;
aIdStream.AddFile('file', 'e:\wm_req.xml', 'multipart/form-data');
aIdStream.Position := 0;
astrm.LoadFromStream(aIdStream);
aRes := TIdURI.URLDecode(aHTTP.Post(aURL, aStrm));
aXML := TNativeXml.Create(nil);
try
aXML.ReadFromString(aRes);
aNode := aXML.Root.FindNode('feedId');
cmdUpdFeedID.Parameters.ParamByName('ID').Value := aRes;
cmdUpdFeedID.Parameters.ParamByName('FeedID').Value := aNode.Value;
cmdUpdFeedID.Execute;
finally
aXML.Free;
end;
except
on E:Exception do
begin
if E is EIdHTTPProtocolException then
aErrMsg := E.Message + #13#10 + (E as EIdHTTPProtocolException).ErrorMessage
else
aErrMsg := E.Message;
dmMain.InsLog(cLTError, aErrMsg, aIDPlatforms);
end;
end;
aStrm.Free;
end;
finally
aHTTP.Free;
end;
end;
This is the sent data
----------060718213723831
Content-Disposition: form-data; name="file"; filename="wm_req.xml"
Content-Type: multipart/form-data
Content-Transfer-Encoding: binary
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<MPItemFeed xmlns:ns2="http://walmart.com/">
<MPItemFeedHeader>
<version>3.1</version>
<requestId>{164726BF-288E-4953-A1DA-1EC7595C5725}</requestId>
<mart>WALMART_US</mart>
</MPItemFeedHeader>
<MPItem>
<processMode>REPLACE_ALL</processMode>
<sku>w-cc-a1-4pk-white-l</sku>
<productIdentifiers>
<productIdentifier>
<productIdType>UPC</productIdType>
<productId>192082276845</productId>
</productIdentifier>
</productIdentifiers>
<MPProduct>
..... the other part of the XML
</MPProduct>
<MPOffer>
<price>25.00</price>
<ShippingWeight>
<measure>11.80</measure>
<unit>lb</unit>
</ShippingWeight>
<ProductTaxCode>2038895</ProductTaxCode>
</MPOffer>
</MPItem>
</MPItemFeed>
----------060718213723831--
Returned data captured by the intercepting part is the same as the one already posted with again the Cteonnt-Length: 77 header, but I guess if I succeed in reading the gzip data, I can reach an error which can be forwarded to the support.
HTTP/1.1 500 Internal Server Error
Method: null
URI: null
WM_CONSUMER.ID: 16248274-1f53-4e6d-880e-b5417698b878
WM_CONSUMER.INTIMESTAMP:
WM_QOS.CORRELATION_ID: {CB0208AB-2135-4A12-B45F-DD2889273971}
WM_SEC.AUTH_TOKEN:
WM_SEC.REFRESH_AUTH_TOKEN:
WM_SVC.CLASS_NAME: com.walmart.services.impl.GMPGatewayService$$EnhancerBySpringCGLIB$$2ca3f3b6
WM_SVC.ENV: prod
WM_SVC.INTIMESTAMP: 1528396669253
WM_SVC.METHOD_NAME: doPostMultiPart
WM_SVC.NAME: Walmart Marketplace
WM_SVC.OUTTIMESTAMP: 1528396669365
WM_SVC.SERVER_IP: 10.65.34.2
WM_SVC.SERVER_NAME: PartnerGMPService-5480119-1-96141274
WM_SVC.VERSION:
X-Powered-By: soari-interceptors-4.4.4
Content-Type: text/plain
Cteonnt-Length: 77
Server: web
Content-Encoding: gzip
Content-Length: 89
Vary: Accept-Encoding
Expires: Thu, 07 Jun 2018 18:37:49 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Thu, 07 Jun 2018 18:37:49 GMT
Connection: close
EDIT3
For my Edit1, I am not adding Content-Disposition at all (it is commented) - with so many tests I have left it in my post here, so sorry for the confusion.
For Edit2 - thanks for pointing this out, actually I made my first tests without transforming into TStringStream, but I wanted to see what exactly goes to them.
Anyway now I directly used the TIdMultipartFormDataStream and changed the code in this way
aIdStream := TIdMultiPartFormDataStream.Create;
with aIdStream.AddFile('xml', 'e:\wm_req.xml', 'application/xml') do
begin
FileName := '';
//ContentType := '';
FileName := 'wmreq.xml';//somewhere I think I saw that only letters are allowed here
end;
aIdStream.Position := 0;
aStrm := TMemoryStream.Create;
//aRes := TIdURI.URLDecode(aHTTP.Post(aURL, aIdStream));
aHTTP.Post(aURL, aIdStream, aStrm);
and thanks to Remy I added TIdCompressorZLib and now I get meaningful error:
HTTP/1.1 500 Internal Server Error No message body writer has been found for response class FeedAcknowledgement.
I guess this is from them, not from me, will need to check with them.
EDIT4 Solution
OK, crazy or not, maybe my problem, but once Remy said that I am not asking for compressed answer and they send me such, I decided to change // Result.Request.Accept:='application/xml, */*'; to only // Result.Request.Accept:='application/xml'; and all started working properly! In fact having */* means that I am accepting everything, isn't it? It doesn't matter if you are using TIdMultipartFormDataStream or the body is manually prepared, it just works. Thank you Remy for the different point of view, another pair of eyes always helps. Crazy ... one week of stupid tests!
