How to use following list entries to create REST API request details - in Python

Question

Following is the list I am getting from source.

[("!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: gSoCamgSoCam\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n    maxR: 2\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: gm_tt_SedCa}\n    sysAt:\n      undefined: {enabled: fl}\n  reqCktl:\n  - dMD:\n      cA: {}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: gm_tt_SedCa_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: GM_19_02-01-2019_236166987\n  uCaId: gm_tt_SedCa\nversion: '1.0'\n",), ("!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: GTEdRec\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: ult Picks}\n    sysAt: {}\n  reqCktl:\n  - dMD:\n      cA: {}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: ult/ult_pi_var_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: ult/ult_Picks_11-01-2019_1003834703\n  uCaId: ult/gm_tt_SedCa\nversion: '1.0'\n",), ("!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: GTEdRec\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n    campaignIds: gm_tt_SedCa_2018\n    maxR: 3\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: Vik Sea 2017}\n    sysAt:\n      taUsecaseDateRange: {enabled: fl}\n  reqCktl:\n  - dMD:\n      cA: {Cat: Ed}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: ult/gm_tt_SedCa8-copy_variant_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: ult/gm_tt_SedCa_2018-Copy_11-01-2017_1003834668\n  uCaId: ult/gm_tt_SedCa-Copy\nversion: '1.0'\n",)]

I need to use that list to create REST URLs like

URL: http://<url>/<use value in-front of uCaId> 
Method:<use value available for reqTy>
Req Body: { 
"rParam": { 
 "cItId":["[<use cttSceId>]<random id>"]
  "aMarB":<use value for aMarB>
 } 
}

So it should traverse and will give me 3 URLs in this case. (which later I am planning o pass to PyTest assertions)
Not sure how to achieve this with Python. can anyone help?

PS: from comments suggestion I've tried

import yaml

descriptors1 = [
    [(
     "!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: gSoCamgSoCam\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n    maxR: 2\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: gm_tt_SedCa}\n    sysAt:\n      undefined: {enabled: fl}\n  reqCktl:\n  - dMD:\n      cA: {}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: gm_tt_SedCa_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: GM_19_02-01-2019_236166987\n  uCaId: gm_tt_SedCa\nversion: '1.0'\n",),
     (
     "!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: GTEdRec\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: ult Picks}\n    sysAt: {}\n  reqCktl:\n  - dMD:\n      cA: {}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: ult/ult_pi_var_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: ult/ult_Picks_11-01-2019_1003834703\n  uCaId: ult/gm_tt_SedCa\nversion: '1.0'\n",),
     (
     "!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n  dMD:\n    cA: {}\n    descMap: {en: R Des}\n    dispNm: {en: Rq Nm}\n    sysAt: {}\n  id: R_I_001\n  para:\n    mt: GTEdRec\n    cT: 1\n    cttSceId: ['2']\n    aMarB: fl\n    campaignIds: gm_tt_SedCa_2018\n    maxR: 3\n  pR: null\n  reqCon: {reqTy: GET}\nuCs:\n- dMD:\n    cA: {}\n    descMap: {}\n    dispNm: {en: Vik Sea 2017}\n    sysAt:\n      taUsecaseDateRange: {enabled: fl}\n  reqCktl:\n  - dMD:\n      cA: {Cat: Ed}\n      descMap: {}\n      dispNm: {en: Default}\n      sysAt: {}\n    id: ult/gm_tt_SedCa8-copy_variant_0\n    reqL:\n    - *id001\n    slpara: {spPeMin: 1, spPerMx: 100}\n  stUCad: ult/gm_tt_SedCa_2018-Copy_11-01-2017_1003834668\n  uCaId: ult/gm_tt_SedCa-Copy\nversion: '1.0'\n",)]
]

a = yaml.dump(descriptors1,explicit_start=True, default_flow_style=False) ## a is type 'str' and giving o
#a = yaml.safe_dump(descriptors1,explicit_start=True, default_flow_style=False)

class PSLoader(yaml.SafeLoader):
    def construct_python_tuple(self, node):
        return tuple(self.construct_sequence(node))

PSLoader.add_constructor(
    u'tag:yaml.org,2002:python/tuple',
    PSLoader.construct_python_tuple)


yamlholder = yaml.load(a, Loader=PSLoader)

print(yamlholder)

now yamlholderis of type 'list'

and op is like this op is [("!!cm.mymodel.ucm.model.uCConf.UCConf\nrequests:\n- &id001\n dMD:\n cA: {}\n descMap: {en: Rq Nm}\n dispNm: {en: Rq Nm}\n sysAt: {}\n id: REQ_ID_001\n para:\n mT: getSortedCampaigns\n cT: 1\n cttSceId: ['2']\n aMarB: false\n maxR: 2\n pR: null\n reqCon: {reqTy: GET}\nuCs:\n- dMD:\n cA: {}\n descMap: {}\n dispNm: {en: gm_tt_SedCa}\n sysAt:\n undefined: {enabled: fl}\n reqCktl:\n - dMD:\n cA: {}\n descMap: {}\n dispNm: {en: Default}\n sysAt: {}\n id: gm_tt_SedCa_0\n reqL:\n - *id001\n slpara: {spPeMin: 1, spPerMx: 100}\n stUCad: GM_19_02-01-2019_236166987\n uCaId: m_tt_SedCa\nversion: '1.0'\n",), ...........)]

What have you tried for yourself? Or is it more a question of the concepts and not code? Edit: There are many ways to achieve this. You should probably start by using iterating over your list and do something with either a dictionary or a formatted string. — JustLudo, Mar 19 '20 at 07:30
@Ludo21South What I tried is: Pulled required data from list using Regex (like uCaIds and cttSceId). Then pushed that into excel (for reference). Now pulling again from excel to build URLs. But as I have used Regex, I won't able to pull all values when there is change in data pattern. So scrapped that and thinking on best way to implement this. — Finch, Mar 19 '20 at 07:39
This input looks like yaml to me, so [PyYaml](https://github.com/yaml/pyyaml) would help. — bereal, Mar 19 '20 at 07:43
Yep its yaml. But stored in db. when I pull that in my code its showing python type as 'list' by default — Finch, Mar 19 '20 at 07:44
Like bereal says: parse that first. The input shouldn't matter a lot in this case. Also, update your question with the things you tried. You are more likely to get an answer that way. — JustLudo, Mar 19 '20 at 08:36
@bereal I have tried: `print(type(descriptors1)) print(type(descriptors1)) ##descriptors1 is where yaml like list stored a = yaml.dump(descriptors1,explicit_start=True, default_flow_style=False) ## a is type 'str' ` — Finch, Mar 19 '20 at 09:18
Yes, `descriptors1` is a list, `descriptors1[0]` is a tuple, `descriptors1[0][0]` is a string. A string can be parsed. — bereal, Mar 19 '20 at 09:39
@bereal could you please suggest me how to proceed next? cause when I am accessing the element `print(a[500])` its accessing char but not entire string — Finch, Mar 19 '20 at 10:42
@bereal Tried this `print(yaml.safe_load(a))` and got this **yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/tuple' in "", line 2, column 3: - !!python/tuple** — Finch, Mar 19 '20 at 12:17
@bereal Followed the solution written over https://stackoverflow.com/questions/9169025/how-can-i-add-a-python-tuple-to-a-yaml-file-using-pyyaml and parsed the yaml. But again it is showing type as list — Finch, Mar 19 '20 at 12:36
@Rekam03 I cannot help without your full code and exact input. — bereal, Mar 19 '20 at 12:37
@Rekam03 it will be way more helpful when it is [mcve] which I can paste and run as is. — bereal, Mar 19 '20 at 13:10

score 0 · Answer 1 · answered Mar 20 '20 at 09:59

0

In your code, you first dump your data (which is a list) to yaml, then parse it back. No wonder you're getting a list again. Instead, you need to extract the yaml from the list first, and then parse it:

class PSLoader(yaml.SafeLoader):
    pass

def constructor(loader, node):
    return loader.construct_mapping(node)

# to handle the custom tag
PSLoader.add_constructor(
    'tag:yaml.org,2002:cm.mymodel.ucm.model.uCConf.UCConf', 
    constructor
)

for entry, in descriptors1[0]:
    parsed = yaml.load(entry, Loader=PSLoader)
    print(parsed)

Then you will see that the parsed entries are dicts, which you can use to send the requests.

answered Mar 20 '20 at 09:59

bereal

32,519
6
58
104

Thanks for this. But I am getting _for entry, in descriptors1[0]: ValueError: too many values to unpack (expected 1)_ error – Finch Mar 20 '20 at 10:30
It's because your actual data is a list of tuples, and the one that you posted is a list of list of tuples. So replace `descriptors1[0]` with `descriptors1`. – bereal Mar 20 '20 at 12:17
Yep. got it. +1 :) And other than iterating the dict, do you know other way to create those rest requests efficiently? In addition to that when I checked **parsed.keys()** I can see only three keys _requests, uCaId & version_ So is there any way to access **reqTy, cttSceId & aMarB** values? (Sorry to bother you. I am newbie) – Finch Mar 20 '20 at 12:52
@Rekam03 you can access dict values with `parsed['requests']` which will return a list of dicts. Remaining is an exercise. – bereal Mar 20 '20 at 12:56
https://stackoverflow.com/users/770830/bereal I am able to access the strings. Can you guide me, how to use them one-by-one in Rest URL with their own body params? – Finch Apr 06 '20 at 12:42
So you have a list of dictionaries, which you want to convert to a list of urls. Do you know how to make a url from a dictionary? – bereal Apr 06 '20 at 12:47
https://stackoverflow.com/users/770830/bereal 1. Not sure how to create url from dict. will check that. Right now I am creating urls simply like ` url = 'http://:port//'+urllib.parse.quote_plus( uCaId)+'?userId=testuser'` 2. But how to pass exact request body params (as mentioned in question's description) to respective rest urls? that's where I am stuck. 3.Coz when I check 'type' of extracted dict values, its showing 'str'. [I am using _extractedMT = parsed['requests'][0]['para']['mT'] to get all mT values_] – Finch Apr 06 '20 at 14:12
@Rekam03 create a list of tuples `(url, body_params)`. – bereal Apr 06 '20 at 14:16
https://stackoverflow.com/users/770830/bereal If possible could you please post sample or relevant code snippet which I can refer for my solution? – Finch Apr 07 '20 at 10:43
@Rekkam03 please ask a new question with more focus on your current problem and the relevant code. – bereal Apr 07 '20 at 10:50

How to use following list entries to create REST API request details - in Python

1 Answers1