How to create temporal/virtual files?

Question

I need to save a list in a temporary or virtual file that after being executed in a linux terminal from a python, the temporary file must be deleted.

This is the list:

formato_db=[{'op': 'core/column-reorder', 'columnNames': ['\ufeffNúmero de Cliente', '[Account.AccountCode?]', 'CRM Origen', 'Monto inicial', '[ModInitCta?]', '[DeudaRealCuenta?]', '[BillCycleName?]', 'Nombre Campaña', 'Nombre Casa de Cobro', 'Fecha de Asignacion', 'Deuda Gestionable', 'Dirección Completa', 'Fecha Final', 'Email', 'Telefono 1', 'Telefono 2', 'Telefono 3', 'Telefono 4', 'Segmento', '[Documento?]', '[AccStsName?]', 'Ciudad', '[InboxName?]', 'Nombre del Cliente', 'Id de Ejecucion', 'Fecha de Vencimiento', 'Numero Referencia de Pago', 'MIN', 'Plan', 'Precio Subscripcion'], 'description': 'Reorder columns'}, {'op': 'core/column-addition', 'engineConfig': {'facets': [], 'mode': 'row-based'}, 'baseColumnName': 'Telefono 1', 'expression': "join ([coalesce(cells['Telefono 1'].value,''),coalesce(cells['Telefono 2'].value,''),coalesce(cells['Telefono 3'].value,''),coalesce(cells['Telefono 4'].value,'')],' / ')", 'onError': 'keep-original', 'newColumnName': 'Telefonos', 'columnInsertIndex': 15, 'description': "Create column Telefonos at index 15 based on column Telefono 1 using expression join ([coalesce(cells['Telefono 1'].value,''),coalesce(cells['Telefono 2'].value,''),coalesce(cells['Telefono 3'].value,''),coalesce(cells['Telefono 4'].value,'')],' / ')"}, {'op': 'core/column-reorder', 'columnNames': ['\ufeffNúmero de Cliente', '[Account.AccountCode?]', 'CRM Origen', 'Monto inicial', '[ModInitCta?]', '[DeudaRealCuenta?]', '[BillCycleName?]', 'Nombre Campaña', 'Nombre Casa de Cobro', 'Fecha de Asignacion', 'Deuda Gestionable', 'Dirección Completa', 'Fecha Final', 'Email', 'Telefonos', 'Segmento', '[Documento?]', '[AccStsName?]', 'Ciudad', '[InboxName?]', 'Nombre del Cliente', 'Id de Ejecucion', 'Fecha de Vencimiento', 'Numero Referencia de Pago', 'MIN', 'Plan', 'Precio Subscripcion'], 'description': 'Reorder columns'}, {'op': 'core/column-reorder', 'columnNames': ['\ufeffNúmero de Cliente', '[Account.AccountCode?]', 'CRM Origen', 'Monto inicial', '[ModInitCta?]', '[DeudaRealCuenta?]', '[BillCycleName?]', 'Nombre Campaña', 'Nombre Casa de Cobro', 'Fecha de Asignacion', 'Deuda Gestionable', 'Dirección Completa', 'Fecha Final', 'Email', 'Telefonos', 'Segmento', 'Ciudad', '[InboxName?]', 'Nombre del Cliente', 'Id de Ejecucion', 'Fecha de Vencimiento', 'Numero Referencia de Pago', 'Plan', 'Precio Subscripcion'], 'description': 'Reorder columns'}, {'op': 'core/text-transform', 'engineConfig': {'facets': [], 'mode': 'row-based'}, 'columnName': 'Monto inicial', 'expression': 'value.toNumber()', 'onError': 'keep-original', 'repeat': False, 'repeatCount': 10, 'description': 'Text transform on cells in column Monto inicial using expression value.toNumber()'}, {'op': 'core/text-transform', 'engineConfig': {'facets': [], 'mode': 'row-based'}, 'columnName': 'Dirección Completa', 'expression': 'value.toTitlecase()', 'onError': 'keep-original', 'repeat': False, 'repeatCount': 10, 'description': 'Text transform on cells in column Dirección Completa using expression value.toTitlecase()'}, {'op': 'core/text-transform', 'engineConfig': {'facets': [], 'mode': 'row-based'}, 'columnName': '\ufeffNúmero de Cliente', 'expression': 'grel:value.substring(0, value.length()-2)', 'onError': 'keep-original', 'repeat': False, 'repeatCount': 10, 'description': 'Text transform on cells in column \ufeffNúmero de Cliente using expression grel:value.substring(0, value.length()-2)'}, {'op': 'core/text-transform', 'engineConfig': {'facets': [], 'mode': 'row-based'}, 'columnName': 'Nombre Campaña', 'expression': 'grel:value.substring(4)', 'onError': 'keep-original', 'repeat': False, 'repeatCount': 10, 'description': 'Text transform on cells in column Nombre Campaña using expression grel:value.substring(4)'}]

These are the functions I am using.

def get_transformations(formato_db):
    formato_db = bytes(formato_db)
    with tempfile.SpooledTemporaryFile() as tfile:
        json.dump(formato_db, tfile)
        tfile.flush()
    return tfile

def apply_transformation_db(proyecto_id, tfile):
    err = None
    proyecto_id = str(proyecto_id) #hash of OpenRefine project id
    cmd = ['sudo', 'docker', 'run', '--rm', '--network=host', '-v', 
          '/home/ubuntu/api:/data:z', 'felixlohmeier/openrefine-client:v0.3.10', 
          '-H', 'localhost', '--apply', tfile, proyecto_id
    ]
    subprocess.run(cmd)
    return err 
tfile = transform.get_transformations(formato_db)
err_transf_application = apply_transformation_db(proyecto_id, formato_db)

I used to save the list as a json file in this way:

with open('transformaciones.json', 'w') as f:
         json.dump(formato_db, f)

However now I need to save it as a temporal o virtual file. I tried to use tempfile.SpooledTemporaryFile() but give me an error.

TypeError: 'dict' object cannot be interpreted as an integer

Any idea how I can use temporary files in those functions?

score 1 · Answer 1 · answered Aug 10 '22 at 20:57

1

Looks like the problem is in line two where you try to convert your dictionary into bytes. Try converting to a string first.

def get_transformations(formato_db):
    formato_db = bytes(str(formato_db),encoding='utf8')
    ...
    ....

Edit: To add, it doesn't look (to me) like the problem has anything to do with temporary files or spools or anything like that, it is just a matter of converting a using the bytes() function on a str vs on a dict.

answered Aug 10 '22 at 20:57

scotscotmcc

2,719
1
6
29

Thanks for your comment, I tried your suggestion but it gives me the following error: TypeError: Object of type bytes is not JSON serializable. I need the list to be saved as a file because otherwise I can't run it properly on terminal. – frankh07 Aug 10 '22 at 21:15
That error is exactly what it says it is: you cannot turn bytes into JSON. So look at your code where you are doing that, which is in your `json.dumps(formato_db,...)` line. So why are you converting formato_db into bytes anyway? You should probably just completely remove that line (the line that I put in my post here) and try to do `json.dumps(...)` on the original formato_db dictionary. – scotscotmcc Aug 11 '22 at 13:46

How to create temporal/virtual files?

1 Answers1