Saving 1D numpy arrays of different sizes to txt file columnwise

Question

Below is the illustration code

from pylab import *

 a = array([1,2,3])
 b = array([4,5])

What I want test.out has is

1 4
2 5
3

previously, people has given solution of store of 1D numpy arrays of different sizes to txt file rowwise: Saving numpy array to txt file row wise

Then how to save them columnwise?

Of course you can use three array like this

 a = array([1,4])
 b = array([2,5])
 c=array([3])

and save them in row wise however it is not a smart way, when there is a lot of 1D arrays.

Open a file, and write the numbers out line by line. Use ordinary Python string formatting to create each line. That is basically what `np.savetxt` does. — hpaulj, Dec 12 '15 at 06:11
I will learn pandas later.@Colonel savetxt only saves arrays with the same sizes @hpaulj — user3737702, Dec 12 '15 at 06:18
That's why I suggest doing your own write. I mention `savetxt` to make it clear that a homespun writer is just as good - better if it handles your problem. This is just an ordinary Python file write problem. — hpaulj, Dec 12 '15 at 06:24
no need to write your own write ... pandas is just simplifying life a lot ;) — Colonel Beauvel, Dec 12 '15 at 06:36

Colonel Beauvel · Answer 1 · 2015-12-12T06:22:44.893

2

A simple approach with pandas:

import pandas as pd

d  = dict(a=a, b=b)
df = pd.DataFrame.from_dict(d, orient='index').transpose().fillna('')

#   a  b
#0  1  4
#1  2  5
#2  3

And write in a csv (you do not want to write the index nor the column):

df.to_csv('file.csv', index=False, header=False)

edited Dec 12 '15 at 06:22

answered Dec 12 '15 at 06:14

Colonel Beauvel

30,423
11
47
87

I am learning how to install pandas now, seemingly that .whl is difficult to install than numpy.exe etc. – user3737702 Dec 12 '15 at 06:38
you can directly install Anaconda https://www.continuum.io/downloads, it's python including most of the needed library (numpy, pandas, etc) – Colonel Beauvel Dec 12 '15 at 06:44
I succeeded to install it using pypi, but Why i got this 4 5 is the first column? 1 2 3 second? ...confused – user3737702 Dec 12 '15 at 07:05
Somehow, the sequence of columns is not fixed as a and b... Is it because the pandas package is installed improperly? – user3737702 Dec 13 '15 at 03:32
what is your result? you can reorder the columns if needed. – Colonel Beauvel Dec 13 '15 at 05:54
my result is column 1 is 4, 5. column 2 is 1,2,3. While yours are correct, 1st column is 1,2,3, 2nd column is 4,5. very tricky!!! – user3737702 Dec 13 '15 at 07:13

hpaulj · Answer 2 · 2017-06-29T15:28:17.963

My answer in https://stackoverflow.com/a/34242952/901925 could be adapted to this case

3 arrays of different length; could be lists

In [715]: heights=np.array([40,50])
In [716]: widths=np.array([60,65,70])
In [717]: yrs=np.array([1995,1996,1997,1998])

use itertools.zip_longest to iterate through them, with a fillvalue where values are missing:

In [718]: for xyz in itertools.zip_longest(yrs,widths,heights,fillvalue=''):
    print('%-12s,%-12s,%-12s'%xyz)
   .....:     
1995        ,60          ,40          
1996        ,65          ,50          
1997        ,70          ,            
1998        ,            ,

to write to a file use:

In [719]: with open('temp.txt','w') as f:
    for xyz in itertools.zip_longest(yrs,widths,heights,fillvalue=''):
        f.write('%-12s,%-12s,%-12s\n'%xyz)
   .....:         
In [720]: cat temp.txt
1995        ,60          ,40          
1996        ,65          ,50          
1997        ,70          ,            
1998        ,            ,

(itertools.zip_longest in PY3, izip_longest in Py2).

Saving 1D numpy arrays of different sizes to txt file columnwise

2 Answers2