Mathematica "AppendTo" function problem

Question

I'm a newbie in Mathematica and I'm having a major malfunction with adding columns to a data table. I'm running Mathematica 7 in Vista. I have spent a lot of time RFD before asking here.

I have a data table (mydata) with three columns and five rows. I'm trying to add two lists of five elements to the table (effectively adding two columns to the data table).

This works perfectly:

Table[AppendTo[mydata[[i]],myfirstlist[[i]]],{i,4}] 

Printing out the table with: mydata // TableForm shows the added column.

However, when I try to add my second list

Table[AppendTo[mydata[[i]],mysecondlist[[i]]],{i,5}]

either Mathematica crashes(!) or I get a slew of Part::partw and Part::spec errors saying Part 5 does not exist.

However, after all the error messages (if Mathematica does not crash), again printing out the data table with: mydata // TableForm shows the data table with five columns just like I intended. All TableForm formatting options on the revised data table work fine.

Could anyone tell me what I'm doing wrong? Thanks in advance!

Answer 1

Let's try to clarify what the double transpose method consists of. I make no claims about the originality of the approach. My focus is on the clarity of exposition.

Let's begin with 5 lists. First we'll place three in a table. Then we'll add the final two.

food = {"bagels", "lox", "cream cheese", "coffee", "blueberries"};
mammals = {"fisher cat", "weasel", "skunk", "raccon", "squirrel"};
painters = {"Picasso", "Rembrandt", "Klee", "Rousseau", "Warhol"};
countries = {"Brazil", "Portugal", "Azores", "Guinea Bissau", 
             "Cape Verde"};
sports = {"golf", "badminton", "football", "tennis", "rugby"};

The first three lists--food, mammals, painters--become the elements of lists3. They are just lists, but TableForm displays them in a table as rows.

(lists3 = {food, mammals, painters}) // TableForm

mydata will be the name for lists3 transposed. Now the three lists appear as columns. That's what transposition does: columns and rows are switched.

(mydata = Transpose@lists3) // TableForm

This is where the problem actually begins. How can we add two additional columns (that is, the lists for countries and sports)? So let's work with the remaining two lists.

(lists2 = {countries, sports}) // TableForm

So we can Join Transpose[mydata] and lists2....

(lists5 = Join[Transpose[mydata], lists2]) // TableForm

[Alternatively, we might have Joined lists3 and lists2 because the second transposition, the transposition of mydata undoes the earlier transposition. lists3 is just the transposition of mydata. (and vice-versa).]

In[]:= lists3 === Transpose[mydata]
Out[]:= True

Now we only need to Transpose the result to obtain the desired final table of five lists, each occupying its own column:

Transpose@lists5 // TableForm

I hope that helps shed some light on how to add two columns to a table. I find this way reasonably clear. You may find some other way clearer.

Answer 2

There are several things to cover here. First, the following code does not give me any errors, so there may be something else going on here. Perhaps you should post a full code block that produces the error.

mydata = Array[Subscript[{##}] &, {5, 3}];
myfirstlist = Range[1, 5];
mysecondlist = Range[6, 10];

Table[AppendTo[mydata[[i]], myfirstlist[[i]]], {i, 4}];
mydata // TableForm

Table[AppendTo[mydata[[i]], mysecondlist[[i]]], {i, 5}];
mydata // TableForm

Second, there is no purpose in using Table here, as you are modifying mydata directly. Table will use up memory pointlessly.

Third, there are better ways to accomplish this task.

See How to prepend a column and Inserting into a 2d list

I must retract my definitive statement that there are better ways. After changing Table to Do and running a few quick tests, this appears to be a competitive method for some data.

I am using Mathematica 7, so that does not appear to be the problem.

Answer 3

As mentioned before, there are better alternatives to adding columns to a list, and like Gareth and Mr.Wizard, I do not seem to be able to reproduce the problem on v. 7. But, I want to focus on the error itself, and see if we can correct it that way. When Mathematica produces the message Part::partw it spits out part of the offending list like

Range[1000][[1001]]
Part::partw: Part 1001 of  
    {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
    31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,<<950>>} 
    does not exist.

So, the question I ask is which list is giving me the problems? My best guess is it is mysecondlist, and I'd check Length @ mysecondlist to see if it is actually 5 elements long.

Answer 4

Well, here's my two cents with what I believe is a very fast and IMHO most easily understandable construction.

First, some test arrays:

m = RandomInteger[100, {2000, 10000}];
l1 = RandomInteger[100, 2000];
l2 = RandomInteger[100, 2000];
{r, c} = Dimensions[m];

I increased the test array sizes somewhat to improve accuracy of the following timing measurements.

The method involves the invoking of the powers of Part ([[...]]), All and Span (;;).

Basically, I set up a new working matrix with the future dimensions of the data array after addition of the two columns, then add the original matrix using All and Span and add the additional columns with All only. I then copy back the scrap matrix to our original matrix, as the other methods also return the modified data matrix.

n = ConstantArray[0, {r, c} + {0, 2}];
n[[All, 1 ;; c]] = m;
n[[All, c + 1]] = l1;
n[[All, c + 2]] = l2;
m = n;

As for timing:

Mean[
 Table[
  First[
   AbsoluteTiming[
    n2 = ConstantArray[0, {r, c} + {0, 2}];
    n2[[All, 1 ;; c]] = m;
    n2[[All, c + 1]] = l1;
    n2[[All, c + 2]] = l2;
    m2 = n2;
    ]
   ],
  {10}
  ]
 ]

0.1056061

(an average of 10 runs)

The other proposed method with Do (Mr.Wizard and the OP):

Mean[
 Table[
  n1 = m;
  First[
   AbsoluteTiming[
    Do[AppendTo[n1[[i]], l1[[i]]], {i, 2000}]; 
    Do[AppendTo[n1[[i]], l2[[i]]], {i, 2000}];
    ]
   ],
  {10}
  ]
 ]

0.4898280

The result is the same:

In[9]:= n2 == n1

Out[9]= True

So, a conceptually easy and quick (5 times faster!) method.

Answer 5

I tried to reproduce this but failed. I'm running Mma 8 on Windows XP; it doesn't seem like the difference should matter, but who knows? I said, successively,

myData = {{1, 2, 3}, {2, 3, 4}, {8, 9, 10}, {1, 1, 1}, {2, 2, 2}}
myFirstList = {9, 9, 9, 9, 9}
mySecondList = {6, 6, 6, 6, 6}
Table[AppendTo[myData[[i]], myFirstList[[i]]], {i, 4}]
Table[AppendTo[myData[[i]], mySecondList[[i]]], {i, 5}]
myData // TableForm

and got (0) no crash, (1) no errors or warnings, and (2) the output I expected. (Note: I used 4 rather than 5 in the limit of the first set of appends, just like in your question, in case that was somehow provoking trouble.)

The Mma documentation claims that AppendTo[a,b] is always equivalent to a=Append[a,b], which suggests that it isn't modifying the list in-place. But I wonder whether maybe AppendTo sometimes does modify the list when it thinks it's safe to do so; then if it thinks it's safe and it isn't, there could be nasty consequences. Do the weird error messages and crashes still happen if you replace AppendTo with Append + ordinary assignment?