using an Autoencoder for dimensionality reduction

Question

i have a data set that contains a ton of vector where each vector has 21300 values, naturally i want to reduce the dimension of each vector i.e compress the vectors

my data set is not split into training and testing datasets because i want all the vectors to be compressed, i have already tried to fit the auto encoder with the first vector in my data set and then proceeded to use the resulting encoder to compress the remaining vectors, however all the remaining ended up looking very similar to the first vector,my question is how can i go about compressing these vectors?

I’m voting to close this question because it is not about programming as defined in the [help] but about ML theory and/or methodology - please see the intro and NOTE in https://stackoverflow.com/tags/machine-learning/info — desertnaut, May 07 '22 at 21:13

score 0 · Accepted Answer · answered May 07 '22 at 17:01

0

Build the autoencoder model using all the data not just the first vector.

Check here for an example in python.

answered May 07 '22 at 17:01

joaopfg

1,227
2
9
18

and then i use the encoder on the same data again right? – Barry Mccockinner May 07 '22 at 17:08
Yes, that's it =) – joaopfg May 07 '22 at 17:10

using an Autoencoder for dimensionality reduction

1 Answers1