Stack Overflow
Stack Overflow

Questions tagged [dimension-reduction]

38 questions
1
vote
1 answer

Employing correlation coefficients (Pearson) for dimension reduction [Python]

I'm utilizing this answer in order to find the correlation coefficients greater than a given limit, f, in a matrix (ndarray) that is of shape (29421, 11001) [i.e. 29,421 rows and 11,001 columns]. I've adapted the code as follows (the random bit…
user3684314
  • 707
  • 11
  • 31
1
vote
1 answer

Find neighbours outside of 2d grid which is reduced into a 1d array

I have a two dimensional grid where width and height are always the same. [0][1][2] [3][4][5] [6][7][8] I reduced it's data source into a one-dimensional array. [0][1][2][3][4][5][6][7][8] Access of elements works, but here comes the tricky part:…
Jankapunkt
  • 8,128
  • 4
  • 30
  • 59
1
vote
0 answers

SSVD for dimensional reduction +Clustering

I have run the ssvd by mahout to apply LSA (Latent semantic analysis). I have text documents each contains many features(from 100 to 2000 terms). I would like to use LSA on the documents to get the top terms or phrases which appear together…
Darsh Sofyan
  • 11
  • 2
1
vote
1 answer

Dimension reduction for logical arrays

I have measurements of 5 devices at two different points of time. A measurement basically consists of an array of ones and zeros corresponding to a bit value at the corresponding location: whos measurement1_dev1_time1 Name …
Richard Laurant
  • 647
  • 1
  • 8
  • 21
1
vote
0 answers

Activity Recognition - Dimension reduction for continuous HMMs

I am a novice at HMMs but I have tried to build a code using Jahmm for the UCI Human Activity Recognition data set. The data set has 561 features and 7352 rows, and also includes the xyz inertial values of both the accelerometer and gyroscope, and…
user3416268
  • 83
  • 5
0
votes
0 answers

convergence issues with dynamic factor analysis

I am using the R package MARSS to run a dynamic factor analysis. I have 8 timeseries and all of the time series have at least 1 NA value (range 1-20 of 50 years/timeseries). When I ran my model with just 23 years of data (the years where all…
tnt
  • 1,149
  • 14
  • 24
0
votes
0 answers

Brain Teaser: Algorithm for constructing fewest categorical vectors from overlapping binary variables

I have a dataset containing the dates and post_id for which writer of the post received a reaction: userid date postid u1 d1 a u1 d1 b u1 d1 c u1 d2 a u1 d2 b u1 d2 d u1 d3 b u1 d3 …
ozaroos
  • 61
  • 4
0
votes
1 answer

goodness of fit of umap

I'm given a 20122-dim matrix, each column is a 20-element vector. Now I apply umap to the matrix and get a 2122 matrix and a umap plot. How to measure the goodness of fit of this umap model? Is there any standard way to do that?
Josie G
  • 145
  • 5
0
votes
2 answers

numpy convert N-D array to a list of (N-1)D array without loop

how to conver a numpy nd array to a list of (n-1)d withou loop assume we have a array 3d numpy array arr3d = np.random.randint(1, 100, size=7 * 3 * 6).reshape((7, 3, 6)) with loop (slow solution): arr_what_i_want= [] for i in range(7): arr2d…
guzuomuse
  • 11
  • 4
0
votes
1 answer

The lstm autoencoder does not use the full dimensions of the latent space for dimension reduction

I am trying to train a lstm autoencoder to convert the input space to a latent space and then visualize it, and I hope to find some interesting patterns in the latent space. The input is data from 9 sensors. They are to be transformed into a…
Mohannad Assad
  • 1
0
votes
0 answers

Why is it ok to remove variables with low variance from a dataset

It is a common practice in data analysis to remove features (independent variables) with low variance for dimensionality reduction, with the justification that a feature with low variance cannot explain much of the variance in the response variable…
Amnon
  • 195
  • 3
  • 12
0
votes
1 answer

How to compute/extract the residual variance from an Isomap [vegan] model in R

I am currently trying to understand how Isomap results will differ from PCA and MDS and if they are more suited for my data. For this I started to work with the isomap function provided by vegan in R using the BCI dataset and their basic example…
Philipp Maurischat
  • 1
  • 2
0
votes
1 answer

Mapping a numerical function with two inputs onto one with one input

I‘m quite bad at programming, so please bear with me. I‘m not even sure what the concept I need right now is called, so i don’t know what to google for or write in the title of this post. My issue is, I numerically integrated a function on…
Zyrax
  • 1
0
votes
1 answer

After dimension reduction using SVD, what is the meaning of the reduced dimsnion?

I don't understand SVD at algorithm level. But I do know that people use it to reduce dimension. I have two co-occurrence matrix (dimension is 50,000 words by 50,000 words) that store information on the number of times any words were used together.…
Ying Li
  • 1
  • 2
0
votes
1 answer

How to use QR-Decomposition to reduce the dimension of a dataset?

Let A be a matrix of dimension m×n, representing the original data set. The QR decomposition, [Q, R] = qr (A) produces: Upper triangular matrix R of the same dimension as A Unitary matrix Q so that A = Q*R If [m, n] = size (A), then Q is m-by-m…
shdotcom
  • 125
  • 1
  • 1
  • 11
1
2
3