Stack Overflow
Stack Overflow

Questions tagged [reindex]

459 questions
1
vote
1 answer

Reindex a dataframe with the index of an other dataframe using the sum to fill the values

First create dataframe with regular index, this is the df that I want to resample using th index of df1 df0 = pd.DataFrame(index=pd.date_range(start='2018-10-31 00:17:24', periods=50,freq='1s')) I didn't know how to create a df that has an…
jokerp
  • 157
  • 1
  • 8
1
vote
1 answer

Reindex in Elasticsearch

I've created an index template, and now trying to reindex my logs according to that index template. This is a log just for instance: "_score": null, "_source": { "@timestamp": "2021-05-25T08:38:36", "host": "172.18.20.22", "Level":…
Farid Niasti
  • 43
  • 7
1
vote
1 answer

Re-indexing dataframe on index with duplicate values

DataFrame df1 and df2 are defined as follows: >>> from datetime import datetime >>> df1 = pd.DataFrame({'date':[datetime(2020,1,6), datetime(2020,1,8), datetime(2020,1,8), datetime(2020,1,8), datetime(2020,10,13), datetime(2021,5,27),…
Gerry
  • 606
  • 6
  • 16
1
vote
1 answer

How to reindex columns in two pandas dataframes so that they both have the same columns?

I have two dataframes df1 >>> a b 0 1 4 1 2 5 2 3 6 df2 >>> a c 0 9 8 1 0 1 2 0 0 What I would like to do is create the non-overlapping columns in each dataframe and fill them with zero's. So, because df1 does not contain…
PyRsquared
  • 6,970
  • 11
  • 50
  • 86
1
vote
2 answers

Python/Pandas: filter and organize the rows and columns of a dataframe based on another dataframe

I have two dataframes like df1, df2 below. I would like to: filter df1: that is, remove rows and columns, so that it has the same index elements and columns as df2. The elements within the table for the columns and rows kept should be not be…
StatsScared
  • 517
  • 6
  • 20
1
vote
0 answers

How to reindex data in Solr without deleting the core?

Could someone give me step by step instructions as to how to re-index data from Solr after I changed the schema field types, without deleting the data ? E.g. is there a way of copying the data of a Solr core and reindexing the same data after…
blah
  • 674
  • 3
  • 17
1
vote
1 answer

getting error "ValueError: index must be monotonic"

I have some minute data for the stock SRE, that is a csv file. I have imported it and created a dataframe using the data. I then created a rolling 20 day moving average but I am getting an error. The code is: import pandas as pd ticker = 'SRE' df =…
benito.cano
  • 797
  • 1
  • 8
  • 23
1
vote
2 answers

Avoid timeout in Elasticsearch re-indexing in Java

Below code returned a timeout in client (Elasticsearch Client) when number of records are higher. CompletableFuture future = new CompletableFuture<>(); client.reindexAsync(request, RequestOptions.DEFAULT, new…
Ruchira Gayan Ranaweera
  • 34,993
  • 17
  • 75
  • 115
1
vote
1 answer

Pandas convert quarterly to daily while keeping distinct tickers in mind

I'm working on pulling financial data, in which some is formatted in quarterly and other is daily. My model will need all of it daily, therefore I need that same quarterly value repeated for each day. I've been using this stack post and trying to…
t.pellegrom
  • 313
  • 3
  • 10
1
vote
0 answers

Magento 1.9 site where "SQL ERROR: SQLSTATE[42S22]" keeps showing up after daily reindex (and in other instances)

I own a Magento CE 1.7 based website where we keep getting variations of the following error, usually after the daily morning site reindex: "SQL ERROR: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'main_table.data' in 'field list' SQL…
Marta João
  • 23
  • 3
1
vote
1 answer

Elasticsearch reindex only missing documents

I am trying to reindex an index of 200M of documents from cluster A to cluster B. I used the Reindex API with a remote source and everything worked fine. In the menwhile of my reindex some documents were added into the cluster A so I want to add…
ale_tri
  • 333
  • 4
  • 9
1
vote
2 answers

How to rebuild indexes in an existing table after adding index in MySQL?

I have a table with 50k rows in MySQL and recently added a new index. When I run (which shows the size of each index in my table): SELECT database_name, table_name, index_name, ROUND(stat_value * @@innodb_page_size / 1024 / 1024, 2) size_in_mb FROM…
Nfff3
  • 321
  • 8
  • 24
1
vote
0 answers

Reindex elasticsearch from remote host

I am migrating from an older elasticsearch to newer one. To copy indices from older cluster, I am using the _reindex api provided by elastic search. Now the problem is, my previous clustr is behind a proxy wall and I am getting 401 unauthorised. The…
monster
  • 808
  • 10
  • 23
1
vote
0 answers

Rolling average after filling missing date values

I have a data frame with 'Date','countyFIPS','Topic_label' and 'Score columns'. I want to aggregate data to 3-day level for each county for each topic in 'Topic_label'. There are missing dates in Date columns. I need continuos date column by…
NAS_2339
  • 353
  • 2
  • 13
1
vote
1 answer

Missing Date Values

I have a table of entries as listed below. The table has missing date entries which I want to populate with end of the month missing date values. For the 1st columns, the added rows of missing date entries should be populated with the same values…
Bappa
  • 71
  • 6
1 2 3
…
30 31