Stack Overflow
Stack Overflow

Questions tagged [large-scale]

34 questions
1
vote
1 answer

What is the best approach for upserting large number of rows into a single table?

Im working on a product that involves large number of upsert operations into a single table. We are dealing with a time-based data and using timescaledb hypertables with 7 days chunk interval size. we have concurrent tasks that upserts data into a…
Ben Ben Sasson
  • 103
  • 2
  • 8
1
vote
1 answer

MySQL Performance of one vs. many tables

I know that MySQL usually handles tables with many rows well. However, I currently face a setting where one table will be read and written by multiple users (around 10) at the same time and it is quite possible that the table will contain 10 billion…
IceFire
  • 4,016
  • 2
  • 31
  • 51
1
vote
1 answer

How can I resolve an ILP (integral linear programming) model whose system matrix is 1187550*391275?

I tried lingo software. More than 100 hours passed, I haven't get the result. I tried MATLAB software. When I run this code "A=zeros (1187550,391275)" it occurs error: The biggest variable values beyond the program allows So, is there some good ways…
杜昱萱
  • 23
  • 5
1
vote
2 answers

How to scale an algorithm/service/system with multiple machines?

I had some interviews recently and it's quite normal to be asked some scale problems. For example, you have a long list of words(dict) and list of characters as the inputs, design an algorithm to find out a shortest word which in dict contains all…
cangoal
  • 63
  • 1
  • 7
1
vote
0 answers

clustering tags in large scale user-tag matrix in matlab

I am trying to cluster a set of tags in huge sparse matrix of user-tag in matlab, say: 35000*74000 with 99% sparsity. I probably have two choices: 1: Making Co-occurrence matrix of tags and apply clustering on Tag-Tag co-occurrence matrix. 2:…
Morteza R
  • 7
  • 4
1
vote
1 answer

How to create summary tables efficiently

During a month a process inserts a large number of rows in some database tables ~1M. This happens daily and the whole process lasts ~40mins. That is fine. I created some "summary tables" from these inserts so as to query the data fast. This works…
Jim
  • 18,826
  • 34
  • 135
  • 254
1
vote
2 answers

How to train neural network incrementally in Matlab? and iteratively combine them

I have very big train set so that Matlab. And I need to do large scale training. Is it possible to split the training set into parts and iteratively train the network and on each iteration update the "net" instead of over-writing to it? The code…
alper
  • 2,919
  • 9
  • 53
  • 102
1
vote
3 answers

Unit-testing and Integration Testing for Large or Medium-Complexity System Models in SIMULINK

PROBLEM I have a fairly large Hydro-Pneumatic system model composed of roughly 20-25 different subsystems. Each of the subsystem is then composed of digital logic, edge delay blocks, and gateway to external output ports (real-world outputs).…
ha9u63a7
  • 6,233
  • 16
  • 73
  • 108
0
votes
0 answers

Using walberla framework

I have a code for Latice model written in C. I want to link my code with waLBerla library which is written by C++. Unfortunately, there is no tuto on internet explaining or giving examples how to use walberla for large-scale simulations. Could…
lucas mure
  • 1
  • 1
0
votes
0 answers

Pyscipopt (in Google Collab) does not show/print any output for the large-size network optimization problem in Google Collab

I am working on a non-linear network-optimization problem. When I run a large problem instance on the lab high-performance computers (imposing the time limit of 1 or 2 hours), those computers give me the output when the time is exhausted. I wanted…
Lidiia S
  • 11
  • 1
0
votes
1 answer

Can OMNET with INET4.4 be used for large-scale simulations?

everybody! I want to simulate a large-scale (more than thousand nodes) satellite network,e.g.,starlink_p1 constellation with 1584 satellites. We model satellite as router, and each router connects to a standhost for generating traffic. We use the…
yang
  • 11
  • 2
0
votes
1 answer

Changing the "page" in MarionetteJS

I'm currently working on a large scale Backbone application (using marionetteJS) and having some architecture troubles with the best practices/patterns involved in switching views or changing the "page". Marionette's layouts & regions provide some…
Tom
  • 430
  • 4
  • 14
0
votes
1 answer

How to create module based application in J2EE for the large scale application?

We are going to develop large scale application by using J2EE. The application has contains many number of modules and few modules are not directly communicate with the client.(ie it will be processing in separate thread like SNMP Listener, job…
Murali
  • 341
  • 2
  • 7
  • 24
0
votes
1 answer

Large scale recommender

I am dealing with a recommendation problem, which involves 3 million users and 500,000 products. The purpose of the recommendation is to recommend 5-10 more products to the particular user when visiting a given product. We finished the real-time…
user1841342
  • 63
  • 1
  • 4
-1
votes
1 answer

Large Scale Data Logistic Regression

I have following input data: head(data1) VarA VarB VarC VarD VarE VarG VarH VarI 2016-06-01 09:30:05 14.2 31228 ABCD IS Equity 1 139 192 23 2016-06-01 09:30:07 14.2 31128 ABCD IS Equity 0 0 0 …
Zico
  • 185
  • 12
1
2
3