Stack Overflow
Stack Overflow

Questions tagged [mini-batch]

Use mini-batch when working with neural networks and in particular with mini-batch stochastic gradient descent.

A mini-batch is a subset of the input data (batch) broken up into many smaller pieces (mini-batch) that are used to train the neural network.

References

84 questions
2
votes
0 answers

Collecting a mini-batch one by one efficiently with Keras

I have to build a network that receives images of different sizes. I don't want to resize or crop so i am using a fully convolutional network. The problem is that i can't pre-create minibatchs because of the different sizes of each image. One…
Alon
  • 293
  • 2
  • 6
1
vote
1 answer

Databricks Autoloader is getting stuck and does not pass to the next batch

I have a simple job scheduled every 5 min. Basically it listens to cloudfiles on storage account and writes them into delta table, extremely simple. The code is something like this: df = (spark .readStream .format("cloudFiles") …
Maksym
  • 11
  • 3
1
vote
1 answer

mini batch backpropagation clarification

Read through a lot of articles and now I am in my braindrain end, and need a fresh perspective on the concept of mini batch. I am new to machine learning and would appreciate any advice on whether my process is correct. Here is my premise: I have a…
axia_so2
  • 27
  • 6
1
vote
0 answers

How to use continuation learning for XGBoost with MultiOutputRegressor?

I'm trying to train multi-output XGBoost regression model using the continuation training, but I get this error: TypeError: ('Unknown type:', MultiOutputRegressor(estimator=XGBRegressor(base_score=None, booster=None) My code is as follows: params…
LeparaLaMapara
  • 61
  • 6
1
vote
0 answers

How to distribute different models to multiple nodes with Pytorch?

I had a pytorch task, which worked with DP: One same network copied to multiple GPUs sharing the same weights, but each copy receives a different data batch, so it speeds up training by increasing equivalent batch size. But now I hope to introduce…
Jim Wang
  • 421
  • 6
  • 17
1
vote
0 answers

How to implement momentum and decay correctly - SGD

I am trying to apply momentum and decay to a mini-batch SGD: What would be the right way to update my weights, I get weird results as soon as decay is set.. import numpy as np def _mini_batch(self,X,y,batch_size): # sack data for shuffle -…
Bennimi
  • 416
  • 5
  • 14
1
vote
1 answer

Manual mini-batch generation for PyTorch Geometric

Currently I have pytorch tensors with shape (batch_size, height, width, channel_size) and I want to convert it to a mini-batch described here. My current idea is to convert each example from tensor representation to graph representation separately…
Original-Thunderbird
  • 11
  • 1
1
vote
1 answer

Linear Regression Mini-batch Gradient Descent in python: split training and validation data by batch size

I was trying to split training and validation data by batch size, and finding the validation rmse. The size for training feature data is 11000. The size for validation feature data is 2750. However, when I use the zip function in for loop. It limits…
Tony
  • 11
  • 2
1
vote
1 answer

Slicing for creating Mini-batches

I intend to create mini-batches for my deep learning neural network program, from a training set consisting 'm' number of examples. I have tried: # First Shuffle (X, Y) permutation = list(np.random.permutation(m)) shuffled_X = X[:,…
Jatin Singh Bhati
  • 133
  • 1
  • 3
  • 14
1
vote
0 answers

Should sequence nature of input data within a mini-batch be maintained?

Suppose a multivariate time series prediction problem for the following data rows = 2000 cols = 6 data = np.arange(int(rows*cols)).reshape(-1,rows).transpose() print(data[0:20]) print('\n {} \n'.format(data.shape)) print(data[-20:]) which prints…
Ather Cheema
  • 416
  • 2
  • 14
1
vote
0 answers

loss.backward() with minibatch in pytorch

I came across this code online and I was wondering if I interpreted it correctly. Below is a part of a gradient descent process. full code available through the link https://jovian.ml/aakashns/03-logistic-regression. My question is as followed:…
anyang_peng
  • 11
  • 4
1
vote
1 answer

Do I calculate one loss per mini batch or one loss per entry in mini batch in deep reinforcement learning?

New to neural network and Pytorch. I have 300 replay memories in each mini batch. I've seen people calculate one loss for the 300 replay memories, but it doesn't really make sense to me. The 300 replay memories are from very different game states…
Andy Dong
  • 11
  • 1
1
vote
1 answer

Difference betweeen Mini Batch K-Means and Sequential/online KMeans

I am trying out examples of K-Means and its variants using scikit-learn library sklearn.cluster. What is the difference between minibatch K-Means clustering and online/sequential K-Means clustering ? I could not find the implementation of online…
Nick
  • 47
  • 1
  • 8
1
vote
0 answers

How to generate custom mini-batches using Tensorflow 2.0, such as those in the paper "In defense of the triplet loss"?

I want to implement a custom mini-batch generator in Tensorflow 2.0 using tf.data.Dataset API. Concretely, I have image data, 100 classes with ~200 examples each. For each mini-batch, I want to randomly sample P classes, and K images from each…
Čedomir Lukić
  • 31
  • 2
1
vote
0 answers

How to implement mini-batch gradient descent for maximum likelihood estimation python?

Currently, I have some code written that finds the combination of parameters that maximizes the log-likelihood function with some field data. The model now randomly selects the parameters out of a uniform distribution and then selects the best…
mj1496
  • 61
  • 1
  • 9
1
2
3 4 5 6