Questions tagged [custom-training]

90 questions
2
votes
0 answers

Customizing fit() function in keras leads evaluate() function returning empty list

I am training a VAE using the Keras tutorial: https://keras.io/examples/generative/vae/. This involves creating a VAE class and specifying a custom training procedure, which is described more here:…
Jane Sully
  • 3,137
  • 10
  • 48
  • 87
2
votes
1 answer

Training models interactively in Pytorch

I need to train two models in parallel. Each model has a different activation function with trainable parameters. I want to train model one and model two in the way that the parameters of the activation function from model one (e.g., alpha1) is…
Kevin
  • 145
  • 1
  • 12
1
vote
1 answer

issues with passing command line args to custom training job vertex AI

I am trying to run basic custom training job job = aiplatform.CustomContainerTrainingJob( display_name='testjob-name', container_uri='gcr.io/prj-id/image-name:latest', project=project_id, credentials= credentials, staging_bucket=…
1
vote
1 answer

Vertex pipeline model training component stuck running forever because of metadata issue

I'm attempting to run a Vertex pipeline (custom model training) which I was able to run successfully in a different project. As far as I'm aware, all the pieces of infrastructure (service accounts, buckets, etc.) are identical. The error appears in…
1
vote
1 answer

Transferring Exponential Moving Average (EMA) of Tensorflow custom model to another instance of the model

I have made two instances of the same custom model in Tensorflow 2.9.1 (i.e., model = Model() and ema_model = Model()). During the training of model in a custom loop, I want to calculate its EMA and update the ema_model with these variables. Having…
Cesar
  • 69
  • 1
  • 3
1
vote
1 answer

First tensorflow object detection model - from image to .pb (finally to myriad X blob for Oak-D) - rookie questions

This is my first model, i´m new to python and this is my second post on Stackoverflow so please let me know if there is anything i should elaborate and keep in mind there could be a easy solution to my problems. EDIT - I have found a way to test the…
1
vote
1 answer

Saving accuracy and loss with callback on colab

So im trying to train a model on colab, and it is going to take me roughly 70-72 hr of continues running. I have a free account, so i get kicked due to over-use or inactivity pretty frequently, which means I cant just dump history in a pickle…
1
vote
1 answer

How can I run ImageAI on my GPU and not my CPU

So I am quite new to this I was trying to find answers on google but it is kind of not working. So I am trying to run this library ImageAI library I am able to run it normally on the CPU at least I think it runs on CPU by just calling python…
1
vote
1 answer

how to apply ModelCheckpoint in a custom training loop for tensorflow 2.0?

we can set tf.keras.callbacks.ModelCheckpoint(), then pass a callbacks argument to fit() method to save the best modelcheckpoint, but how to make the same thing in a custom training loop?
1
vote
1 answer

Custom Keras layer with some non-trainable weights

I would like to have a layer in a sequential model that has some fixed, non-trainable weights, sparsely distributed inside the layer. For example: I create a model with few layers model = Sequential() model.add(Dense(units=n_nodes,…
1
vote
2 answers

Pytorch with CUDA throws RuntimeError when using pack_padded_sequence

I am trying to train a BiLSTM-CRF on detecting new NER entities with Pytorch. To do so, I am using a snippet of code derivated from the Pytorch Advanced tutorial. This snippet implements batch training. I followed the READ-ME in order to present…
1
vote
0 answers

How to convert the polygon json into an image mask for image segmentation?

I'm currently working on an image segmentation project and I'm using Labelbox to annotate my training/validation sets. After labelling, the export gives a JSON metadata file. How can I convert this JSON file into masks ? The idea here is to have…
1
vote
1 answer

NER spacy custom trained model not predicting the label properly

Trained NER spacy custom training model using the document https://towardsdatascience.com/train-ner-with-custom-training-data-using-spacy-525ce748fab7 and https://spacy.io/usage/processing-pipelines by sample test case dataset to find the currency…
1
vote
0 answers

Training spaCy NER models on multiple GPUs (not just one)

I am training my NER model using the following code. Start of Code: def train_spacy(nlp, training_data, iterations): if "ner" not in nlp.pipe_names: ner = nlp.create_pipe('ner') nlp.add_pipe("ner", last = True) …
1
vote
0 answers

Training Network with sub networks

I am planning to train the following network from end-to-end. I have two questions: Question 1 I have 4 ground truths. Segmentation of distorted images Parameters Corrected images Segmentation of corrected images My problem is that as it can be…