Stack Overflow
Stack Overflow

Questions tagged [multimodal]

32 questions
0
votes
3 answers

Neural Network parameters are not being updated

I tried to train a multi modal model on the 2d heat equation. CONTEXT : The best I have is a CNN with a 5*5 kernel that is optimised to output temperature maps with a given diffusion coefficient. Now I try to give the model other coefficient and…
Antoine Grand
  • 11
  • 2
0
votes
0 answers

Is there a way to tokenize sentences with Longformer?

I have forked the Multimodal Transformers package and created a new version with Longformer support here --> https://github.com/jtfields/Multimodal-Toolkit-Longformer/tree/master. Georgian.io maintains the Multimodal Transformers package and here…
John Fields
  • 21
  • 4
0
votes
0 answers

Change Keras Distiller() class so that the student and teacher model can have two different inputs

I am doing my Master's thesis in Multimodal Emotion Recognition, more precisely, I want to apply knowledge distillation from a multimodal teacher model to an unimodal student model. I am using Keras Distiller() class…
rodrigo.cgcdl
  • 1
  • 1
0
votes
0 answers

Is it possible to parallelise a call in the training loop of a multimodal deep learning model

In the forward pass of my network, I encode two modalities, concatenate them, and use a decoder to produce a final output. Here is a sample code: out_a = self.encoder_a(input_a) out_v = self.encoder_v(input_v) out = torch.cat((out_a, out_v),…
Theo Patron
  • 23
  • 6
0
votes
0 answers

How to load multiple, varying references for BLEU metrics with DataLoader?

I have an Image Captioning dataset, where each sample is composed by an image and a list of captions. Each sample has one or more captions The number of captions can be different for each sample. Here's a visual example: I am using PyTorch and I…
Ciodar
  • 103
  • 1
  • 5
0
votes
0 answers

Keras concatenated model: ValueError: Data cardinality is ambiguous

I am creating a concatenated model using keras. For now, I am keeping it simple, using only Dense layers and without any kind of hyperparameters optimization. My model should be able to get data from two different datasets, with a different number…
Foxtrot_Romeo
  • 16
  • 3
0
votes
0 answers

Why is this error in tensorflow js when model predict?

If I execute my code, I get the below error error: Uncaught (in promise) TypeError: model.predict is not a function My code const model = tf.loadLayersModel('./tf_js/model.json'); const video1 =…
yeon
  • 5
  • 2
0
votes
0 answers

Downloading Fakeddit dataset and storing in drive through colab

I am trying to download the Fakeddit dataset (multimodal) through colab notebook, but getting the following error when running !python image_downloader.py: usage: image_downloader.py [-h] type image_downloader.py: error: the following arguments are…
Kamonashish Saha
  • 1
0
votes
1 answer

Combining text and tabular data in PyTorch for classification model

I have a dataset that consists of customers and their product purchases for an ecommerce company that sells clothes.  Along with this data, I have application logs that show the customer’s interactions on the site. The data looks something like…
Negative Correlation
  • 813
  • 1
  • 11
  • 26
0
votes
1 answer

Modelling and fitting bi-modal lognormal distributions in a loop using lmfit

I have been spending FAR too much time trying to figure this out - so time to seek help. I am attempting to use lmfit to fit two lognormals (a and c) as well as the sum of these two lognormals (a+c) to a size distribution. Mode a centers around…
GingerMonster
  • 33
  • 5
0
votes
0 answers

Image text retrieval evaluation metric

I am currently working on text image retrieval where basically, given a test query, I should be able to retrieve all the related images. For example- 'children playing in park', should retrieve all relevant images. I do know the evaluation metric to…
Nidutt
  • 185
  • 2
  • 17
0
votes
1 answer

Multi-start in OpenMDAO

Is there any straightforward way to use a multi-start option for solving optimization problems in OpenMDAO? I am currently using the SLSQP driver of ScipyOptimize, and would like to start the search from multiple initial points (either in a sequence…
Kasia
  • 105
  • 6
0
votes
1 answer

Deep Learning - generate patches for 3D multimodal data

I have chosen the problem of Semantic Segmentation of Brain Tumors using Deep Learning. I am using the BRATS2015 dataset. It has 274 patient MRI scans each of size 240x240x155. There are four modalities for each patient (T1, T2, T1c, FLAIR). So I am…
Pranjal
  • 8,083
  • 3
  • 8
  • 13
0
votes
0 answers

How to create multimodal data set?

I want to make music recommendation system based on multimodal dataset. Ive been looking for publicly available dataset but I couldn't find it anywhere . I want to know where will I get multimodal dataset for music or book ? I watched many YOUTUBE…
user12613694
0
votes
1 answer

ValueError: Input 0 of layer conv1_pad is incompatible with the layer: expected ndim=4, found ndim=2. Full shape received: [None, 260]

I'm getting an error for Multi-modal problem. Input shape: img input:- (3740, 150, 150, 3), word input:- (3740, 260) where 3740 is the number of samples. Here have attached model as a function where build_img_encoder describes IMG encoder model and…
Bhuvan S
  • 213
  • 1
  • 4
  • 10
1
2
3