Stack Overflow
Stack Overflow

Questions tagged [pruning]

Use pruning for questions related to algorithms used on data structures to facilitate generalization and performance optimization.

Pruning is a general technique to guard against the creation of an algorithm too specific to the training data so that it can be applied to other data sets.

References

193 questions
0
votes
4 answers

Python: How to efficiently select a subset of a list of object instances based on conditions on their attributes?

The minimal demonstrative example what I am trying to do is as follows: Suppose I have a list of instances of a point class, resembling points in the xy-plane, defined by: class point: def __init__(self,x_value, y_value): self.x =…
Britzel
  • 205
  • 3
  • 8
0
votes
1 answer

How to prune a classification decision tree based on classification thresholds

I'm using sklearn to try to train a binary classification decision tree to classify spam vs not spam. My classification threshold is 50% (i.e I'll flag it as Spam if I think there's a 50%+ chance that it is). Assume the classes aren't…
Apollo
  • 11
  • 3
0
votes
0 answers

Remove whole conv2d channels from trained TF model

Given a trained Tensorflow model (using a pruning approach that yields channel masks for conv2d layers), how can I remove whole channels from said model? I created a custom layer that wraps an existing conv2d layer and adds a binary channel mask…
Christian Steinmeyer
  • 824
  • 11
  • 23
0
votes
1 answer

behaviour of the partition filtering on a BigQuery table

Considering a table T with a date based partition named date_part. SELECT * FROM T where date_part="2023-06-06" obviously will show datas from 2023-06-06... If I want datas from 7 days in the past from 2023-06-06, I will do : SELECT * FROM T where…
Sebastien
  • 115
  • 1
  • 11
0
votes
0 answers

Remove tokens from Hugging Face tokenizer and save

In the current implementation, what is the recommended way for removing tokens from a any Hugging Face PreTrainedTokenizer? Simply creating a new vocabulary.txt and loading it with from_pretrained is deprecated and does not scale to all tokenizers.…
Bram Vanroy
  • 27,032
  • 24
  • 137
  • 239
0
votes
0 answers

The test accuracy remains the same after being pruned

I recently tried to run the offical code from the repository "Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration" by Yang He. I found that everything went well when I tried pruning the Resnet(the accuracy…
Jake
  • 1
  • 1
0
votes
0 answers

TypeError: bbox_iou() got an unexpected keyword argument 'x1y1x2y2' in python 3

Epoch GPU_mem box_loss obj_loss cls_loss Instances Size 0%| | 0/203 [00:00 main(opt) File "train.py", line 580, in main …
Ashley Muzuro
  • 11
  • 4
0
votes
0 answers

Tekton Pruning - Separate pipelineruns vs taskruns

I've installed Tekton Operator and Tekton Config as described here: https://tekton.dev/docs/operator/tektonconfig/ In the tektonconfig, it has a block like this: pruner: resources: - taskrun - pipelinerun keep: 3 …
Gary Turner
  • 189
  • 9
0
votes
0 answers

How to prune a transformer?

I am trying to reduce memory and speed up my own fine-tuned transformer. I came across the tutorial for pruning on the huggingface site. I am referring to the following snippet. The trainer.train() is missing, so I added it. It ran without error,…
Jyoti yadav
  • 108
  • 6
0
votes
0 answers

pytorch pruning - saving mask

In each epoch I : prune the model optimize model remove prune mask and save model However, I want to maintain exactly the same mask -> zero out the same weights at the beginning of each epoch. My current code: for module in model.modules(): if…
kaycaborr
  • 49
  • 1
  • 5
0
votes
0 answers

How to query with partition pruning Apache Spark table over ADLS

I have data in ADLS in form of parquet tables "partitioned" by date: gold/database/fact.parquet/YYYY=2022/YYYYMM=202209/YYYYMMDD=20220904/ Data is loaded into these partitions using: file_path_gold =…
DejanS
  • 96
  • 9
0
votes
0 answers

post quantization int8 and prune my model after i trained it using ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8

im trying to inference my model in Arduino 33BLE and to do so i trained my model using ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8 i got a model size with 6.5mb with 88%map@0.5IOU which is nice so i tried to quantize the model using int8 but the…
saeed adeeb
  • 1
  • 1
0
votes
0 answers

How can I fix the issue of unknown layer: PruneLowMagnitude when loading a pruned model?

I have a CNN that I pruned using Tensorflow. Keras and saved it as a .h5 file. Now when I try to load it with pr_model=load_model('Pruned_CNN.h5'), I always get the following error: /tmp/ipykernel_4729/1589964011.py in ----> 1 pmod =…
nechi
  • 53
  • 6
0
votes
1 answer

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'

I am trying to optimize my machine learning model by using weight pruning. But no matter what I do I cant get rid of the error AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign' Here is my code for…
garrett_17
  • 51
  • 5
0
votes
0 answers

How to save memory for iterative pruning pytorch

Here is some MWE of the iterative pruning code I'm trying to run: import torch from torch import nn import numpy as np import torch.nn.utils.prune as prune class myNN(nn.Module): def __init__(self,dims): super(myNN, self).__init__() …
Josemi
  • 314
  • 1
  • 4
  • 20
1 2 3
…
12 13