Questions tagged [data-pipeline]
168 questions
-2
votes
1 answer
How to automate data pipeline in Palantir Foundry?
I'm new to Palantir Foundery. I need to know how to automate the data pipeline? Is that same if we do the data ingestion to foundry while setting schedule at data sync? OR after all done (such as coding to data transformation etc.) when we see the…
user
- 145
- 1
- 2
- 12
-3
votes
2 answers
combining data from different sources in apache spark
I am exploring apache spark for a project where I want to get data from different sources - database tables (postgres and BigQuery), and text. The data will be processed and fed into another table for analytics. My choice of the programming language…
pg238
- 1,145
- 2
- 15
- 26
-4
votes
1 answer
Add images from disk to a Tensorflow dataset
I am using Tensorflow Datasets' tfds.load function to load my data:
import tensorflow_datasets as tfds
import tensorflow as tf
(raw_train, raw_validation, raw_test), metadata = tfds.load(
'cats_vs_dogs',
split=['train[:80%]',…
Stat Tistician
- 813
- 5
- 17
- 45