Stack Overflow
Stack Overflow

Questions tagged [airbyte]

25 questions
0
votes
0 answers

Different timezone when Airbite integrate SQL Server 2017 to Bigquery

I am integrate data from SQL Server 2017 with CDC enable to BigQuery. I found that first full updated data seem to have a different timezone with later incrementally updated data, so that the first updated data will not be updated anymore. Just like…
UUBOY scy
  • 1
0
votes
0 answers

How to perform pre-processing steps before data being read by Airbyte S3 connector?

I am trying to use S3 connector from Airbyte to send data from AWS S3 (csv files) to my Postgres database. When setting up and testing the connector everything seems ok, but the problem comes when trying to add a destination. When I click on add…
brenda
  • 656
  • 8
  • 24
0
votes
0 answers

Column type changes after table is integrated from SQL Server to BigQuery by Airbyte

I am using Airbyte to integrate SQL Server 2017 table to BigQuery, and I found that some columns' type are different from it were in SQL Server. Original "date" column is displayed as "String" in BigQuery. I have no idea how this happen and how to…
UUBOY scy
  • 1
0
votes
0 answers

Integrate big SQL Server tables with Airbyte

I built Airbyte on a public GKE cluster to integrate tables in SQL Server 2017 to BigQuery. This is what I configured for the SQL Server source: Replication Method -> Logical Replication (CDC) Data to Synk -> Existing and new Initial Snapshot…
UUBOY scy
  • 1
0
votes
0 answers

Unable to establish airbyte connection from MongoDB to S3 using Octavia CLI

I used octavia list workspace connections to get the destination and source ids already configured using the airbyte UI. I am trying to generate the same connection (with same source and destination IDs) with different source and destination names…
Vaidehi Khalpada
  • 1
  • 1
0
votes
1 answer

airflow airbyte operator - module not found

Context I'm using Airflow hosted on Google kubernetes engine, I've added apache-airflow-providers-airbyte 3.2.0 to the requirements.txt which is installed and appears on the Airflow - Admin > Providers" UI along with the default…
IOIOIOIOIOIOI
  • 277
  • 3
  • 15
0
votes
1 answer

Airbyte Missing Columns in tables

I am considering Airbyte as a replacement for singer-io because of its user-friendly UI and code reduction features. However, facing an issue where Airbyte is unable to detect certain columns in our source (Atlas hosted on AWS), which is a…
Manjul Tripathi
  • 1
0
votes
0 answers

Only pick selected fields when ingesting mongodb data with airbyte

I need to ingest data with airbyte from a mongodb collection into bigquery. The documents contain hundreds of fields, some of which we would prefer not to ingest for security reasons. Is there an easy way to collect only specific fields from the…
greg hor
  • 682
  • 6
  • 17
0
votes
1 answer

ERROR: extension "pgoutput" is not available while trying to use it with airbyte to replicate the data from PostgreSQL in real-time using CDC

I am planning to use airbyte to replicate data from my PostgreSQL database to Elasticsearch in real-time using CDC. According to the documentation The pgoutput plug-in is always present, meaning that no additional libraries must be installed, and…
pd farhad
  • 6,352
  • 2
  • 28
  • 47
-1
votes
1 answer

The command "./run-ab-platform.sh" in Windows PowerShell does not execute

I have been trying to install airbyte in Docker Desktop using the following instructions: git clone https://github.com/airbytehq/airbyte.git cd airbyte ./run-ab-platform.sh The problem is: the command "./run-ab-platform.sh" does not execute. It…
RaulLeite
  • 1
  • 1
1
2