Stack Overflow
Stack Overflow

Questions tagged [pipelinedb]

Questions related to PipelineDB - A PostgreSQL extension for high-performance time-series aggregation

51 questions
4
votes
1 answer

read only queries on hot standby server fail due to acquiring transaction id

we just backed up a primary onto a hot standby server successfully. however, when we try to query to hot standby, we get the following error: ERROR: cannot assign TransactionIds during recovery I enabled verbose mode and see the error occurs at…
Eric Fonseca
  • 43
  • 4
3
votes
2 answers

TimescaleDB and PipelineDB Integration

(I was going to post this on the DBA SE, but there's no timescale tag.) I'm using TimescaleDB to store a real-time stream of about 500-1000 inserts per second, and need to pass this data to a client application as soon as it comes in. I don't…
The AI Architect
  • 1,887
  • 2
  • 17
  • 24
3
votes
0 answers

PIpelineDB continuous view with mysql foreign data wrapper

I am getting different results when I complete a stream join on a foreign table vs when I join a continuous view to a foreign table. Two queries that I would expect to be the same appear to be different. Does the delay between my local pipeline…
bvk
  • 31
  • 3
2
votes
1 answer

Can we add data to a continuous view in pipelinedb externally

I would like to add data to a specific continuous view. I do not want to feed it through the stream as I want to add it only to this specific view with out disturbing others. I have tried adding rows to the cv_mrel table directly but I was unable…
Sunand
  • 703
  • 4
  • 9
2
votes
3 answers

How to stream output from a continous view in pipelinedb?

I have setup pipelinedb and it works great! I would like to know if its possible to stream data out of a continuous view after the value in the view has been updated? That is, have some external process act on changes to a view. I wish to stream…
tensai
  • 1,628
  • 3
  • 17
  • 22
2
votes
2 answers

How do I list all streams and continuous views in pipelinedb?

In pipelinedb I can't seem to locate a way to list all of the streams and continuous views that I've created. I can back into the CVs by looking for the "mrel" tables that are created but it's kind of clunky. Is there a system table or view I can…
Andy
  • 55
  • 5
2
votes
1 answer

Visualizing time-series from a SQL Database (Postgres)

I am building an app that applies a datascience model on a SQL Database, for sensor metrics. For this purpose I chose PipelineDB (based on Postgres) that enables me to build a Continuous View on my metrics and apply the model to each new line. For…
pbo
  • 218
  • 2
  • 11
2
votes
1 answer

Extracting the counter from the AVG aggregate in PipelineDB

I have this continuous view: =# CREATE CONTINUOUS VIEW v AS SELECT id::int, count(v::int) AS counter, avg(v) AS average FROM mystream GROUP BY id; So I need the average, and the number of elements used to compute this average (counter). is there a…
nfo
  • 314
  • 3
  • 12
1
vote
0 answers

How to add a column to continuous transform in pipelinedb

There are many continuous views, two continuous stream and a continuous transform in the same schema. Now, I want to add a column within varhcar to one of the streams and the transform. Add column to stream is no problem, but when drop continous…
joe
  • 11
  • 4
1
vote
1 answer

Idempotent streams or preventing duplicate rows using PipelineDB

My application produces rotating log files containing multiple application metrics. The log file is rotated once a minute, but each file is still relatively large (over 30MB, with 100ks of rows) I'd like to feed the logs into PipelineDB (running on…
Shmulik Asafi
  • 203
  • 1
  • 6
1
vote
1 answer

PipelineDB: continuous view output stream unexpectedly shows same (old) and (new) values

I am using PipelineDB 0.9.7u3 I played a little with continuous view output stream to find out if I could get a new continuous view just with some the updates. This is my test case. CREATE STREAM stream_test(ticketid text, val int, status text); --…
AnaG
  • 17
  • 1
  • 5
1
vote
1 answer

Pipelinedb: How to group stream data by each N minutes in continuous view

How to group data from pipelinedb's stream by each N minutes in continuous view select? Pipelinedb's stream gets data about the events that comes from a many remote hosts. I need to group this events by type, ip and time intervals in 5 minutes, for…
Zaklinatel
  • 15
  • 6
1
vote
1 answer

unable to import data from CSV into PipelineDB

I have successfully installed PipelineDB. I want to import data into the DB from a CSV file. I have searched a lot but could not find a solution for this. How do I do this?
akansharaut
  • 11
  • 6
1
vote
1 answer

pipelinedb continuous views do not work

First I go and create stream: CREATE STREAM raw_searches_total ( stamp text, searches integer ); I see: CREATE STREAM Fine. Then I create continuous view as follows: CREATE CONTINUOUS VIEW searches_total_10min AS SELECT substring(stamp from…
Fancy John
  • 38,140
  • 3
  • 27
  • 27
0
votes
1 answer

Try parsing data types in PipelineDB and streaming errors to failing text table?

We were using Pipeline DB to receive data into a streaming table, and in two streaming views, in one view, filter out records that would fail typecasting validataion errors, and in the other view, filter in the records that failed typecasting…
Mark
  • 538
  • 6
  • 17
1
2 3 4