I am looking at using some Dataflow programming techniques in a clojure program but I am having difficulty in finding much information from projects using Java, C#, or other mainstream languages that have used such techniques in the real world. I would be grateful to hear if anyone has any expereinces they could share regarding this.
Asked
Active
Viewed 2,399 times
6
-
Yes, Labview is one product I really like the look of. Which languages does it support? – yazz.com Dec 30 '10 at 21:12
3 Answers
5
Here, we are! We've made... (quotation is from one of my older post):
We've designed and implemented a DF server for our automation project (dispatcher, component iterface, a bunch of components, DF language, DF compiler, UI). It is written in bare C++, and runs on several Unix-like systems (Linux x86, MIPS, avr32 etc., Mac OSX). It lacks several features, e.g. sophisticated flow control, complex thread control (there is only a not too advanced component for it), so it is just a prototype, even it works. We're now working on a full-featured server. We've learnt lot during implementing and using the prototype.
Also, we'll make a visual editor some day.
There're dataflow systems wich don't even mention dataflow approach:
- SynthEdit: http://www.synthedit.com/ - It's an audio related framework and component set for creating VST plugins
- TinyOS: http://www.tinyos.net/ - It's an embedded operating system/framework
- Digital synthetisers/samplers are dataflow systems, programmed - supposedly - in C or some parts in Assembly, check my answer to another post about some examples.
- Quartz Composer, a graphic magic tool for Mac,
- Blender has dataflow subsystem for image composing.
Writing a dataflow system is not rocket science. Here's my older post about the basics of dataflow framework.
The term dataflow is wide. There are realtime synchronous dataflow systems, like synthetisers and samplers, there are asynchronous ones, like our home aut. system (the system is in idle unless the user presses a button or a timer runs out), and there're even different architectures, like spreadsheets or make.
Wanna reading more about dataflow programming? Read J. Paul Morrison's site and book.
-
I've created a dataflow editor concept using SGV/Raphaël https://github.com/ern0/dataflow-editor-concept - demo is also available – ern0 Jun 02 '17 at 08:04
2
Pervasive DataRush is a framework for parallel dataflow programming for any JVM language, including Clojure.
Pervasive DataRush uses a dataflow architecture. The architecture implements a program that executes as a graph of computation nodes interconnected by dataflow queues. The nodes use the queues to share data. As the data is streaming, only data required by any active operation needs to be in memory at any given time, allowing very large data sets to be analyzed. Besides offering the potential for scaling to problems larger than available memory, dataflow graphs exploit multiple forms of parallelism.
Customers are using DataRush for big data analytics and data preparation (ETL).
joedubin
- 21
- 2
0
We've made another one: a collaborative spreadsheet with MySQL/PHP backend and AJAX frontend. The software is in beta state, documentation is under construction.
ern0
- 3,074
- 25
- 40
-
Not yet. I'm writing the documantation now, and there're minor issues. Unfortunatelly, it requires a "background job", written in PHP, which must be started in every 1 secs, which most hosting services does not support (incl. our one) so it's a challenge to show it on internet, but we *must* solve it. – ern0 Mar 18 '11 at 11:58