0

i'm running through a simple and useless toy using PCF on azure, trying to create and run the stream 'time | log'

i successfully get SCDF started, and the stream created, but when i try to deploy the stream, SCDF creates two (cf) apps that won't run - they exist as far as cf-apps is concerned

○ → cf apps
Getting apps in org tess / space tess as admin...
OK

name                              requested state   instances   memory   disk   urls
yascdf-server                     started           1/1         2G       2G     yascdf-server.apps.cf.tess.info
yascdf-server-LE7xs4r-tess-log    stopped           0/1         512M     2G     yascdf-server-LE7xs4r-tess-log.apps.cf.tess.info
yascdf-server-LE7xs4r-tess-time   stopped           0/1         512M     2G     yascdf-server-LE7xs4r-tess-time.apps.cf.tess.info

if i try to view the logs for either, nothing ever returns. but the logs in apps manager look like this:

2017-08-10T10:24:42.147-04:00 [API/0] [OUT] Created app with guid de8fee78-0902-4df7-a7ae-bba8a7710dca
2017-08-10T10:24:43.314-04:00 [API/0] [OUT] Updated app with guid de8fee78-0902-4df7-a7ae-bba8a7710dca ({"route"=>"97e1d26b-d950-479e-b9df-fe1f3b0c8a74", :verb=>"add", :relation=>"routes", :related_guid=>"97e1d26b-d950-479e-b9df-fe1f3b0c8a74"})

the routes don't work:

404 Not Found: Requested route ('yascdf-server-LE7xs4r-tess-log.apps.cf.tess.info') does not exist.

and trying to (re)start the route i get:

○ → cf start yascdf-server-LE7xs4r-tess-log 
Starting app yascdf-server-LE7xs4r-tess-log in org tess / space tess as admin...
Staging app and tracing logs...
The app package is invalid: bits have not been uploaded
FAILED

here's the SCDF shell stuff i ran, if this helps:

server-unknown:>dataflow config server http://yascdf-server.apps.cf.tess.info/
Successfully targeted http://yascdf-server.apps.cf.cfpush.info/
dataflow:>app import --uri http://.../1-0-4-GA-stream-applications-rabbit-maven
Successfully registered applications: [<chop>]
dataflow:>stream create tess --definition "time | log"
Created new stream 'tess'
dataflow:>stream deploy tess
Deployment request has been sent for stream 'tess'
dataflow:>

anyone know what's going on here? i'd be grateful for a nudge...

Spring Cloud Data Flow: Server 1.2.3 (using built spring-cloud-dataflow-server-cloudfoundry-1.2.3.BUILD-SNAPSHOT.jar)

Spring Cloud Data Flow: Shell 1.2.3 (using downloaded spring-cloud-dataflow-shell-1.2.3.RELEASE.jar)

Deployment Environment PCF v1.11.6 (on Azure) pcf dev v0.26.0 (on mac)

App Starters http://bit-dot-ly/1-0-4-GA-stream-applications-rabbit-maven

Logs stream deploy log

Wings_Of_Fire Equinox_AJ
  • 1
  • 2
  • 1
    If you could update the post with all the relevant details from this [wiki page](https://github.com/spring-cloud/spring-cloud-dataflow/wiki/Reporting-Issues), that'd help with troubleshooting. Also, please attach the server logs of the deployment event. – Sabby Anandan Aug 10 '17 at 14:39
  • fwiw i get the same behavior in pcf dev, so i'm pretty sure it's something dumb i'm missing – Wings_Of_Fire Equinox_AJ Aug 10 '17 at 14:44
  • The `app import` command looks a little odd. We provide bit ly links for all the GA releases. If the applications aren't accessible by the server, it will fail to resolve and deploy at runtime. A quick way to verify that is to review the docs for any app. Example: `app info source:time` - if this fails, you're still not able to resolve the apps. – Sabby Anandan Aug 10 '17 at 14:49
  • Also, it is not clear what version of the server are you using. If you can attach the logs, we can review what else might be wrong. One other thing - it'd be better that you use the [latest GA release of the apps](http://cloud.spring.io/spring-cloud-stream-app-starters/#stream-app-starters-and-spring-cloud-data-flow-). You're on 1.0.4. The latest is at 1.2.0 (aka: Bacon release-train). – Sabby Anandan Aug 10 '17 at 14:51
  • the app import was fine, but stack-overflow groused at my using a bit.ly URL, so i used ... - everything imported – Wings_Of_Fire Equinox_AJ Aug 10 '17 at 15:34
  • There seem to be OOM errors in the logs. How much memory is the server setup with? (we recommend 2G for the server, but it is not entirely used). Also, is this the same error you're seeing on PCFDev, too? If not, let's review that as well. – Sabby Anandan Aug 10 '17 at 17:05
  • Never mind, I see it on the table in the original post. It is set to 2G. – Sabby Anandan Aug 10 '17 at 17:07
  • i saw that as well (since you made me look at the logs :). i upped memory to 4GB and got same error fwiw. – Wings_Of_Fire Equinox_AJ Aug 10 '17 at 17:35
  • While discussing this with the java-buildpack (jbp) lead, we were wondering whether you are using the jbp-4.0 version? Could you confirm that? The memory calculator is rewritten on this release, so there's an allocation issue that needs to be fixed. This doesn't answer why you're seeing failures in pcfdev, though - we need to review the logs for this. – Sabby Anandan Aug 10 '17 at 17:50
  • java-buildpack-offline-v4.4.zip and java-buildpack-v4.4.zip are installed. and i'm sorry, i know i owe you the pcf-dev logs too – Wings_Of_Fire Equinox_AJ Aug 10 '17 at 18:10
  • Thanks for the confirmation! SCDF is currently _not_ vetted/compatible with JBP4 yet. I'd recommend downgrading to JBP 3.19 (latest on this 3.x line) for SCDF. Once you try it out, we can share the outcome as an answer to this thread. In the meantime, the JBP team will review how to resolve the memory allocation problem in JBP4. – Sabby Anandan Aug 10 '17 at 18:45
  • JBP 3.19 works, as you predicted. thanks. let me circle back on the pcf-dev mention and see what the error was - i have a feeling i may have jumped the gun there, since i was having trouble with the azure cluster – Wings_Of_Fire Equinox_AJ Aug 10 '17 at 19:31

1 Answers1

0

It has been identified that java-buildpack 4.4 (JBP4) was used by OP and by running SCDF against this version, there is an issue with memory allocation in reactor-netty (used by JBP4 internally), which causes the out-of-memory error. The reactor team is addressing this issue in the upcoming 0.6.5 release. JBP4 will adapt to it eventually.

With all this said, SCDF is still not compatible with JPB4. It is recommended to downgrade to JPB 3.19 or latest in this release line instead.

Sabby Anandan
  • 5,636
  • 2
  • 12
  • 21