0

I am using Big SQL from Analytics for Apache Hadoop in Bluemix and would like to look into logs in order to debug (e.g. map reduce job log - usually available under http://my-mapreduce-server.com:19888/jobhistory, bigsql.log from the Big SQL worker nodes). Is there a way in Bluemix to access those logs?

William 'Bill' Wentworth
  • 1,791
  • 11
  • 21
HLeu
  • 13
  • 2
  • What have you searched till now, to solve your issue. – Nehal Dec 15 '15 at 13:16
  • I already followed the offered learnings for the service Analytics for Apache Hadoop in https://www.ng.bluemix.net/docs/services/AnalyticsforHadoop/index.html and searched the Bluemix documentation in https://www.ng.bluemix.net/docs/. I guess it might work with cf logs, already installed the cf CLI on my machine from https://github.com/cloudfoundry/cli/releases but cannot find out how to successfully connect and access my logs. Now, I keep trying based on https://developer.ibm.com/bluemix/2014/10/29/accessing-application-logs-bluemix/. – HLeu Dec 16 '15 at 15:38
  • Meanwhile, I could access Bluemix via cf as described in https://www.ng.bluemix.net/docs/starters/upload_app.html but there are no apps running to get logs from. That leads to more precise questions: – HLeu Dec 17 '15 at 13:23
  • What cf api do I need to connect in order to get access to logs from Bluemix service Analytics for Apache Hadoop Big SQL? What app inside of this API would give access to bigsql.log? What app inside of this API would give access to map reduce job logs? – HLeu Dec 17 '15 at 13:26
  • What format are the Big SQL log files written in bluemix HDFS? Meanwhile I found the Big SQL logs in HDFS in Bluemix https://bi-hadoop-prod-.services.dal.bluemix.net:8443/gateway/default/hdfs/explorer.html#/app-logs/bigsql/logs/application__0001- but they are in a format I cannot read. I am still looking for the map reduce job history logs in bluemix. – HLeu Jan 04 '16 at 14:40

1 Answers1

0

Log files for most IOP components (e.g. MapReduce Job History Log, Resource Manager Log) are accessible from Ambari console's Quick Links. Just navigate to the respective service page. Log files for BigSQL is currently not available. Since the cluster is not hosted as Bluemix appls, they cannot be retrieved using the Bluemix cf command.

louismau
  • 13
  • 2
  • Unfortunately, the quick links do not work in Bluemix. If I e.g. access the HDFS Service quick link for name node UI, it tries to access https://mn01.services.dal.bluemix.net:8443/gateway/default/hdfs/dfshealth.html resulting in a "server not found" page. I replaced mn01.services.dal.bluemix.net with the bi-hadoop-prod-.services.dal.bluemix.net and got a name node UI - but it does not contain access to the job history logs anymore. – HLeu Jan 07 '16 at 08:29
  • At least, it contains webHDFS access. There, I could find but not access a job history log file in /mr-history/done/. But if I try to access it, I get an error "Path does not exist on HDFS or WebHDFS is disabled. Please check your path or enable WebHDFS". – HLeu Jan 07 '16 at 08:47