0
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sam/Documents/freenet/nifi-automation/src/compose.py", line 122, in <module>
    compose_services(env_config, types, NIFI_VERSION, False, bench)
  File "/home/sam/Documents/freenet/nifi-automation/src/compose.py", line 11, in compose_services
    pg = ProcessorGroups(NIFI_VERSION)
  File "/home/sam/Documents/freenet/nifi-automation/src/components/processor_group.py", line 9, in __init__
    processor_groups = nipyapi.canvas.list_all_process_groups(pg_id='root')
  File "/home/sam/Documents/freenet/nifi-automation/venv/lib/python3.6/site-packages/nipyapi/canvas.py", line 178, in list_all_process_groups
    root_flow = recurse_flow(pg_id)
  File "/home/sam/Documents/freenet/nifi-automation/venv/lib/python3.6/site-packages/nipyapi/canvas.py", line 64, in recurse_flow
    return _walk_flow(get_flow(pg_id))
  File "/home/sam/Documents/freenet/nifi-automation/venv/lib/python3.6/site-packages/nipyapi/canvas.py", line 85, in get_flow
    raise ValueError(err.body)
ValueError: No applicable policies could be found. Contact the system administrator.

Process finished with exit code 1

I have a few NIFI automation scripts which work perfectly fine when I run on a unsecure cluster (localhost/or somehwere) but I get this error when I run against a URL which is behind KNOX gateway. I can see a few functions in nipyapi.access_api class

  • def knox_callback(**kwargs)
  • def knox_callback_with_http_info(self, **kwargs):
  • def knox_request(self, **kwargs):
  • def knox_request_with_http_info(self, **kwargs):

I cant understand how any of these or in combination with anyother function from the class is/are to be used to overcome this? any idea?

EDIT1: IM using the security.py functions first is secure_login. In the docs its written Login requires a secure connection over https. Prior to calling this method, the host must be specified and the SSLContext should be configured (if necessary). set_service_ssl_context this function serves the purpose but I'm not sure if I need it or not as for one way TSL it isn't required.

But I have a confusion. I have two URLs one Knox URl with LDAP login other direct URI(although it also redirects to knox-Ldap sequence) When I give the Knox url I get a different error as compared to direct one.

from the direct URI i get 

File /nipyapi/security.py", line 130, in service_login
    username=username, password=password)
nipyapi.nifi.rest.ApiException: (409)
Reason: Conflict
HTTP response body: Username/Password login not supported by this NiFi.

And in case of Knox uri it throws the same connection error exception on the same lines but 

nipyapi.nifi.rest.ApiException: (404)
Reason: Not Found

So im assuming I have to use the direct url. secondly why it says user not for. I can login manually. from the LDAP sequence. My current request is going as an anonymous user so I'm going to use the Certs and try the set_service_ssl_context function with the PEM files.

Btw below are the two urls.

"nifi_host": "https://****.****.net:8443/nifi-api", DIRECT URL

"nifi_host": "https://****-****.****.net:8443/gateway/****-sso/nifi-api", knox url

EDIT 2: my request is being received as anonymous at server even with the following code. 

nipyapi.security.set_service_ssl_context(service='nifi', ca_file=None, client_cert_file="bi.keystore", client_key_file=None, client_key_password="infraop6043")
nipyapi.security.service_login(username='myuser', password='mypass')

it gives Connection Error here

nipyapi.nifi.AccessApi().create_access_token( username=username, password=password) And show this error HTTP response body: Username/Password login not supported by this NiFi

I'm not sure how to use the set_service_ssl_context properly. Maybe instead of bi.keystore I should try using directly the letsencrypt-root-ca: letsencrypt.org/certs/isrgrootx1.pem.txt or my local system ca-certs.

my properties file for toolkit-cli was

*baseUrl=https://svc-hadoop-utilities-pre-c3-02.jamba.net:18443

keystore=/home/jread/nifi-toolkit/bi.keystore

keystoreType=JKS

keystorePasswd=infraop6043

keyPasswd=

truststore=/usr/lib/java/jre/lib/security/cacerts

truststoreType=JKS

truststorePasswd=changeit

proxiedEntity=CN=bijobs.jamba.net*

Dennis Jaheruddin
  • 21,208
  • 8
  • 66
  • 122
Sam
  • 497
  • 1
  • 10
  • 34

1 Answers1

0

The error message you have 'No applicable policies could be found. Contact the system administrator.' is typically produced by the Ranger plugin when security is enabled and the user you are connecting with is not permitted, I have not seen Knox produce it.

Can you confirm that you have not enabled Ranger without policies when you enable Knox for NiFi, and that is the error message you are getting? If you look in the nifi-app.log I suspect you'll see the same error being produced, which would suggest to me that NiPyAPi is just transporting the error that NiFi is generating due to an incomplete security setup.

Chaffelson
  • 1,249
  • 9
  • 20
  • ranger is setup. Previously, as mentioned I was using unsecured cluster to deploy things. Now I want to use this one which has Knox and ranger. But I can't figure out the API to use or the sequence. – Sam Dec 11 '18 at 18:36
  • Yes, the previously deployed flows were by anonymous users on local. whichi obviously wouldn't be authorized here. I know if one wants to do some authentication inside NIFI have to do either LDAP/Kerberos/OpenID etc and then get an access token and pass onto the API. But I dont know how to do it with Ranger+Knox. any help would be appreciated – Sam Dec 12 '18 at 08:18
  • So what you will need to do is follow the Ranger documentation to integrate your particular identity provider (say, LDAP) with Ranger, then create a NiFi policy in Ranger to allow a particular user to do stuff. Then in Nipy you will be able to use the security.py methods to login as that user, and it should handle the token for you – Chaffelson Dec 12 '18 at 22:06