Spinnaker:AKS account not showing on UI

Question

I've configured spinnaker cloud provider as kubernetes with below commands

hal config provider kubernetes enable
kubectl config current-context
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add my-k8s-v2-account     --provider-version v2     --context $CONTEXT
hal config features edit --artifacts true

but this account is not visible on spinnaker UI

and in logs its shows error as below

Nov 29 12:07:43 47184UW2DDevLVM2 gate[34594]: 2019-11-29 12:07:43.860 ERROR 34594 --- [TaskScheduler-5] c.n.s.g.s.DefaultProviderLookupService   : Unable to refresh account details cache, reason: timeout

please advise.. thanks..

here's my hal deploy diff command output

+ Get current deployment
  Success
+ Determine config diff
  Success
~ EDITED
default.persistentStorage.redis
- port 6379 -> null
- host localhost -> null
~ EDITED
telemetry

I've provisioned new VM and did all installation process from scratch but still same issue :(

here is ~/.kube/config file

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxx
    server: https://xxx:443
  name:xxx
contexts:
- context:
    cluster: xxx
    user: xxx
  name: xxx
current-context: xxx
kind: Config
preferences: {}
users:
- name: xxx
  user:
    client-certificate-data: xxx
    client-key-data: xxx
    token: xxx

and here is ~/.hal/config file

currentDeployment: default
deploymentConfigurations:
- name: default
  version: 1.17.2
  providers:
    appengine:
      enabled: false
      accounts: []
    aws:
      enabled: false
      accounts: []
      bakeryDefaults:
        baseImages: []
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: xxx
      defaults:
        iamRole: BaseIAMRole
    ecs:
      enabled: false
      accounts: []
    azure:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: azure-linux.json
        baseImages: []
    dcos:
      enabled: false
      accounts: []
      clusters: []
    dockerRegistry:
      enabled: false
      accounts: []
    google:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: gce.json
        baseImages: []
        zone: us-central1-f
        network: default
        useInternalIp: false
    kubernetes:
      enabled: true
      accounts:
      - name: xxx
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        context: xxx
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: /home/xxx/.kube/config
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: xxx
    oracle:
      enabled: false
      accounts: []
      bakeryDefaults:
        templateFile: oci.json
        baseImages: []
    cloudfoundry:
      enabled: false
      accounts: []
  deploymentEnvironment:
    size: SMALL
    type: LocalDebian
    imageVariant: SLIM
    updateVersions: true
    consul:
      enabled: false
    vault:
      enabled: false
    customSizing: {}
    sidecars: {}
    initContainers: {}
    hostAliases: {}
    affinity: {}
    tolerations: {}
    nodeSelectors: {}
    gitConfig:
      upstreamUser: spinnaker
    livenessProbeConfig:
      enabled: false
    haServices:
      clouddriver:
        enabled: false
        disableClouddriverRoDeck: false
      echo:
        enabled: false
  persistentStorage:
    persistentStoreType: azs
    azs:
      storageAccountName: xxx
      storageAccountKey: xxx
      storageContainerName: xxx
    gcs:
      rootFolder: front50
    redis: {}
    s3:
      rootFolder: front50
    oracle: {}
  features:
    auth: false
    fiat: false
    chaos: false
    entityTags: false
    artifacts: true
  metricStores:
    datadog:
      enabled: false
      tags: []
    prometheus:
      enabled: false
      add_source_metalabels: true
    stackdriver:
      enabled: false
    newrelic:
      enabled: false
      tags: []
    period: 30
    enabled: false
  notifications:
    slack:
      enabled: false
    twilio:
      enabled: false
      baseUrl: https://api.twilio.com/
    github-status:
      enabled: false
  timezone: America/Los_Angeles
  ci:
    jenkins:
      enabled: false
      masters: []
    travis:
      enabled: false
      masters: []
    wercker:
      enabled: false
      masters: []
    concourse:
      enabled: false
      masters: []
    gcb:
      enabled: false
      accounts: []
  repository:
    artifactory:
      enabled: false
      searches: []
  security:
    apiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:8084/
    uiSecurity:
      ssl:
        enabled: false
      overrideBaseUrl: http://xxx:9000/
    authn:
      oauth2:
        enabled: false
        client: {}
        resource: {}
        userInfoMapping: {}
      saml:
        enabled: false
        userAttributeMapping: {}
      ldap:
        enabled: false
      x509:
        enabled: false
      iap:
        enabled: false
      enabled: false
    authz:
      groupMembership:
        service: EXTERNAL
        google:
          roleProviderType: GOOGLE
        github:
          roleProviderType: GITHUB
        file:
          roleProviderType: FILE
        ldap:
          roleProviderType: LDAP
      enabled: false
  artifacts:
    bitbucket:
      enabled: false
      accounts: []
    gcs:
      enabled: false
      accounts: []
    oracle:
      enabled: false
      accounts: []
    github:
      enabled: false
      accounts: []
    gitlab:
      enabled: false
      accounts: []
    gitrepo:
      enabled: false
      accounts: []
    http:
      enabled: false
      accounts: []
    helm:
      enabled: false
      accounts: []
    s3:
      enabled: false
      accounts: []
    maven:
      enabled: false
      accounts: []
    templates: []
  pubsub:
    enabled: false
    google:
      enabled: false
      pubsubType: GOOGLE
      subscriptions: []
      publishers: []
  canary:
    enabled: false
    serviceIntegrations:
    - name: google
      enabled: false
      accounts: []
      gcsEnabled: false
      stackdriverEnabled: false
    - name: prometheus
      enabled: false
      accounts: []
    - name: datadog
      enabled: false
      accounts: []
    - name: signalfx
      enabled: false
      accounts: []
    - name: aws
      enabled: false
      accounts: []
      s3Enabled: false
    - name: newrelic
      enabled: false
      accounts: []
    reduxLoggerEnabled: true
    defaultJudge: NetflixACAJudge-v1.0
    stagesEnabled: true
    templatesEnabled: true
    showAllConfigsEnabled: true
  plugins:
    plugins: []
    enabled: false
    downloadingEnabled: false
    pluginConfigurations:
      plugins: {}
  webhook:
    trust:
      enabled: false
  telemetry:
    enabled: false
    endpoint: https://stats.spinnaker.io
    instanceId: xxx
    connectionTimeoutMillis: 3000
    readTimeoutMillis: 5000

Here are the commands used to install spinnaker

az login
az aks get-credentials --resource-group xxx --name xxx
curl -O https://raw.githubusercontent.com/spinnaker/halyard/master/install/debian/InstallHalyard.sh
sudo bash InstallHalyard.sh --user xxx
hal config provider kubernetes enable
CONTEXT=$(kubectl config current-context)
hal config provider kubernetes account add xxx \
    --provider-version v2 \
    --context $CONTEXT
hal config features edit --artifacts true  
hal config deploy edit --type localdebian
hal config storage azs edit --storage-account-name xxx --storage-account-key xxx
hal config storage edit --type azs
hal version list
hal config version edit --version 1.17.2
sudo hal deploy apply
echo "host: 0.0.0.0" | tee \
    ~/.hal/default/service-settings/gate.yml \
    ~/.hal/default/service-settings/deck.yml
hal config security ui edit \
    --override-base-url http://xxx:9000/
hal config security api edit \
    --override-base-url http://xxx:8084/
sudo hal deploy apply

Found below exceptions logs

Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: 2019-12-02 11:12:07.424 ERROR 23908 --- [1-7002-exec-105] c.n.s.k.w.e.GenericExceptionHandlers     : Internal Server Error
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: java.lang.NullPointerException: null
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at com.netflix.spinnaker.clouddriver.kubernetes.health.KubernetesHealthIndicator.health(KubernetesHealthIndicator.java:48) ~[clouddriver-kubernetes-6.4.1-20191111102213.jar:6.4.1-20191111102213]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:95) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpoint.health(HealthEndpoint.java:50) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]
Dec  2 11:12:07 47184UW2DDevLVM2 clouddriver[23908]: #011at org.springframework.boot.actuate.health.HealthEndpointWebExtension.health(HealthEndpointWebExtension.java:53) ~[spring-boot-actuator-2.1.7.RELEASE.jar:2.1.7.RELEASE]

plus localhost 7002 is not responding

hexunix@47184UW2DDevLVM2:~$ curl -v http://localhost:7002/credentials
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 7002 (#0)
> GET /credentials HTTP/1.1
> Host: localhost:7002
> User-Agent: curl/7.58.0
> Accept: */*
>

P Ekambaram · Answer 1 · 2019-12-02T08:08:55.217

1

This is how i have done in my environment

kubeconfig_path="/home/root/.hal/kube-config"
kubernetes_account="my-account"
docker_registry="docker.io"
hal config provider kubernetes account add $kubernetes_account --provider-version v2 \
  --kubeconfig-file "$kubeconfig_path" \
  --context $(kubectl config current-context --kubeconfig "$kubeconfig_path") \
  --omit-namespaces=kube-system,kube-public \
  --docker-registries "$docker_registry"

make necessary updates and apply the changes. It should work.

from hal config it is clear that kubernetes account is added.

    kubernetes:
      enabled: true
      accounts:
      - name: xxx
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        context: xxx
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: /home/xxx/.kube/config
        oAuthScopes: []
        onlySpinnakerManaged: false
      primaryAccount: xxx

edited Dec 02 '19 at 08:08

answered Nov 29 '19 at 12:43

P Ekambaram

15,499
7
34
59

@P Ekambaram - isn't it the kubeconfig_path should be ~/.kube/config instead of ~/.hal/config ? because ~/.hal/config is not responding k8s custername on this command - kubectl config current-context --kubeconfig "$kubeconfig_path"... and however I tried this command again with ~/.kube/config path but its still not working.. I executed "sudo hal deploy apply" as well... – Jaydeep Soni Nov 29 '19 at 13:48
@p Ekambaram - I've tried this with updated path, but it's still not working.. I've done atleast 4-5 times back & forth configuration in the VM with other cloud provider enable and disable, so may be next week I'll create new vm with all configuration scratch and will see then.. thanks for your reply.. – Jaydeep Soni Nov 29 '19 at 23:26
Share /home/spinnaker/.hal/config – P Ekambaram Nov 30 '19 at 04:25
@P Ekambaram - I've provisioned new VM and did all installation process from scratch but still same issue :( I've edited question with ~/.kube/config and ~/.hal/config file.. – Jaydeep Soni Nov 30 '19 at 22:47
Share /home/spinnaker/.hal/config – P Ekambaram Dec 01 '19 at 04:47
@P Ekambaram - there is no .hal inside /home/spinnaker... when I ran installation command was like "sudo bash InstallHalyard.sh --user " – Jaydeep Soni Dec 02 '19 at 07:55
@P Ekambaram - sorry for chasing.. anything I can do more troubleshooting or share more info ? its first step only of installation and starting spinnaker where its failing ? may be is it possible that AKS is not supported yet directly as per https://azure.microsoft.com/en-in/blog/spinnaker-continuous-delivery-platform-now-with-support-for-azure/ ? – Jaydeep Soni Dec 02 '19 at 10:05

Spinnaker:AKS account not showing on UI

1 Answers1