RequestsDependencyWarning urllib3 (1.25.6) or chardet (3.0.4) doesn't match a supported version

edit  /usr/lib/python3/dist-packages/requests/__init__.py


 # Check urllib3 for compatibility.
    major, minor, patch = urllib3_version  # noqa: F811
    major, minor, patch = int(major), int(minor), int(patch)
    # urllib3 >= 1.21.1, <= 1.22
    assert major == 1
    assert minor >= 21
###    assert minor <= 22
    assert minor <= 25                        <<<<<<<<<<===================== new line



    # Check chardet for compatibility.
    major, minor, patch = chardet_version.split('.')[:3]
    major, minor, patch = int(major), int(minor), int(patch)
    # chardet >= 3.0.2, < 3.1.0
    assert major == 3
    assert minor < 1
    assert patch >= 2

CDN requirements

Few CDN requirements that we would like to see and test right away.

#1. cross domain functionality
#2. auth token for all assets
#3. exclusion rules for file extension and urls
#4. compression on large content
#5. compatibility with jwplayer to invoke media content
#6. able to transcode media streaming into m3u8 format
#7. various traffic reporting on each cnames
#8. caching rules for nodes and expiration policies
#9. APIs for accessing origin server
#10. Able to map cnames to a specific path

AWS EKS ~/.kube/config (heptio vs awscli)

You can config your kubectl to talk to EKS both ways

AWSCli Token
pankaj@PGautam-X280:~/terraform/eks3-thar$ aws eks get-token --cluster-name thar-eks-dev-cluster-2
{"kind": "ExecCredential", "apiVersion": "client.authentication.k8s.io/v1alpha1", "spec": {}, "status": {"expirationTimestamp": "2019-11-01T21:59:14Z", "token": "k8s-aws-v1.aHR0cHM6Ly9zdHMuYW1hem9uYXdzLmNvbS8_QWN0aW9uPUdldENhbGxlcklkZW50aXR
5JlZlcnNpb249MjAxMS0wNi0xNSZYLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZY
Q1MTRaJlgtQW16LUV4cGlyZXM9NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JTNCeC1rO
HMtYXdzLWlkJlgtQW16LVNpZ25hdHVyZT0yNTdmNTE4YjBjOWNjZGE4MmNmOTk1YWI2ND
FmYzZjZTM2YTdhYTZiM2UwNjBmZGUwZDg3ZTMzMWQ1NGYyODQ5"}}


Heptio Token
pankaj@PGautam-X280:~/terraform/eks3-thar$ aws-iam-authenticator token -i eks-thar-techops-dev-cluster-1
{"kind":"ExecCredential","apiVersion":"client.authentication.k8s.io/v1alpha1","spec":{},"status":{"token":"k8s-aws-v1.aHR0cHM6Ly9zdHMuYW1hem9uYXdzLmNvbS8_QWN0aW9uPUdldENhbGxlcklkZW50aX
R5JlZlcnNpb249MjAxMS0wNi0xNSZYLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYL
JlgtQW16LUV4cGlyZXM9NjAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JTNCeC1rOHMtYXdzL
WlkJlgtQW16LVNpZ25hdHVyZT00M2Q2NmUzNWI4ZWNlNDExNzEwYmMxOTNjNzI4YmEzMGM
wN2E2ZjZjNjRmZDIxN2Y1ZjYwODk0NGNmZmMwM2Zk"}}


AWSCli configuration file:

apiVersion: v1
clusters:
- cluster:
    server: https://093E146E63E428CC642F01362F6BE661.sk1.us-west-2.eks.amazonaws.com
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURB
ZtQm1WQ1U4OXZDamZpOXBNaQo2Nmp3NkxTWSttbnFQYzhVcUdVbks5WURhZU01b0ZvULQo=
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: aws
  name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws
      env:
      - name: AWS_PROFILE
        value: opssbx
      args:
      - "eks"
      - "get-token"
      - "--cluster-name"
      - "thar-eks-dev-cluster-2"
      # - "--role"
      # - "arn:aws:iam::191829356374:role/ops"

 
Heptio configuration file:
apiVersion: v1
clusters:
- cluster:
    server: https://093E146E63E428CC642F01362F6BE661.sk1.us-west-2.eks.amazonaws.com
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURB
TkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHV
npNQjRYRFRFNU1URXdNVEU1TURJMU1Wb1hEVEk1TVRBeU9URTVNREkxTVZvd0ZURVR
NQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQl
GSUNBVEUtLS0tLQo=
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: aws
  name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      env:
      - name: AWS_PROFILE
        value: opssbx
      args:
        - "token"
        - "-i"
        - "thar-eks-dev-cluster-2"

AWS EKS cluster authentication Unauthorized

Scenario: After you created a EKS cluster with worker nodes using Terraform. You are not able to connect to the cluster endpoint. The cluster was created from an account which has admin rights to assume roles in other accounts. We confirmed the authorized roles and policies are correct. This terraform was used multiple times to create a dev cluster, however this time we are creating the cluster in a different account.
It seems like the EKS is not able to validate the user, or we are not able to send the right user/credential to the cluster. So its not an authorization issue but an authentication problem.
So, we have explore more into authentication modules rather then EKS role based access control (RBAC)

One of the first thing you can do is enable cluster auth logs and see which user cloudwatch is capturing as access denied.

Errors:

pankaj@PGautam-X280:~/terraform/eks3-thar$ kubectl versionClient Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
error: You must be logged in to the server (the server has asked for the client to provide credentials)


pankaj@PGautam-X280:~/terraform/eks3-thar$ kubectl get nodes
error: the server doesn't have a resource type "nodes"

pankaj@PGautam-X280:~/terraform/eks3-thar$ kubectl version --v=7
I1030 15:55:05.651723   23213 loader.go:359] Config loaded from file /home/pankaj/.kube/config
I1030 15:55:05.653152   23213 round_trippers.go:383] GET https://A4A4CXXX0XXXXXXXE.gr7.us-west-2.eks.amazonaws.com/version?timeout=32s
I1030 15:55:05.653363   23213 round_trippers.go:390] Request Headers:
I1030 15:55:05.653585   23213 round_trippers.go:393]     Accept: application/json, */*
I1030 15:55:05.653855   23213 round_trippers.go:393]     User-Agent: kubectl/v1.11.2 (linux/amd64) kubernetes/bb9ffb1
I1030 15:55:06.023642   23213 round_trippers.go:408] Response Status: 401 Unauthorized in 369 milliseconds
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
I1030 15:55:06.035173   23213 helpers.go:201] server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "the server has asked for the client to provide credentials",
  "reason": "Unauthorized",
  "details": {
    "causes": [
      {
        "reason": "UnexpectedServerResponse",
        "message": "unknown"
      }
    ]
  },
  "code": 401
}]
F1030 15:55:06.036149   23213 helpers.go:119]
error: You must be logged in to the server (the server has asked for the client to provide credentials)

=========================================
Solution:

Cluster is active
pankaj@PGautam-X280:~/terraform/eks3-thar$ aws eks --profile ops --region us-west-2 describe-cluster --name eks-dev-cluster-1 --query cluster.status
"ACTIVE"

Heptio token works:
pankaj@PGautam-X280:~/terraform/eks3-thar$ aws-iam-authenticator token -i eks-dev-cluster-1
{"kind":"ExecCredential","apiVersion":"client.authentication.k8s.io/v1alpha1","spec":{},"status":{"token":"k8s-aws-v1.aHR0cHM6Ly9zdHMuYW1hem9uYXdzLmNvbS8_QWN0aW9uPUdldENhbGxlcklkZ
W50aXR5JlZlcnNpb249MjAxMS0wNi0xNSZYLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNI
QTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFaUkhORE9GVzJPM0ZXUjdIJTJGMjAxOTEw
N0JTNCeC1rOHMtYXdzLWlkJlgtQW16LVNpZ25hdHVyZT1jZjk4NTg5MDI2ZjA2MjMyM2Y5Njh
iNDljMzI2YWZmYmI5MDcxNzBlMTUxZDNlN2QxYzA0NGFiOWE0MDcxNTc3"}}

Update .kube config with env:
pankaj@PGautam-X280:~/terraform/eks3-thar$ echo $KUBECONFIG
/home/pankaj/terraform/eks3-thar/config

- name: aws
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: aws-iam-authenticator
      env:
      - name: AWS_PROFILE
        value: opssbx
      args:
        - "token"
        - "-i"
        - "eks-dev-cluster-1"


pankaj@PGautam-X280:~/terraform/eks3-thar$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6-eks-5047ed", GitCommit:"5047edce664593832e9b889e447ac75ab104f527", GitTreeState:"clean", BuildDate:"2019-08-21T22:32:40Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

AWS EKS - elb in pending state

## ELB Error ( in pending state)
Error creating load balancer (will retry): failed to ensure load balancer for service default/hellopankajgautam:

Solution: Tagging required for VPC/Subnets and private
https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html

"kubernetes.io/cluster/${var.product}-${var.env}-${var.cluster_name}" = "shared"
"kubernetes.io/role/internal-elb" = "1"

in my experience so far - the only time elb is in pending state -
    if there not tagging available as mentioned above
    or there is no ips left
    or if there is any soft limit errors