Tuesday, April 27, 2021

ansible error with python boto3

 Ansible Error:

TASK [provision-ai-vpc : Create VPC] ************************************* task path: /Users/pankajgautam/infra/ansible/roles/provision-ai-vpc/tasks/main.yml:14 redirecting (type: modules) ansible.builtin.ec2_vpc_net to amazon.aws.ec2_vpc_net fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (botocore or boto3) on Veevas-MacBook-Pro.local's Python /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"} PLAY RECAP ********************************************************************* localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0 


It seems like Mac comes with python2.7 and cannot be touched/removed

pankajgautam@Veevas-MBP ~ % /usr/bin/python --version

Python 2.7.16


If you install awscli it comes with its own python

pankajgautam@Veevas-MBP ~ % /usr/local/aws-cli/aws --version

aws-cli/2.1.37 Python/3.8.8 Darwin/20.4.0 exe/x86_64 prompt/off


You can install python with pip or pip3 that generally installs under /Library

pankajgautam@Veevas-MBP ~ % ls  /Library/Python/ 

2.7 3.8


Then you can also install python using brew that gets installed under /usr/local/Cellar

pankajgautam@Veevas-MBP ~ % ls -l /usr/local/Cellar/ | grep python

drwxr-xr-x  3 pankajgautam  admin  96 Apr 21 12:58 python@3.9




You can specify which python to be used with ansible using ansible_python_interpreter

and also make sure you have boto3 module imported in that python version

 

AWS_PROFILE=aiprod ansible-playbook -e "aws_region=us-east-1 release_type=gr ansible_python_interpreter=/usr/local/Cellar/python@3.9/3.9.4/bin/python3" -i ansible/inventory/prod ansible/provision-ai-vpc-pb.yml -vv 2>&1 | tee provision.log 


pankajgautam@Veevas-MBP % sudo pip3 install boto3 

WARNING: The directory '/Users/pankajgautam/Library/Caches/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Collecting boto3 Downloading boto3-1.17.54-py2.py3-none-any.whl (131 kB) |████████████████████████████████| 131 kB 640 kB/s 

Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.9/site-packages (from boto3) (0.10.0) 

Collecting botocore<1.21.0,>=1.20.54 

Downloading botocore-1.20.55-py2.py3-none-any.whl (7.4 MB) |████████████████████████████████| 7.4 MB 545 kB/s Collecting s3transfer<0.5.0,>=0.4.0 Downloading s3transfer-0.4.1-py2.py3-none-any.whl (79 kB) |████████████████████████████████| 79 kB 16.1 MB/s Collecting urllib3<1.27,>=1.25.4 Downloading urllib3-1.26.4-py2.py3-none-any.whl (153 kB) |████████████████████████████████| 153 kB 18.2 MB/s Collecting python-dateutil<3.0.0,>=2.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) |████████████████████████████████| 227 kB 16.1 MB/s Collecting six>=1.5 

Downloading six-1.15.0-py2.py3-none-any.whl (10 kB) Installing collected packages: six, urllib3, python-dateutil, botocore, s3transfer, boto3 Successfully installed boto3-1.17.54 botocore-1.20.55 python-dateutil-2.8.1 s3transfer-0.4.1 six-1.15.0 urllib3-1.26.4 


pankajgautam@Veevas-MBP infra % /usr/local/Cellar/python@3.9/3.9.4/bin/python3

Python 3.9.4 (default, Apr 5 2021, 01:49:30) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. 

>>> import boto3 >>> 


Thursday, February 6, 2020

migrating to (aws nitro based system) machine type m4 to m5


Tested the conversion with RHEL8 and AmzonLinx2
from m4=>m5 and r4=>r5


As of now following instances are based on the Nitro system:
A1, C5, C5d, C5n, G4, I3en, Inf1, M5, M5a, M5ad, M5d, M5dn, M5n, p3dn.24xlarge, R5, R5a, R5ad, R5d, R5dn, R5n, T3, T3a, and z1d

Before changing your instance to a nitro based system, make sure
1. The (ENA) elastic network adapter is installed and enabled for the instance.
2. The NVMe driver is installed on the instance and is loaded in the initramfs image of the instance.
3. Use "/etc/fstab" to mount the file systems using UUID/Label.

EBS volumes
[root@pod ~]# lsblk
NAME    MAJ:MIN  RM  SIZE RO TYPE MOUNTPOINT
xvda    202:0     0  200G  0 disk
├─xvda1 202:1     0    1M  0 part
└─xvda2 202:2     0  200G  0 part /
xvdd    202:48    0  500G  0 disk /data
xvde    202:80    0  500G  0 disk /data/logs
xvdf    202:160   0  500G  0 disk /data/active

NVMe EBS volume
[root@pod ~]# lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1     259:8    0  200G  0 disk
├─nvme0n1p1 259:9    0    1M  0 part
└─nvme0n1p2 259:10   0  200G  0 part /
nvme1n1     259:3    0  500G  0 /data
nvme2n1     259:4    0  500G  0 /data/logs
nvme3n1     259:2    0  500G  0 /data/active

NVMe map command:
# /sbin/ebsnvme-id /dev/nvme1n1
Volume ID: vol-0fb8db1fddd1f5834
xvde

Suggested /etc/fstab
UUID=35e495d3-a1df-40d9-bb6c-e37b81aec11f /data xfs noatime 0 0
UUID=075bd9b0-370e-48b1-9de2-4d061d16ca5b /data/logs xfs noatime 0 0
UUID=5e5fde64-1346-45e0-9eeb-6c68be8b81bd /data/active xfs noatime 0 0

Commands to check instance if its ready to changed to nitro system

lsinitrd /boot/initramfs-$(uname -r).img|grep nvme
modinfo nvme
modinfo ena
dracut -f -v



Lesson learned:

Currently AWS takes care of these volume device name change from 4 series instance to 5 series nitro based instances on the amazon linux 2 only. And the same steps should also work with Redhat hosts if we apply the udev rule and python script.

With amazon linux 2 the magic code that is creating the symlinks from /dev/xvdb, or /dev/xvdc devices to /dev/nvme1n1 or /dev/nvme2n1 is by the udev rules

In this file "/etc/udev/rules.d/70-ec2-nvme-devices.rules",

and more specifically this line KERNEL=="nvme[0-9]*n[0-9]*", ENV{DEVTYPE}=="disk", ATTRS{model}=="Amazon Elastic Block Store", PROGRAM="/sbin/ebsnvme-id -u /dev/%k", SYMLINK+="%c"
If we read that - this is saying, when a device is attached to the system, that has a kernel device name that matches the pattern given, run this program and create a symlink with the output of that program.

Is a python script (/sbin/ebsnvme-id) to get the underlying mount information then to create a sym link, 


One thing to keep in mind here is that this only works if the device presented to amazon linux 2 hosts are in /dev/xvdX format and NOT /dev/sdb or /dev/sdc which is an old format. In that case you may have tweak the rules to get the results you desire


Recommendation from  c5_m5_checks_script
OK     NVMe Module is installed and available on your instance
OK     ENA Module with version 2.0.2K is installed and available on your instance


Printing correct fstab file below:
# /etc/fstab
UUID=a727b695-0c21-404a-b42b-3075c8deb6ab /                       xfs     defaults        0 0
UUID=587cde86-c167-4e73-92cb-b67739d9991d /data/logs xfs  noatime 0 0
UUID=6410c3de-3714-4198-b31b-afeca374ef43 /data/active xfs  noatime 0 0

Tuesday, November 19, 2019

RequestsDependencyWarning urllib3 (1.25.6) or chardet (3.0.4) doesn't match a supported version

edit  /usr/lib/python3/dist-packages/requests/__init__.py


 # Check urllib3 for compatibility.
    major, minor, patch = urllib3_version  # noqa: F811
    major, minor, patch = int(major), int(minor), int(patch)
    # urllib3 >= 1.21.1, <= 1.22
    assert major == 1
    assert minor >= 21
###    assert minor <= 22
    assert minor <= 25                        <<<<<<<<<<===================== new line



    # Check chardet for compatibility.
    major, minor, patch = chardet_version.split('.')[:3]
    major, minor, patch = int(major), int(minor), int(patch)
    # chardet >= 3.0.2, < 3.1.0
    assert major == 3
    assert minor < 1
    assert patch >= 2

Friday, November 1, 2019

CDN requirements


Few CDN requirements that we would like to see and test right away.

#1. cross domain functionality
#2. auth token for all assets
#3. exclusion rules for file extension and urls
#4. compression on large content
#5. compatibility with jwplayer to invoke media content
#6. able to transcode media streaming into m3u8 format
#7. various traffic reporting on each cnames
#8. caching rules for nodes and expiration policies
#9. APIs for accessing origin server
#10. Able to map cnames to a specific path