The Issue

Developer reported to me that there is handshake problem with an internal API gateway.

 javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Troubleshooting

I found curl and wget works well. So I decided to do a packet capture with tcpdump:

sudo /usr/sbin/tcpdump -Z appadm -w api.pcap host 10.xxx.xxx.xxx

Here are facts:

1. curl can get Server Hello and go through the complete SSL handshake
2. Java cannot get Server Hello
3. Both are using TLS 1.2

So I doubt it is about the Cipher Suites. The Java supported Cipher Suites are:

Cipher Suites (29 suites)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
    Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
    Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 (0xc026)
    Cipher Suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 (0xc02a)
    Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x006b)
    Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
    Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
    Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA (0xc005)
    Cipher Suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA (0xc00f)
    Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
    Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
    Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
    Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 (0xc025)
    Cipher Suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 (0xc029)
    Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (0x0067)
    Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 (0x0040)
    Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
    Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
    Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
    Cipher Suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA (0xc004)
    Cipher Suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA (0xc00e)
    Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
    Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
    Cipher Suite: TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0x00ff)

The curl pcap uses TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 which is not in the list. So I used following command to scan the supported cipher suites:

[appadm local]$ nmap --script ssl-enum-ciphers -p 443 lab-api-int.xxx.com

Starting Nmap 5.51 ( http://nmap.org ) at 2020-06-29 22:58 UTC
Nmap scan report for lab-api-int.xxx.com (10.xxx.xxx.xxx)
Host is up (0.00038s latency).
rDNS record for 10.yyy.yyy.yyy: lab2-api-int.yyy.yyy.com
PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers: 
|   TLSv1.2
|     Ciphers (2)
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|     Compressors (1)
|_      uncompressed

Nmap done: 1 IP address (1 host up) scanned in 0.77 seconds

So it is clear the handshake failure is about the Cipher Suites. Per stackover flow (https://stackoverflow.com/questions/21289293/java-7-support-of-aes-gcm-in-ssl-tls), JDK 1.7, which is in use, doesn’t support AES-GCM cipher suites.

In this case, I had to ask developers to turn to JDK 1.8 and it works well!

My Requirement

I was trying to add some information regarding my code into my Flask app. Like version and build time. This is needed for all templates as a footer it should be processed once.

My version information is stored in __version__.py in this format:

__title__ = 'fcapp'
__description__ = 'My app'
__url__ = 'https://felixchr.github.com/'
__version__='1.0.4'
__build__='0x5ee41d6d'
__author__ = 'Felix Cao'
__author_email__ = 'felix.cao@XXX.com'
__license__ = 'MIT'

The Solution

I studied online to find out the resolution. And eventually I found it on stackoverflow and here is my code here:

__init__.py of my Flask app

I added code to read and load the variables immediately after app is initialized:

app = Flask(__name__)
app.config.from_object('config')

@app.context_processor
def inject_stage_and_region():
    here = os.path.abspath(os.path.dirname(__file__))
    about = dict()
    with open(os.path.join(here, '__version__.py'), 'r') as f:
        exec(f.read(), about)
return {'about': about}

The return value is a dict. Part of this code is borrowed from Kenneth Reitz’’s requests.

The base.html of my main template

In my base.html now I can add those to this code:

  <div>
      <hr/>
      Version: {{about.__version__}}<br/>
      Build Time: {{about.__build__}}
  </div>

It works now!

The Background

I need to develop Python and Bash scripts for our applications which running on Linux but my workstation is a Windows 10 laptop. I would like to test immediately on my local machine instead of test on remote servers, in spite of that I have Jenkins set up for testing. My company’s VPN has very tough network restrictions. It force all outbound traffic goes to VPN gateway. That is to say, I cannot reach to any other IP addresses other than 127.0.0.1 locally.

Install and Configure VirtualBox

I would not elaborate how to install VirtualBox on Windows 10. It is straightforward. Just download and install.

Guest OS installation is not my topic as well.

Network Configuration

Network Types

Bridge: act like another computer in the same network. It can get same IP addresses with your computer.

NAT: Use your computer as gateway. Both your computer and VM will have IP in same subnet

Host-only: accessible only from Host (I’m not if this is correct. Maybe updated later)

Use NAT Network

Due to the restriction of my company’s VPN, I had to use NAT Network for my guest OS.

In Network tab of Settings of VM, set the adapter to attach to NAT Network.

The default network is 10.0.2.0/24. 10.0.2.1 and 10.0.2.2 are occupied. You’d better to use static instead of DHCP because we’ll need to set Port Forwarding. You won’t want to change the Port Forwarding frequently if you have multiple VMs.

Port Forwarding

Port Forwarding is globally. It’s in File -> Preference -> Network -> NatNetwork. Click Edit NatNetwork and Port Forwarding.

Share Folder Between Host and Guest

Use Samba

This was tested with VMWare Player and CentOS 6. After I switched to VirtualBox, I didn’t try it as my VPN restricts me to access any IP other than 127.0.0.1. I put it here as this may help your situation.

1. Install samba

$ sudo yum -y install samba

1. Configure Samba server

   The configuration file is /etc/samba/smb.conf

[global]
    workgroup = MYGROUP
    server string = Samba Server Version %v
    log file = /var/log/samba/log.%m
    max log size = 50
    security = user
    passdb backend = tdbsam
    load printers = no
    cups options = raw
[homes]
    comment = Home Directories
    browseable = no
    writable = yes

1. configure password

[root@diamond samba]# smbpasswd -a felixc
New SMB password:
Retype new SMB password:
Added user felixc.

1. Mount on Windows

   On windows, run: \<IP-ADDR>Use “local\usernmae” and password to login.

Use VirtualBox shared folder

This is tested with VirtualBox 6.1 and CentOS 7. This is my current test environment.

1. install guest addons 1.1 Devices -> Insert Guest Additions CD images

   1.2 update kernel on Guest OS

$ sudo yum update kernel -y
$ sudo yum install kernel-headers kernel-devel gcc make -y
$ sudo init 6

1.3. mount cdrom and install the additions to OS.

# check the cdrom device
$ blkid
$ sudo mount /dev/sr0 /a
$ cd /a
$ sudo ./VBoxLinuxAdditions.run

1. set up share folder in VirtualBox on host

   Use menu VM -> Settings -> Shared Folders,

2. Add user to vboxfs group in order to gain permission to the shared folder

$ sudo usermod -G vboxfs felixc

1. mount on guest OS

$ VBoxControl sharedfolder list
$ mount.vboxsf share /app

Development Environment

Use VS Code Remote Development Plugin

VS Code is awesome! The Remote Development plugin works good with Linux environment.

I installed following plugins on my host for remote development:

1. Remote Development
2. Remote - SSH
3. Remote - SSH: Editing Configuration Files
4. Remote - WSL

Following plugins are installed on develop server:

1. Python
2. GitLens
3. Prettier

VS Code supports pipenv very well. It loads python virtualenv automatically.

Use Git Bash for Windows

Git bash is a very good way to provide Bash environment and GIT. It would be best alternative for Windows Command Line.

Issues and Fixes

1. rc=-5640

This is due to some antivirus software. So far I haven’t found any fix for it. But you can try to few times to start your VM. Sometimes I tried more than ten times in order to start my VM.

Someone says increasing the memory size would increase the starting success rate. I set my VM to 2GB RAM and it seems better now.

1. Cannot PING gateway under NAT Network

This happens very often. To fix it, use VBoxManage to restart the network. I wrote a bash script and put it into PATH of my Git Bash. Each time when I have this issue I run it.

/c/fc/local/bin/vboxnat-restart.sh

#!/bin/bash
echo "Stopping..."
/c/Program\ Files/Oracle/VirtualBox/VBoxManage.exe natnetwork stop --netname NatNetwork
sleep 2
echo "Starting..."
/c/Program\ Files/Oracle/VirtualBox/VBoxManage.exe natnetwork start --netname NatNetwork
echo "Done"

What is my requirement?

Due to the COVID19 pandemic my son has to stay at home. So I give him some math quiz compiled by myself for practice. Like addition, substraction, etc. I used MS Excel for the layout and print and I used Python to generate some quiz as well. Each time I generate the quiz I had to manually import in Excel to keep the format and layout in order to fill in one print page. But I was tired with that so I searched online to find the new way.

Unfortunately there is no one module can manipulate the xlsx directly. I got some ideas and decided to copy from template and update the value in new file and try to print it automatically.

Copy format from template and generate the new file

import openpyxl
from openpyxl import load_workbook, Workbook

infile = load_workbook('../tmp/tmpl.xlsx')
outfile = Workbook()
new_sheet = outfile.active
default_sheet = infile.active

from copy import copy

for row in default_sheet.rows:
    for cell in row:
        #new_cell = new_sheet.cell(row=cell.row, column=cell.col_idx,
        new_cell = new_sheet.cell(row=cell.row, column=cell.column,
                value= cell.value)
        if cell.has_style:
            new_cell.font = copy(cell.font)
            new_cell.border = copy(cell.border)
            new_cell.fill = copy(cell.fill)
            new_cell.number_format = copy(cell.number_format)
            new_cell.protection = copy(cell.protection)
            new_cell.alignment = copy(cell.alignment)
# copy the height and width
for rowno in range(1, 13):
    new_sheet.row_dimensions[rowno].height = default_sheet.row_dimensions[rowno].height
for col in 'ABCDE':
    new_sheet.column_dimensions[col].width = default_sheet.column_dimensions[col].width
# the merged file was not copied
new_sheet.merge_cells('A1:E1')
# save to new file
outfile.save('../tmp/out.xlsx')

The code is a simple update from Stackoverflow, based on Charlie’s answer

Print the file

I’m trying to print on Windows 10 so to code is:

import win32api
import win32print

win32api.ShellExecute (
  0,
  "print",
  r"C:\fc\local\tmp\out.xlsx",
  '/d:"%s"' % win32print.GetDefaultPrinter (),
  ".",
  0
)

This is from this link

For Linux platform you can find at this stackoverflow page, at the answer from Anuj

• sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1305 (42000): FUNCTION rcc_airflow.JSON_VALID does not exist

airflow needs mysql 5.7

• Exception: Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql

set explicit_defaults_for_timestamp to 1 in my.cnf

• cryptography not found - values will not be stored encrypted.

pip install cryptography

• empty cryptography key - values will not be stored encrypted.

https://airflow.readthedocs.io/en/stable/howto/secure-connections.html

generate a key to replace in airflow.cfg

• use mysql-connector-python instead of Mysql-python (this proved not working)

in airflow.cfg:

mysql+mysqlconnector://user:password@10.10.10.101:3333/airflow

• Unicodedecodeerror: ‘utf-8’ codec can’t decode byte 0x80 in position 0: invalid start byte

Please install ‘apache-airflow[mysql]’

mysql-connector-python is proved not working with Airflow

• BashOperator: TemplateNotFoundthe command needs to follow with a space:

bash_command='{}/backup.sh '.format(backup_dir),

5/7/2020 I’m starting a new blog site here. It will be my permanent blog site!