Python Requests Default CA Certs

Back in 2015 I switched jobs and joined a company that had some services written in python. A few months after I joined the inevitable happened and we lost a senior engineer who had been around for the design and build of those python services. If you’ve been an engineer for any amount of time you know what happened next: Murphy’s law kicked.

The service that guy had helped build started failing with nearly nothing in the logs to tell us what was happening.

After plenty of digging (and more time taking packet captures than I should admit to) we found that the version of requests we were using had an incredibly old cacerts file bundled with it, and one of our dependencies had updated.

Which lead me to create this life principle

If one makes an HTTPS request, one must know how to print the path to ones certificate trust store

This quote from the advanced requests docs sheds some light on why

Before version 2.16, Requests bundled a set of root CAs that it trusted, sourced from the Mozilla trust store.

If you were a well meaning developer you might assume that requests would just whatever ships with the system, but that’s where certs get complicated: different systems put them different places.

For example in Redhat apparently they can go into /etc/pki/ca-trust . On Ubuntu they go into /etc/ssl/certs/ , and on Windows they go into the windows trust store (I’ll let you read up on that yourself). So rather than deal with all of that variation the requests maintainers just bundled their own certs.

Which is why it’s important to know how to get your HTTPS client to tell you what certs it’s connecting to. Rather than sort through documentation, reading source code, and guessing at where you should copy a certificate file during an outage, you should know how to find the root CA path before you have a problem

Here’s an example of getting to the same info in python requests

from requests.utils import DEFAULT_CA_BUNDLE_PATH
print(DEFAULT_CA_BUNDLE_PATH)
C:\Users\bo671\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\certifi\cacert.pem
import certifi
certifi.where()
'C:\\Users\\bo671\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\certifi\\cacert.pem'

And the same thing from the latest python docker image

>>> from requests.utils import DEFAULT_CA_BUNDLE_PATH
>>> print(DEFAULT_CA_BUNDLE_PATH)
/usr/local/lib/python3.9/site-packages/certifi/cacert.pem
>>> import certifi
>>> certifi.where()
'/usr/local/lib/python3.9/site-packages/certifi/cacert.pem'

Do yourself a favor and dig through your HTTPS client’s documentation until you can find how to print out where you cert file path is. You’ll never regret having that on hand during a problem!

If you like my writing consider supporting me by joining medium.