Python Security

This page is an attempt to document security vulnerabilities in Python and the versions including the fix.

Pages

Packages and PyPI

Check for known vulnerabilities

Vulnerabilites in the Package Index

Index Vulnerability: Unchecked File Deletion

Improper checking of ACLs would have allowed any authenticated user to delete any release file hosted on the Package Index by supplying its md5 to the :files action in the pypi-legacy code base.

  • Disclosure date: 2017-10-12 (Reported via security policy on pypi.org)
  • Disclosed by: Max Justicz
Fixed In
  • PyPI “Legacy Codebase” (2017-10-12) fixed by commit 18200fa (2017-10-12)
Audit

After mitigating the attack vector and deploying it, the responding Package Index maintainer worked to verify that no release files had been improperly removed using this exploit.

The Package Index maintains an audit log in the form of a “Journal” for all actions initiated. It was determined that exploitation of this attack vector would still remove files via the existing interface an audit log would still be written.

Using this information, we were able to reconstruct the users with access to legitimately remove release files at point in time of each file removal using the audit log.

The output of this script were used to determine that no malicious actors exploited this vulnerability. All flagged journal entries were related to one of the following scenarios:

  • Username updates that were not properly updated in the Journal
  • Administrator intervention to remove packages
Timeline

Timeline using the disclosure date 2017-10-12 as reference:

  • 2017-10-12: Issue reported by Max Justicz following guidelines in security policy on pypi.org
  • 2017-10-12 (+0days): Report investigated by Ernest W. Durbin III and determined to be exploitable
  • 2017-10-12 (+0days): Fix implemented and deployed in commit 18200fa
  • 2017-10-12 (+0days): The audit journals maintained by PyPI were used to reconstruct the full history of file removals to determine that no malicious deletions were performed.
PyPI credential exposure on GitHub
Introduction

A common mistake made by users is committing and publishing “dotfiles” containing private material such as passwords, API keys, or cryptographic keys to public repositories on services such as GitHub.

Compounding this issue, the Python packaging ecosystem historically and currently encourages—albeit with some level of caution—the use of a .pypirc file for storage of passwords consumption by packaging tools. For a summary of the dangers of this methodology, see this article on securing PyPI credentials.

With ever strengthening search tools on GitHub attackers are able to formulate queries which quickly identify and obtain credentials from such hosting sites.

  • Disclosure date: 2017-11-05 (Reported via security policy on pypi.org)
  • Disclosed by: Joachim Jablon
Report

The PyPI security team was notified by Joachim Jablon that .pypirc files containing valid PyPI credentials were obtainable with a straightforward search and scrape of GitHub.

Using tools developed by the reporter the PyPI security team was able to identify 77 valid PyPI logins in 85 public files published to GitHub. These 77 logins had maintainer or administrator access to 146 unique projects on PyPI.

Audit

Action Taken by PyPI team

The PyPI security team followed up by auditing and extending the Proof of Concept tools supplied by the reporter to verify the report.

After running the tooling against the full result set of the GitHub code search the PyPI administrators unset the passphrases for all valid logins found and issued an administrative password reset for exposed users.

Additionally an audit of PyPI’s journals showed no signs of malicious access for the exposed accounts.

The email sent to affected users took the form

Recommendations

All users of PyPI should ensure that their PyPI login credentials are safe and have not been inadvertently exposed in a public repository of dotfiles, in the root of a project directory, or in some other public or shared medium.

The PyPI team does not have the resources to search or scrape all such services and may not have identified all forms of this exposure.

Additionally, reviewing the Audit Journal for your projects on pypi.python.org for suspicous activity is a good idea. If you identify any such activity, please report it per our published security policy.

Timeline

Timeline using the disclosure date 2017-11-05 as reference:

  • 2017-11-05 Issue reported by Joachim Jablon to a single member of the security team listed in our security policy on pypi.org
  • 2017-11-08 (+3days):Issue reported by Joachim Jablon to an additional member of the security team listed in our security policy on pypi.org
  • 2017-11-08 (+3days):Issue reported by Joachim Jablon to all members of the security team listed in our security policy on pypi.org
  • 2017-10-08 (+3days): Report investigated by Ernest W. Durbin III and determined to be valid.
  • 2017-10-09 (+4days): Administrative password resets issued.

PyPI typo squatting

fate0:

See also:

Example of typos:

  • urllib, urllib2: part of the standard library
  • urlib3 instead of urllib3

Python SSL and TLS security

Evolutions of the ssl module.

Cipher suite

Python 2.7 and 3.5-3.7:

_DEFAULT_CIPHERS = (
    'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:'
    'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:'
    '!aNULL:!eNULL:!MD5:!3DES'
    )

Pytohn 3.4:

_DEFAULT_CIPHERS = (
    'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:'
    'DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:'
    '!eNULL:!MD5'
)

Python 3.3:

_DEFAULT_CIPHERS = 'DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2'

Options

  • SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS: CBC IV attack countermeasure (CVE-2011-3389)
  • SSL_OP_NO_SSLv2: SSLv2 is unsafe
  • SSL_OP_NO_SSLv3: SSLv3 is unsafe
  • SSL_OP_NO_COMPRESSION: CRIME countermeasure
  • SSL_OP_CIPHER_SERVER_PREFERENCE
  • SSL_OP_SINGLE_DH_USE
  • SSL_OP_SINGLE_ECDH_USE

Python 3.7:

/* Defaults */
    options = SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS;
    if (proto_version != PY_SSL_VERSION_SSL2)
        options |= SSL_OP_NO_SSLv2;
    if (proto_version != PY_SSL_VERSION_SSL3)
        options |= SSL_OP_NO_SSLv3;
    /* Minimal security flags for server and client side context.
     * Client sockets ignore server-side parameters. */
#ifdef SSL_OP_NO_COMPRESSION
    options |= SSL_OP_NO_COMPRESSION;
#endif
#ifdef SSL_OP_CIPHER_SERVER_PREFERENCE
    options |= SSL_OP_CIPHER_SERVER_PREFERENCE;
#endif
#ifdef SSL_OP_SINGLE_DH_USE
    options |= SSL_OP_SINGLE_DH_USE;
#endif
#ifdef SSL_OP_SINGLE_ECDH_USE
    options |= SSL_OP_SINGLE_ECDH_USE;
#endif
    SSL_CTX_set_options(self->ctx, options);

CA store

SSLContext.load_default_certs() new in Python 3.4.

  • Windows: ssl.enum_certificates(store_name), new in Python 3.4. Use CertOpenStore() and CertEnumCertificatesInStore() functions.
  • Linux: xxx
  • macOS: xxx

See also

SSLContext

New in Python 3.2.

CRLs

  • SSLContext.verify_flags: New in Python 3.4
  • SSLContext.load_verify_locations(): This method can also load certification revocation lists (CRLs) in PEM or DER format. New in Python 3.5.
  • ssl.enum_crls(store_name): new in Python 3.4, specific to Windows

Validate TLS certificates

TLS versions

  • SSLv2 now black listed
  • SSLv3 now black listed

OpenSSL versions

Python bundled OpenSSL in Windows and macOS installers.

OpenSSL versions (read from the Windows installer):

  • Python 3.6.1: OpenSSL 1.0.2k
  • Python 2.7.13, 3.5.3 and 3.6.0: OpenSSL 1.0.2j
  • Python 2.7.12, 3.5.2: OpenSSL 1.0.2h
  • Python 2.7.11, 3.4.4, 3.5.0, 3.5.1: OpenSSL 1.0.2d
  • Python 2.7.10: OpenSSL 1.0.2a
  • Python 2.7.9: OpenSSL 1.0.1j
  • Python 3.3.5: OpenSSL 1.0.1e

Windows: see PCbuild/get_externals.bat (or PCbuild/readme.txt in older versions).

macOS: see Mac/BuildScript/build-installer.py.

macOS:

# Since Apple removed the header files for the deprecated system
# OpenSSL as of the Xcode 7 release (for OS X 10.10+), we do not
# have much choice but to build our own copy here, too.

Example of OpenSSL update: Upgrade installers to OpenSSL 1.0.2k (March 2017).

Python Security

Python branches

  • (Latest update: 2017-03-28) Python 2.6, 3.0, 3.1, 3.2 don’t get security fixes anymore and so should be considered as vulnerable
  • Branches getting security fixes: 2.7, 3.3, 3.4 and 3.5
  • See Status of Python branches

Dangerous functions and modules

  • Python 2 input()
  • Python 2 execfile()
  • eval()
  • subprocess.Popen(shell=True)
  • str.format(), Python 3 str.format_map, and Python 2 unicode.format() all allow arbitrary attribute access on formatted values, and hence access to Python’s introspection features: Be Careful with Python’s New-Style String Format (Armin Ronacher, December 2016)
  • The pickle module executes arbitrary Python code: never use it with untrusted data.
  • archives:
    • tarfile: Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of path, e.g. members that have absolute filenames starting with “/” or filenames with two dots ”..”.
    • zipfile: Never extract archives from untrusted sources without prior inspection. It is possible that files are created outside of path, e.g. members that have absolute filenames starting with “/” or filenames with two dots ”..”. zipfile attempts to prevent that.

Security model

Bytecode

CPython doesn’t verify that bytecode is safe. If an attacker is able to execute arbitrary bytecode, we consider that the security of the bytecode is the least important issue: using bytecode, sensitive code can be imported and executed.

For example, the marshal doesn’t validate inputs.

Sandbox

Don’t try to build a sandbox inside CPython. The attack surface is too large. Python has many introspection features, see for example the inspect module. Python also many convenient features which executes code on demand. Examples:

  • the literal string '\N{Snowman}' imports the unicodedata module
  • the code to log a warning might be abused to execute code

The good design is to put CPython into a sandbox, not the opposite.

Ok, understood, but I want a sandbox in Python. Well...

RNG

The random module must not be used in security sensitive code, except of the random.SystemRandom class.

CPython Security Experts

  • Alex Gaynor
  • Antoine Pitrou
  • Christian Heimes
  • Donald Stufft

Windows

ASLR and DEP

ASLR and DEP protections enabled since Python 3.4 (and Python 2.7.11 if built using PCbuild/ directory).

Unsafe Python 2.7 default installation directory

Python 2.7 installer uses C:Python27directory by default. The created directory has the “Modify” access rights given to the “Authenticated Users” group. An attacker can modify the standard library or even modify python.exe. Python 3 installer now installs Python in “C:Program Files” by default to fix this issue. Override the default installation directory, or fix the directory permissions.

DLL injection

On Windows 8.1 and older, the installer is vulnerable to DLL injection: evil DLL written in the same download directory that the downloaded Python installer. See DLL Hijacking Just Won’t Die.

DLL injection using PATH

Inject a malicious DLL in a writable directory included in PATH. The “pip” step of the Python installer will run this DLL.

We consider that it is not an issue of Python (Python installer) itself.

Once you have write access to a directory on the system PATH (not the current user PATH) and the ability to write binaries that are not validated by the operating system before loading, there are many more interesting things you can do rather than wait for the Python installer to be run.

Misc

Python Security Response Team (PSRT)

TODO list

TODO list for this python-security documentation.

  • Get Red Hat impact from a Red Hat URL?

cookielib

Add https://hackerone.com/reports/26647 vulnerability.

https://bugs.python.org/issue16611
#16611: BaseCookie now parses ‘secure’ and ‘httponly’ flags.
https://bugs.python.org/issue22796
Regression in Python 3.2 cookie parsing
https://bugs.python.org/issue25228
Support for httponly/secure cookies reintroduced lax parsing behavior
https://code.djangoproject.com/ticket/26158
cookie parsing fails with python 3.x if request contains unnamed cookie

YAML template:

- name: "Issue #22796"
  summary: >
    hardened HTTP cookie parsing
  links:
    - http://bugs.python.org/issue22796
  disclosure: "2014-11-04 (issue #22796 created)"
  fixed-in:
   - b1e36073cdde71468efa27e88016aa6dd46f3ec7 # 3.x
  description: >
    HTTP cookie parsing is now stricter, in order to protect against potential
    injection attacks.

    Reported by Tim Graham.