Fix locking

Remove race conditions that cause referenced bug.

The test script does not reproduce said bug, but it provides some
psychological comfort at least that the locking code works no worse
than previously.

See the docstring in the flock.py for more rationale on design
choices.

BUG=chromium:756001
TEST=test script
TEST=unit tests
TEST=Run turtle script

Change-Id: I74070ca9f4fa8118233a8ebd91880521cb47d2a8
Reviewed-on: https://chromium-review.googlesource.com/622246
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
9 files changed
tree: 707fa787557ebc03206903d12f8520b430eaf172
  1. bin/
  2. pip_packages/
  3. venv/
  4. .gitignore
  5. create_venv
  6. PRESUBMIT.cfg
  7. README.md
README.md

infra_virtualenv README

This repository provides a common Python virtualenv interface that infra code (such as chromite) can depend on.

Usage

Virtualenv users should mimic this repository, which itself uses virtualenv for running unit tests.

  1. Create a venv directory. All packages and modules in this directory will be importable inside the virtualenv.
  2. Create a requirements.txt file inside venv to list external packages to install.
  3. Copy bin/python_venv and bin/turtle which serve as templates.

Adding pre-built packages

To add packages to this repository, run:

$ pip wheel -w path/to/pip_packages -r path/to/requirements.txt

Commit the changes and make a CL.

Low level API

The bin/create_venv script prepares a virtualenv using a requirements.txt file.

$ bin/create_venv requirements.txt

The script will print the path to the virtualenv to stdout. Note that the output ends with a newline; Bash handles this, but Python does not.

To run the virtualenv Python, call bin/python under the virtualenv directory.

Together, this might look up:

$ venv=$(bin/create_venv requirements.txt)
$ ${venv}/bin/python

NOTE: it is not generally safe to run the other scripts in the virtualenv's bin directory due to hard-coded paths. Instead of running bin/pip for example, use bin/python -m pip.

Adding arbitrary directories to import path

NOTE: Do not use this for third party dependencies (stuff not owned by ChromiumOS)! This should only be used to set up imports for stuff we own. For example, importing python-MySQL SHOULD NOT use this, but importing chromite from Autotest MAY use this.

This should be handled by the minimum amount of code in the package's __init__.py file.

Example:

"""Autotest package."""

import sys

# Use the minimum amount of logic to find the path to add
_chromite_parent = 'site-packages'
sys.path.append(_chromite_parent)

A solid understanding of the Python import system is recommended (the link is for Python 3, but it is informative).

In brief, __init__.py is executed whenever a package is imported. A package is imported before any submodule or subpackage is imported. A package is only imported once per Python process; future imports are looked up in sys.modules. Thus, __init__.py will modify sys.path exactly once and is guaranteed to be run before anything in that package is imported.