Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow lightning.qubit to skip C++ compilation and provide a Python-only binary #129

Merged
merged 23 commits into from
Aug 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@

### Improvements

* PennyLane-Lightning can now be installed without compiling its C++ binaries and will fall back
to using the `default.qubit` implementation. Skipping compilation is achieved by setting the
`SKIP_COMPILATION` environment variable, e.g., Linux/MacOS: `export SKIP_COMPILATION=True`,
Windows: `set SKIP_COMPILATION=True`. This feature is intended for building a pure-Python wheel of
PennyLane-Lightning as a backup for platforms without a dedicated wheel.
[(129)](https://github.com/PennyLaneAI/pennylane-lightning/pull/129)

* The C++-backed Python bound methods can now be directly called with wires and supplied parameters.
[(125)](https://github.com/PennyLaneAI/pennylane-lightning/pull/125)

Expand Down
215 changes: 120 additions & 95 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -1,95 +1,120 @@
name: Deploy
on:
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-*'

# MacOS specific build settings

CIBW_BEFORE_ALL_MACOS: |
brew cask uninstall --force oclint
brew install gcc libomp

# Python build settings

CIBW_BEFORE_BUILD: |
pip install numpy==1.19.5 scipy pybind11

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy scipy pytest pytest-cov pytest-mock flaky

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report


jobs:

build-wheels:
name: ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.4.1
with:
access_token: ${{ github.token }}

- uses: actions/checkout@v2

- uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Install cibuildwheel
run: |
python -m pip install --upgrade pip
python -m pip install cibuildwheel==1.5.5

- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse

- uses: actions/upload-artifact@v2
with:
name: ${{ runner.os }}-wheels.zip
path: ./wheelhouse/*.whl

upload-wheels:
runs-on: ubuntu-latest
needs: [build-wheels]
steps:
- uses: actions/download-artifact@v2
with:
path: dist/

- name: Publish
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.PYPI }}

upload-source:
runs-on: ubuntu-latest
needs: [build-wheels]
steps:
- uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.7

- name: Build and install Plugin
run: |
python -m pip install --upgrade pip wheel
python setup.py sdist

- name: Publish
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.PYPI }}
name: Deploy
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the diff for this file is not great, I believe my editor changed from \r\n to \n for the new line style. The main changes are highlighted below.

on:
release:
types: [published]

env:
CIBW_BUILD: 'cp37-* cp38-* cp39-*'

# MacOS specific build settings

CIBW_BEFORE_ALL_MACOS: |
brew cask uninstall --force oclint
brew install gcc libomp

# Python build settings

CIBW_BEFORE_BUILD: |
pip install numpy==1.19.5 scipy pybind11

# Testing of built wheels
CIBW_TEST_REQUIRES: numpy scipy pytest pytest-cov pytest-mock flaky

CIBW_TEST_COMMAND: |
pl-device-test --device=lightning.qubit --skip-ops -x --tb=short --no-flaky-report


jobs:

build-wheels:
name: ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder can we extract the artifacts create from the other wheel builder scripts, rather than rebuild here. Assuming a successful deploy, we fail to get the new architectures using this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get the new build scripts (wheel_linux_aarch64.yml, etc) to run also on the published trigger (e.g., like this deploy workflow). We could then have an additional job in each workflow to upload the wheels if the event is published (if: ${{ github.event_name == 'published' }}, taken from here)?

We can then:

  • Take the pure python build into it's own workflow, as you suggested below
  • Reduce this deploy workflow to just uploading the source?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that sounds like a good plan.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Are you ok if we do that in a separate PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, perfectly fine with me.

steps:
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.4.1
with:
access_token: ${{ github.token }}

- uses: actions/checkout@v2

- uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Install cibuildwheel
run: |
python -m pip install --upgrade pip
python -m pip install cibuildwheel==1.5.5

- name: Build wheels
run: |
python -m cibuildwheel --output-dir wheelhouse

- uses: actions/upload-artifact@v2
with:
name: ${{ runner.os }}-wheels.zip
path: ./wheelhouse/*.whl

build-pure-python-wheel:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the above comment, I wonder if this should be offloaded into a separate build file, as the case for the other architectures. What do you think?

runs-on: ubuntu-latest
steps:
- name: Checkout PennyLane-Lightning
uses: actions/checkout@v2
with:
path: main

- uses: actions/setup-python@v2
with:
python-version: '3.7'

- name: Build wheels
run: |
python -m pip install --upgrade pip wheel
cd main
python setup.py bdist_wheel
env:
SKIP_COMPILATION: True

- uses: actions/upload-artifact@v2
with:
name: pure-python-wheels.zip
path: main/dist/*.whl
Comment on lines +61 to +84
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main change


upload-wheels:
runs-on: ubuntu-latest
needs: [build-wheels, build-pure-python-wheel]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also added job as a dependency

steps:
- uses: actions/download-artifact@v2
with:
path: dist/

- name: Publish
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.PYPI }}

upload-source:
runs-on: ubuntu-latest
needs: [build-wheels, build-pure-python-wheel]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as dependency, but not sure it's strictly needed.

steps:
- uses: actions/checkout@v2

- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: 3.7

- name: Build and install Plugin
run: |
python -m pip install --upgrade pip wheel
python setup.py sdist

- name: Publish
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.PYPI }}
58 changes: 58 additions & 0 deletions .github/workflows/tests_without_binary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Testing without binary
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very similar to the tests.yml workflow but with the c++ tests removed and a few tweaks to test only for without binaries.

on:
push:
branches:
- master
pull_request:

env:
COVERAGE_FLAGS: "--cov=pennylane_lightning --cov-report=term-missing --cov-report=xml:../coverage.xml --no-flaky-report -p no:warnings --tb=native"

jobs:
pythontests:
name: Python tests
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04]

steps:
- name: Cancel previous runs
uses: styfle/cancel-workflow-action@0.4.1
with:
access_token: ${{ github.token }}

- name: Checkout PennyLane-Lightning
uses: actions/checkout@v2
with:
path: main

- uses: actions/setup-python@v2
name: Install Python
with:
python-version: '3.7'

- name: Get required Python packages
run: |
cd main
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Install lightning.qubit device
run: |
cd main
pip install -e .
env:
SKIP_COMPILATION: True

- name: Run PennyLane-Lightning unit tests
run: |
cd main/
pytest tests/ $COVERAGE_FLAGS
pl-device-test --device lightning.qubit --skip-ops --shots=20000 $COVERAGE_FLAGS --cov-append
pl-device-test --device lightning.qubit --shots=None --skip-ops $COVERAGE_FLAGS --cov-append

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1.0.12
with:
file: coverage.xml
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sphinx:
configuration: doc/conf.py

python:
version: 3.9
version: 3.8
install:
- requirements: doc/requirements.txt
- requirements: requirements.txt
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ clean:
rm -rf .coverage coverage_html_report/
rm -rf tmp
rm -rf *.dat
rm -rf pennylane_lightning/lightning_qubit_ops*

docs:
make -C doc html
Expand Down
30 changes: 17 additions & 13 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,29 +51,33 @@ PennyLane-Lightning requires Python version 3.7 and above. It can be installed u

$ pip install pennylane-lightning

Alternatively, to build PennyLane-Lightning from source you can run
To build PennyLane-Lightning from source you can run

.. code-block:: console

$ git clone https://github.com/XanaduAI/pennylane-lightning.git
$ cd pennylane-lightning
$ pip install -e .
$ pip install pybind11 pennylane-lightning --no-binary :all:

Note that subsequent calls to ``pip install -e .`` will use cached binaries stored in the
``build`` folder. Run ``make clean`` if you would like to recompile.
A C++ compiler such as ``g++``, ``clang``, or ``MSVC`` is required. On Debian-based systems, this
can be installed via ``apt``:

.. code-block:: console

The following dependencies are required to install PennyLane-Lightning:
$ sudo apt install g++

* A C++ compiler, such as ``g++``, ``clang``, or ``MSVC``.
* `pybind11 <https://pybind11.readthedocs.io/en/stable/>`_ a library for binding C++
functionality to Python.
The `pybind11 <https://pybind11.readthedocs.io/en/stable/>`_ library is also used for binding the
C++ functionality to Python.

On Debian-based systems, these can be installed via ``apt`` and ``pip``:
Alternatively, for development and testing, you can install by cloning the repository:

.. code-block:: console

$ sudo apt install g++
$ pip install pybind11
$ git clone https://github.com/XanaduAI/pennylane-lightning.git
$ cd pennylane-lightning
$ pip install -r requirements.txt
$ pip install -e .

Note that subsequent calls to ``pip install -e .`` will use cached binaries stored in the
``build`` folder. Run ``make clean`` if you would like to recompile.

Testing
-------
Expand Down
31 changes: 30 additions & 1 deletion pennylane_lightning/lightning_qubit.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,18 @@
This module contains the :class:`~.LightningQubit` class, a PennyLane simulator device that
interfaces with C++ for fast linear algebra calculations.
"""
from warnings import warn

from pennylane.devices import DefaultQubit
import numpy as np
from pennylane import QubitStateVector, BasisState, DeviceError, QubitUnitary
from .lightning_qubit_ops import apply, StateVectorC64, StateVectorC128

try:
from .lightning_qubit_ops import apply, StateVectorC64, StateVectorC128

CPP_BINARY_AVAILABLE = True
except ModuleNotFoundError:
CPP_BINARY_AVAILABLE = False

from ._version import __version__

Expand Down Expand Up @@ -122,3 +130,24 @@ def apply_lightning(self, state, operations):
method(wires, inv, param)

return np.reshape(state_vector, state.shape)


if not CPP_BINARY_AVAILABLE:

class LightningQubit(DefaultQubit):

name = "Lightning Qubit PennyLane plugin"
short_name = "lightning.qubit"
pennylane_requires = ">=0.15"
version = __version__
author = "Xanadu Inc."

def __init__(self, *args, **kwargs):
warn(
"Pre-compiled binaries for lightning.qubit are not available. Falling back to "
"using the Python-based default.qubit implementation. To manually compile from "
"source, follow the instructions at "
"https://pennylane-lightning.readthedocs.io/en/latest/installation.html.",
UserWarning,
)
super().__init__(*args, **kwargs)
Loading