This project contains the code written by MITRE during our evaluation of the most prominent quantum computing software frameworks in 2019. The evaluation is meant to gauge the software engineering experience in using each framework on a daily basis, to assess their strengths, weaknesses, and applicability to our work program.
Each framework is evaluated and scored according to these criteria:
- Ease of Use (learning curve, documentation, language features, control flow, development tooling, debugging support)
- Maturity and Activity (community size, frequency of updates, support, standard library functionality)
- Flexibility and Modularity (algorithm support, platform independence, compilation process, open source status)
- Additional Criteria (compiler and simulator performance, quantum error correction support, hardware tooling support, cost of access and training)
We selected eight full-stack frameworks for evaluation, based on the list maintained by the Quantum Open Source Foundation. The frameworks being evaluated are:
- Microsoft's Quantum Development Kit (Q#)
- IBM's Quantum Information Sciences Kit (Qiskit)
- Cirq (formerly by Google)
- Rigetti's Pyquil / Forest
- D-Wave's Ocean
- ProjectQ
- Eclipse's XACC
- Xanadu's Strawberry Fields
Our evaluation approach is to implement a standard set of well-known quantum algorithms and functions in each language, and score our experience against each of the evaluation criteria as we go. The algorithms we selected are as follows:
- Fundamentals
- Superposition
- Entanglement
- Quantum Telportation
- Superdense Coding
- Error Correction
- The simple bit-flip error code (3 qubits)
- Shor's ECC (9 qubits)
- Steane's ECC (7 qubits)
- Oracle Algorithms
- Non-Oracle Algorithms
This repository contains our implementations of these algorithms in each of the quantum software frameworks.
All of our code was built and executed using Visual Studio 2019. Each framework comes with a Visual Studio solution. There are four projects contained within each solution, corresponding to the four categories of standard test algorithms. The algorithms have been implemented within these projects, and each project contains a set of unit tests that ensure the algorithm works for various inputs. Use Visual Studio's built-in Test Explorer to run them, or use your own test runner of choice.
Installation instructions vary per framework; each framework's folder contains a README that describes the installation procedure, or at least how we set up our own machines during our evaluation.
Copyright (C) 2019 The MITRE Corporation.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
This project contains content developed by The MITRE Corporation. If this code is used in a deployment or embedded within another project, it is requested that you send an email to opensource@mitre.org in order to let us know where this software is being used.