Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BREAKING CHANGE: remove Milvus1DocumentStore along with support for Milvus < 2.x #3552

Merged
merged 3 commits into from
Nov 15, 2022
Merged

BREAKING CHANGE: remove Milvus1DocumentStore along with support for Milvus < 2.x #3552

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1279a97
remove milvus1
masci Nov 10, 2022
64ec674
leftover
masci Nov 10, 2022
01e28e5
revert deprecation process
masci Nov 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@ jobs:
env:
TOKENIZERS_PARALLELISM: 'false'
run: |
pytest ${{ env.PYTEST_PARAMS }} -m "not elasticsearch and not faiss and not milvus and not milvus1 and not weaviate and not pinecone and not integration" test/${{ matrix.folder }} --document_store_type=memory
pytest ${{ env.PYTEST_PARAMS }} -m "not elasticsearch and not faiss and not milvus and not weaviate and not pinecone and not integration" test/${{ matrix.folder }} --document_store_type=memory

- uses: act10ns/slack@v1
with:
Expand Down Expand Up @@ -483,7 +483,7 @@ jobs:
env:
TOKENIZERS_PARALLELISM: 'false'
run: |
pytest ${{ env.PYTEST_PARAMS }} -m "not elasticsearch and not faiss and not milvus and not milvus1 and not weaviate and not pinecone and not integration" ${{ env.SUITES_EXCLUDED_FROM_WINDOWS }} test/${{ matrix.folder }} --document_store_type=memory
pytest ${{ env.PYTEST_PARAMS }} -m "not elasticsearch and not faiss and not milvus and not weaviate and not pinecone and not integration" ${{ env.SUITES_EXCLUDED_FROM_WINDOWS }} test/${{ matrix.folder }} --document_store_type=memory

- uses: act10ns/slack@v1
with:
Expand Down
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,13 @@ Haystack is built in a modular fashion so that you can combine the best technolo

| | |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| :ledger: [Docs](https://docs.haystack.deepset.ai) | Components, Pipeline Nodes, Guides, API Reference |
| :ledger: [Docs](https://docs.haystack.deepset.ai) | Components, Pipeline Nodes, Guides, API Reference |
| :floppy_disk: [Installation](https://github.com/deepset-ai/haystack#floppy_disk-installation) | How to install Haystack |
| :mortar_board: [Tutorials](https://github.com/deepset-ai/haystack#mortar_board-tutorials) | See what Haystack can do with our Notebooks & Scripts |
| :beginner: [Quick Demo](https://github.com/deepset-ai/haystack#beginner-quick-demo) | Deploy a Haystack application with Docker Compose and a REST API |
| :vulcan_salute: [Community](https://github.com/deepset-ai/haystack#vulcan_salute-community) | [Discord](https://haystack.deepset.ai/community/join), [Twitter](https://twitter.com/deepset_ai), [Stack Overflow](https://stackoverflow.com/questions/tagged/haystack), [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) |
| :heart: [Contributing](https://github.com/deepset-ai/haystack#heart-contributing) | We welcome all contributions! |
| :bar_chart: [Benchmarks](https://haystack.deepset.ai/benchmarks/) | Speed & Accuracy of Retriever, Readers and DocumentStores |
ZanSara marked this conversation as resolved.
Show resolved Hide resolved
| :bar_chart: [Benchmarks](https://haystack.deepset.ai/benchmarks/) | Speed & Accuracy of Retriever, Readers and DocumentStores |
| :telescope: [Roadmap](https://haystack.deepset.ai/overview/roadmap) | Public roadmap of Haystack |
| :newspaper: [Blog](https://medium.com/deepset-ai) | Read our articles on Medium |
| :phone: [Jobs](https://www.deepset.ai/jobs) | We're hiring! Have a look at our open positions |
Expand Down Expand Up @@ -96,8 +96,8 @@ pip install -e '.[all]' ## or 'all-gpu' for the GPU-enabled dependencies
```

If you cannot upgrade `pip` to version 21.3 or higher, you will need to replace:
- `'.[all]'` with `'.[sql,only-faiss,only-milvus1,weaviate,graphdb,crawler,preprocessing,ocr,onnx,ray,dev]'`
- `'.[all-gpu]'` with `'.[sql,only-faiss-gpu,only-milvus1,weaviate,graphdb,crawler,preprocessing,ocr,onnx-gpu,ray,dev]'`
- `'.[all]'` with `'.[sql,only-faiss,only-milvus,weaviate,graphdb,crawler,preprocessing,ocr,onnx,ray,dev]'`
- `'.[all-gpu]'` with `'.[sql,only-faiss-gpu,only-milvus,weaviate,graphdb,crawler,preprocessing,ocr,onnx-gpu,ray,dev]'`

For an complete list of the dependency groups available, have a look at the `haystack/pyproject.toml` file.

Expand Down
4 changes: 1 addition & 3 deletions conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
def pytest_addoption(parser):
parser.addoption(
"--document_store_type",
action="store",
default="elasticsearch, faiss, sql, memory, milvus1, milvus, weaviate, pinecone",
"--document_store_type", action="store", default="elasticsearch, faiss, sql, memory, milvus, weaviate, pinecone"
)
parser.addoption(
"--mock-dc", action="store_true", default=True, help="Mock HTTP requests to dC while running tests"
Expand Down
5 changes: 1 addition & 4 deletions haystack/document_stores/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,7 @@
SQLDocumentStore = safe_import("haystack.document_stores.sql", "SQLDocumentStore", "sql")
FAISSDocumentStore = safe_import("haystack.document_stores.faiss", "FAISSDocumentStore", "faiss")
PineconeDocumentStore = safe_import("haystack.document_stores.pinecone", "PineconeDocumentStore", "pinecone")
if os.getenv("MILVUS1_ENABLED"):
MilvusDocumentStore = safe_import("haystack.document_stores.milvus1", "Milvus1DocumentStore", "milvus1")
else:
MilvusDocumentStore = safe_import("haystack.document_stores.milvus2", "Milvus2DocumentStore", "milvus")
MilvusDocumentStore = safe_import("haystack.document_stores.milvus", "MilvusDocumentStore", "milvus")
WeaviateDocumentStore = safe_import("haystack.document_stores.weaviate", "WeaviateDocumentStore", "weaviate")
GraphDBKnowledgeGraph = safe_import("haystack.document_stores.graphdb", "GraphDBKnowledgeGraph", "graphdb")
InMemoryKnowledgeGraph = safe_import(
Expand Down
16 changes: 8 additions & 8 deletions haystack/document_stores/milvus2.py → haystack/document_stores/milvus.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
logger = logging.getLogger(__name__)


class Milvus2DocumentStore(SQLDocumentStore):
class MilvusDocumentStore(SQLDocumentStore):
"""
Limitations:
Milvus 2.0 so far doesn't support the deletion of documents (https://github.com/milvus-io/milvus/issues/7130).
Expand Down Expand Up @@ -251,7 +251,7 @@ def write_documents(
:return:
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

index = index or self.index
index_param = index_param or self.index_param
Expand Down Expand Up @@ -412,7 +412,7 @@ def query_by_embedding(
:return:
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

index = index or self.index
has_collection = utility.has_collection(collection_name=index)
Expand Down Expand Up @@ -468,7 +468,7 @@ def delete_documents(
:return: None
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

if ids:
self._delete_vector_ids_from_milvus(ids=ids, index=index)
Expand Down Expand Up @@ -529,7 +529,7 @@ def get_all_documents_generator(
:param batch_size: When working with large number of documents, batching can help reduce memory footprint.
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

index = index or self.index
documents = super().get_all_documents_generator(index=index, filters=filters, batch_size=batch_size)
Expand Down Expand Up @@ -560,7 +560,7 @@ def get_all_documents(
:param batch_size: When working with large number of documents, batching can help reduce memory footprint.
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

index = index or self.index
result = self.get_all_documents_generator(
Expand All @@ -580,7 +580,7 @@ def get_document_by_id(
DocumentStore's default index (self.index) will be used.
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

documents = self.get_documents_by_id([id], index)
document = documents[0] if documents else None
Expand All @@ -602,7 +602,7 @@ def get_documents_by_id(
:param batch_size: When working with large number of documents, batching can help reduce memory footprint.
"""
if headers:
raise NotImplementedError("Milvus2DocumentStore does not support headers.")
raise NotImplementedError("MilvusDocumentStore does not support headers.")

index = index or self.index
documents = super().get_documents_by_id(ids=ids, index=index, batch_size=batch_size)
Expand Down
Loading