Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding filtering to memory - draft #1

Merged
merged 70 commits into from
Jul 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
24b4d44
Adding draft filtering changes
May 25, 2023
bffb431
Merge branch 'microsoft:main' into adding-filtering-to-memory
lukasz-appstream May 29, 2023
e1c8c11
Adding new qdrant filter class and general MemoryFilter
May 30, 2023
ed882b0
Changing approach to QdrantFilter as a generic argument for IMemoryStore
May 31, 2023
ac490cb
Adding generic memory storage available in the context.
May 31, 2023
0d06b04
Adding json attributes to qdrant filter object
May 31, 2023
c11b823
Adding filterable memory getter to kernel plus missing comments.
May 31, 2023
b7839aa
Styling changes; Tests fixed
piotrek-appstream Jun 1, 2023
b6975e6
Missing IValidatable in RangeCondition
piotrek-appstream Jun 1, 2023
1798376
Merge branch 'microsoft:main' into adding-filtering-to-memory
lukasz-appstream Jun 2, 2023
3e48387
Making System.Text.Json serialize properties of Condition-derived cla…
piotrek-appstream Jun 2, 2023
2ba112e
Creating filterable SemanticTextMemory
piotrek-appstream Jun 5, 2023
2985691
Creating ISemanticTextMemory moved to a separate method
piotrek-appstream Jun 5, 2023
c7dd15a
Merge branch 'main' into adding-filtering-to-memory
piotrek-appstream Jun 10, 2023
f8066a3
Using JsonDerivedTypeAttribute for QdrantFilter conditions
piotrek-appstream Jun 10, 2023
e2db0bf
Test added + styling changes
piotrek-appstream Jun 12, 2023
2e8751b
Allowing empty Conditions in QdrantFilter
piotrek-appstream Jun 12, 2023
fd0ed59
Update IMemoryStore.cs
lukasz-appstream Jun 13, 2023
a2a9eeb
Merge branch 'microsoft:main' into adding-filtering-to-memory
lukasz-appstream Jun 14, 2023
acf44e1
Merge branch 'main' into adding-filtering-to-memory
lukasz-appstream Jun 17, 2023
d539d2e
Merge branch 'microsoft:main' into adding-filtering-to-memory
piotrek-appstream Jun 20, 2023
8a65001
Merge branch 'microsoft:main' into adding-filtering-to-memory
lukasz-appstream Jun 22, 2023
f16c52a
Merge branch 'microsoft:main' into adding-filtering-to-memory
lukasz-appstream Jun 27, 2023
a373164
Copilot Chat: Adding missing appsettings chatparticipant config to bi…
teresaqhoang Jun 30, 2023
348c1c8
Chroma - fix for saving same memory record multiple times (#1790)
dmytrostruk Jun 30, 2023
cd5120f
Copilot Chat: Fix ESLint violations + enable linter during build (#1710)
dehoward Jun 30, 2023
3105aa1
Copilot Chat: align bash deploy-webapi script with powershell (#1786)
dehoward Jun 30, 2023
6600204
Remove some unnecessary null checks (#1801)
stephentoub Jul 3, 2023
bb79d41
Fixed function parameter not being used for LoadFromFile (#1803)
Chinouchi Jul 3, 2023
4eaee80
Update Redis Connector README.md with various options to get RediSear…
shpathak-msft Jul 3, 2023
93a172b
Notebook showing how to leverage Chroma for Semantic Memory (#1785)
dluc Jul 3, 2023
b139203
Improve postgres memory (#1794)
JadynWong Jul 3, 2023
c6f783d
Python: Fix for SemanticTextMemory Embedding issue (#1816)
cschadewitz Jul 3, 2023
fccba80
Update java-format.yml
shawncal Jul 3, 2023
5bf9525
Update java-format.yml
shawncal Jul 3, 2023
d4b582e
Update labeler.yml
shawncal Jul 3, 2023
2cb8a9c
Fix in labeler.yml (#1822)
gitri-ms Jul 3, 2023
0ed9c06
Create label-issues.yml
shawncal Jul 3, 2023
3e6b45c
Create label-title-prefix.yml
shawncal Jul 3, 2023
aafb6e7
Update label-title-prefix.yml
shawncal Jul 3, 2023
a447795
Update label-title-prefix.yml
shawncal Jul 3, 2023
b6628b0
Update label-title-prefix.yml
shawncal Jul 3, 2023
cc34648
Update label-title-prefix.yml
shawncal Jul 3, 2023
84ad23e
Update label-title-prefix.yml
shawncal Jul 3, 2023
8c0d25d
Update label-title-prefix.yml
shawncal Jul 3, 2023
9c427f9
Update label-title-prefix.yml
shawncal Jul 3, 2023
5eea92e
Python: Bump ruff from 0.0.275 to 0.0.276 in /python (#1829)
dependabot[bot] Jul 3, 2023
52bf7e5
Bump Microsoft.NET.Test.Sdk from 17.6.2 to 17.6.3 in /dotnet (#1825)
dependabot[bot] Jul 3, 2023
b20d175
Update labeler.yml
shawncal Jul 3, 2023
175a27f
Update label-title-prefix.yml
shawncal Jul 4, 2023
48acc1f
Update dependabot.yml
shawncal Jul 4, 2023
e7ba201
Downgrade several 7.0.* packages to 6.0.* (#1799)
stephentoub Jul 4, 2023
4e27f32
Bump Microsoft.Azure.Functions.Worker.Sdk from 1.10.0 to 1.11.0 in /d…
dependabot[bot] Jul 4, 2023
f70b444
Python: Bump ipykernel from 6.23.3 to 6.24.0 in /python (#1827)
dependabot[bot] Jul 4, 2023
aea0789
.Net: Integration Test Fix
shawncal Jul 4, 2023
a73cc1b
Update label-title-prefix.yml
shawncal Jul 5, 2023
7b1e4de
Update label-title-prefix.yml
shawncal Jul 5, 2023
309cd75
Update label-title-prefix.yml
shawncal Jul 5, 2023
8a271e8
Update label-title-prefix.yml
shawncal Jul 5, 2023
65beb75
Update label-title-prefix.yml
shawncal Jul 5, 2023
eac21ed
Update links in Connectors.Memory.Redis README (#1852)
gitri-ms Jul 5, 2023
6286178
Update dependabot.yml to ignore Microsoft.Bcl.* major versions as wel…
stephentoub Jul 5, 2023
32d61a0
Copilot Chat: Url DataAnnotation is not compatible with Uri. Should b…
jpaarhuis Jul 5, 2023
451b640
Feature/notebook bing search (#1729)
N-E-W-T-O-N Jul 5, 2023
1008504
Python: Bump ruff from 0.0.276 to 0.0.277 in /python (#1859)
dependabot[bot] Jul 5, 2023
899f94c
Removing Copilot Chat from the SK-dotnet.sln (#1830)
shawncal Jul 5, 2023
0a14f1c
.Net: Bump Grpc.Net.Client from 2.54.0 to 2.55.0 in /dotnet (#1858)
dependabot[bot] Jul 5, 2023
db040e2
Python: Enable pinecone tests to run in CI (#1864)
awharrison-28 Jul 5, 2023
5b35b4b
Python: Make pinecone secrets accessible to integration tests (#1865)
awharrison-28 Jul 5, 2023
c4c4861
Merge branch 'microsoft:main' into adding-filtering-to-memory
piotrek-appstream Jul 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 31 additions & 2 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,57 @@

version: 2
updates:
# Maintain dependencies for nuget
- package-ecosystem: "nuget"
directory: "samples/apps/copilot-chat-app/webapi"
schedule:
interval: "weekly"
day: "monday"
labels:
- "copilot chat"
- "dependencies"

# Maintain dependencies for nuget
- package-ecosystem: "nuget"
directory: "dotnet/"
schedule:
interval: "weekly"
day: "monday"
ignore:
# For all System.* and Microsoft.Extensions/Bcl.* packages, ignore all major version updates
- dependency-name: "System.*"
update-types: ["version-update:semver-major"]
- dependency-name: "Microsoft.Extensions.*"
update-types: ["version-update:semver-major"]
- dependency-name: "Microsoft.Bcl.*"
update-types: ["version-update:semver-major"]
labels:
- ".NET"
- "dependencies"

# Maintain dependencies for nuget
- package-ecosystem: "nuget"
directory: "samples/"
directory: "samples/dotnet"
schedule:
interval: "weekly"

day: "monday"

# Maintain dependencies for npm
- package-ecosystem: "npm"
directory: "samples/apps"
schedule:
interval: "weekly"
day: "monday"

# Maintain dependencies for pip
- package-ecosystem: "pip"
directory: "python/"
schedule:
interval: "weekly"
day: "monday"
labels:
- "python"
- "dependencies"

# Maintain dependencies for github-actions
- package-ecosystem: "github-actions"
Expand All @@ -36,3 +64,4 @@ updates:
directory: "/"
schedule:
interval: "weekly"
day: "monday"
31 changes: 24 additions & 7 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Add 'kernel' label to any change within Connectors, Extensions, Skills, and tests directories
kernel:
- "dotnet/src/Connectors/**/*"
- "dotnet/src/Extensions/**/*"
- "dotnet/src/Skills/**/*"
- "dotnet/src/IntegrationTests/**/*"
- "dotnet/src/SemanticKernel.UnitTests/**/*"
- dotnet/src/Connectors/**/*
- dotnet/src/Extensions/**/*
- dotnet/src/Skills/**/*
- dotnet/src/IntegrationTests/**/*
- dotnet/src/SemanticKernel.UnitTests/**/*

# Add 'kernel.core' label to any change within the 'SemanticKernel', 'SemanticKernel.Abstractions', or 'SemanticKernel.MetaPackage' directories
kernel.core:
Expand All @@ -16,11 +16,28 @@ kernel.core:
python:
- python/**/*

# Add 'java' label to any change within the 'java' directory
java:
- java/**/*

# Add 'samples' label to any change within the 'samples' directory
samples:
- samples/**/*

# Add '.NET' label to any change within samples or kernel 'dotnet' directories.
.NET:
- dotnet/src/**/*
- samples/**/dotnet/**/*
- dotnet/**/*

# Add 'copilot chat' label to any change within the 'samples/apps/copilot-chat-app' directory
copilot chat:
- samples/apps/copilot-chat-app/**/*

# Add 'documentation' label to any change within the 'docs' directory, or any '.md' files
documentation:
- docs/**/*
- '**/*.md'

# Add 'memory' label to any memory connectors in dotnet/ or python/
memory:
- dotnet/src/Connectors/Connectors.Memory.*/**/*
- python/semantic_kernel/connectors/memory/**/*
2 changes: 0 additions & 2 deletions .github/workflows/dotnet-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ on:
paths:
- 'dotnet/**'
- 'samples/dotnet/**'
- '**.cs'
- '**.csproj'

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/java-format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ jobs:
pull-requests: write

steps:
- name: Check for command
id: command
uses: xt0rted/slash-command-action@v2
continue-on-error: true
with:
command: spotless
reaction-type: "eyes"

- name: Get command
env:
BODY: ${{ github.event.comment.body }}
Expand Down Expand Up @@ -77,4 +85,4 @@ jobs:
"
gh pr comment $NUMBER --body "$body"
fi
working-directory: java
working-directory: java
44 changes: 44 additions & 0 deletions .github/workflows/label-issues.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Label issues
on:
issues:
types:
- reopened
- opened

jobs:
label_issues:
name: "Issue: add labels"
if: ${{ github.event.action == 'opened' || github.event.action == 'reopened' }}
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/github-script@v6
with:
script: |
// Get the issue body and title
const body = context.payload.issue.body
let title = context.payload.issue.title

// Define the labels array
let labels = ["triage"]

// Check if the body or the title contains the word 'python' (case-insensitive)
if (body.match(/python/i) || title.match(/python/i)) {
// Add the 'python' label to the array
labels.push("python")
}

// Check if the body or the title contains the word 'java' (case-insensitive)
if (body.match(/java/i) || title.match(/java/i)) {
// Add the 'java' label to the array
labels.push("java")
}

// Add the labels to the issue
github.rest.issues.addLabels({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
labels: labels
});
97 changes: 97 additions & 0 deletions .github/workflows/label-title-prefix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Label title prefix
on:
issues:
types: [ labeled ]
pull_request:
types: [ labeled ]

jobs:
add_title_prefix:
name: "Issue/PR: add title prefix"
# Define a matrix of label and prefix pairs
strategy:
matrix:
include:
- {label: 'python', prefix: 'Python'}
- {label: 'java', prefix: 'Java'}
- {label: '.NET', prefix: '.Net'}
- {label: 'copilot chat', prefix: 'Copilot Chat'}

runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write

steps:
- uses: actions/github-script@v6
name: "Issue/PR: update title"
with:
script: |
// Get the label and prefix from the matrix
const label = '${{ matrix.label }}'
const prefix = '${{ matrix.prefix }}'

labelAdded = context.payload.label.name

// Write the contents of context to console
core.info(JSON.stringify(context, null, 2))

// Get the event name, title and labels
let title
switch(context.eventName) {
case 'issues':
title = context.payload.issue.title
break
case 'pull_request':
title = context.payload.pull_request.title
break
default:
core.setFailed('Unrecognited eventName: ' + context.eventName)
}

let originalTitle = title

// Update the title based on the label and prefix
// Check if the issue or PR has the label
if (labelAdded == label) {
// Check if the title starts with the prefix (case-sensitive)
if (!title.startsWith(prefix + ": ")) {
// If not, check if the first word is the label (case-insensitive)
if (title.match(new RegExp(`^${prefix}`, 'i'))) {
// If yes, replace it with the prefix (case-sensitive)
title = title.replace(new RegExp(`^${prefix}`, 'i'), prefix)
} else {
// If not, prepend the prefix to the title
title = prefix + ": " + title
}
}
}

// Update the issue or PR title, if changed
if (title != originalTitle ) {
switch(context.eventName) {
case 'issues':
github.rest.issues.update({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
title: title
});
break
case 'pull_request':
try {
github.rest.pulls.update({
pull_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
title: title
});
}
catch (err) {
core.info("Update PR title failed: " + err.message)
}
break
default:
core.setFailed('Unrecognited eventName: ' + context.eventName)
}
}
6 changes: 2 additions & 4 deletions .github/workflows/node-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,8 @@ jobs:
yarndirs=()
for lockfile in samples/apps/**/yarn.lock; do # loop over all yarn.lock files
dir=$(dirname "$lockfile") # get the directory of the lock file
if [[ "$dir" != "samples/apps" ]]; then # exclude samples/apps directory
echo "Found yarn project in $dir"
yarndirs+=("$dir") # add the directory to the yarndirs array
fi
echo "Found yarn project in $dir"
yarndirs+=("$dir") # add the directory to the yarndirs array
done

echo "All yarn projects found: '${yarndirs[*]}'"
Expand Down
Loading
Loading