-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fields always defined as string collection in app service #88
Comments
Can you advise on how you are setting your field definitions? Fields by default are Collection(Edm.String) unless explicitly specified. |
We've set these up in a way mentioned in your docs.
It was very intermittent but would always work locally. The bizarre thing is that it's now started to work correctly in azure. The only change we've made is the switch to the client's domain rather than using the *.azurewebsites.net one. |
Hi, In what part of your code do you have that code above for configuring the fields? To me this issue sounds like some sort of race condition or that somehow that Configure block of code is not being called for some reason. You could try to change that to PostConfigure instead of Configure. Do you have any errors in your logs that might indicate a related issue? You could also try configuring options like this so that you can get some log statements output: services.AddSingleton<IConfigureOptions<AzureSearchIndexOptions>>(
sp => new ConfigureNamedOptions<AzureSearchIndexOptions>("InternalIndex", options =>
{
var logger = sp.GetRequiredService<ILogger<Startup>>();
logger.LogInformation("Configuring ExamineX options...");
options.FieldDefinitions.AddOrUpdate(new FieldDefinition("FieldName", AzureSearchFieldDefinitionTypes.FullText));
})); |
We have this code in the There's nothing useful in the logs. I'll try sticking in some logging as you've suggested. |
Hi @jayfive, Did you try using the PostConfigure of configuring IOptions? You need to do this in ConfigureServices because you are configuring services in the DI container - namely the ExamineX IOptions. |
@Shazwazza I am having a similar issue. My field definitions seem to be set intermittently, both locally and when hosted on IIS on an on-prem server. I can't confirm if this also happens on Azure App Services just yet. I was previously using I have tried your suggestion above of using I have also tried your logging code and can confirm that the FieldDefinitions are being picked up as the log entry appears but there are no logs suggesting any other issues. My public void ConfigureServices(IServiceCollection services)
{
var config = services
.AddUmbraco(_env, _config)
.AddBackOffice()
.AddWebsite()
.AddComposers()
.AddCustomAppsettingSections()
.AddCustomBackofficeSections()
.AddCustomServices()
.AddCustomContentFinders()
.AddCustomConfigureOptions()
.AddCustomComponents()
.AddCustomNotificationHandlers();
...
} The FieldDefinitions are in the public static IUmbracoBuilder AddCustomConfigureOptions(this IUmbracoBuilder builder)
{
builder.Services.PostConfigure<AzureSearchIndexOptions>(UmbracoIndexes.ExternalIndexName,
options =>
{
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.HomeId, AzureSearchFieldDefinitionTypes.Integer));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Geolocation.Latitudes, AzureSearchFieldDefinitionTypes.Double));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Geolocation.Longitudes, AzureSearchFieldDefinitionTypes.Double));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Schema.PageTitle, AzureSearchFieldDefinitionTypes.Raw));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Sorting.CustomSortIndex, AzureSearchFieldDefinitionTypes.Integer));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Sorting.SortDateTime, AzureSearchFieldDefinitionTypes.DateTime));
options.FieldDefinitions.AddOrUpdate(new FieldDefinition(ExamineConstants.Fields.Sorting.UpdateDateTime, AzureSearchFieldDefinitionTypes.DateTime));
}
);
return builder;
} This produces the following results when doing the following search in Azure Search in the Azure Portal: {
"search": "*",
"select": "homeId,latitudes,longitudes,pageTitle,customSortIndex,sortDateTime,updateDateTime"
} External index - Local {
"@search.score": 1,
"sortDateTime": "2023-08-24T12:00:00Z",
"latitudes": null,
"longitudes": null,
"pageTitle": "Your medical team ",
"updateDateTime": "2023-08-24T12:00:00Z",
"customSortIndex": 0,
"homeId": 1057
} External index - Test {
"@search.score": 1,
"updateDateTime": [
"24/08/2023 12:00:00 PM"
],
"latitudes": null,
"pageTitle": [
"Your medical team "
],
"customSortIndex": 0,
"sortDateTime": [
"24/08/2023 12:00:00 PM"
],
"homeId": [
"1057"
],
"longitudes": null
} The results are clearly different and it's as if the FieldDefinitions are not picked up when the same code runs on the Test environment. I have tried deleting the index and rebuilding multiple times with the same results. Can you suggest anything we could try? Thanks Thanks |
Hi @chrden are you also using any composers to modify your indexes? (i.e. lucene indexes?) |
@Shazwazza We believe this may be related to race conditions. We will keep investigating and report back if necessary. Thanks 👍 |
Hello, we are experiencing a similar issue. We have a custom field which I defined like this:
I have deployed it to the server, in six websites, MyField index type is String but only one of them is Collection, and this makes my search broken on that site. Also locally, with the same database is String as well. |
Hi @pariashiri and others this discussion may be of help #87 (comment) The main point is - if you are using Umbraco composers to customize The reason for that is because there is no guarantee the order in which Umbraco will find and load composers and if you are customizing I will update the docs with regards to this too. Unfortunately there isn't a magic way for ExamineX composers to always run after your own. In future ExamineX versions, we will probably remove ExamineX from being composed by composers and move to extension methods that you need to opt-in for in your Startup.cs. |
Thanks for the reply but, I don't use composer, I 'm adding
I am using Umbraco version 10.8.0-rc |
An update about the index problem, |
@pariashiri if you are using extension methods to configure LuceneDirectoryIndexOptions and not a composer that is great! But the important part is to put that before Alternatively, as you say, the safest bet is to use AzureSearchIndexOptions, but that just means you need to specify both LuceneDirectoryIndexOptions for working locally and AzureSearchIndexOptions for when working against azure search. |
@Shazwazza Hi, after a while, my index was working great suddenly after an index rebuild, again some fields became collections, and my search is broken in live sites, last time I fixed it by changing LuceneDirectoryIndexOptions to AzureSearchIndexOptions but I have no clue how can I fix it now. we must fix this issue asap, do you have any suggestions? |
I have changed my code to this but it is still same:
In start up
|
@pariashiri As mentioned above, if you are using LuceneDirectoryIndexOptions then they should be configured before the .AddComposers() call. But you are using AzureSearchIndexOptions, you should put this after the .AddComposers() call. Also, you have not mentioned what ExamineX version you are using? I've used your exact code snippets and I cannot replicate this. Restarting sites and rebuilding the indexes work 100% of the time with the correct field settings. |
That said, putting your code snippet above or after AddComposers won't matter since you are using PostConfigureOptions, I've tested and verified both ways with your code snippets. |
@Shazwazza thanks for the response, my version is 4.1.8 and my Umbraco version is 10.8.2, the new code is working locally but not on live. My main point is, that before these changes it was working ok for almost 6 months, I hadn't changed anything related to the index but after an external index rebuild it started making collections. So what do you think you could advise, the problem is the live environment, any order works locally. |
next attempt I am going to update to version 4.1.9 remove PostConfigure and move my extension before .AddComposer() |
@pariashiri Definitely recommend using the latest version that you can to ensure that you have any bug fixes/enhancements made. That said, if you re-read above (sorry for any confusion), I would recommend that:
You should only use LuceneDirectoryIndexOptions if you need to run Examine locally with Lucene. If you do need to do this, then don't use a composer and put these before .AddComposer calls |
@Shazwazza I don't have good news, still they are a collection and, as one of them is used for sorting, my sort is broken and I can't patch it, as collection can't be used for sorting. I post all my code:
AddCustomExamineIndexes class:
If you need to see ExaminexExternalIndexFieldFilter class let me know but they are filtering down what fields need to be indexed. Umbraco version 10.8.2 If you run it will work locally but not in live env. It's really frustrating, if there is a way to change an index after indexing or any fix I can do, please let me know. |
@pariashiri Unfortunately I cannot replicate. If it works locally but not in prod, I would urge you to try to determine what the difference is between environments and that the code you are using locally is in fact the code being used in prod. You will also need to ensure that you rebuild your existing indexes in prod when you publish the changes. Also, there is some confusion over how you need to configure the fields that are sortable. These are the field types you can use: https://examinex.online/customization#default-field-types--analyzers You are using
... and will be created as a single string field and is not sortable. If you want sorting, use
... and will be created as a single string field with the sortable property set to true. You are creating the "SortDate" as |
There have been more fixes for this in v5/6, I will backport some of those fixes to v2+ |
Packages & versions
Umbraco 10.6.1
ExamineX.AzureSearch - 4.1.4
ExamineX.AzureSearch.Umbraco - 4.1.4
Hi,
On one of our Umbraco installations we've got an issue where we're setting field definitions to either be a long or full text type. This works fine locally but as soon as we deploy to an azure app service everything is indexed as a
Collection(Edm.String)
.Do you have any thoughts on what might be causing this or what I can do to debug the problem?
Thanks,
John
The text was updated successfully, but these errors were encountered: