-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
indexing on a new environment doesn't run automatically for the first time #385
Comments
Original Redmine Comment To reproduce on a fresh VM/machine.
See if the Solr index is correct (and reflected in browsing GUI) without having to run "index all". |
Original Redmine Comment I am able to replicate this. The issue is that indexing of the new "groups" dvtype introduced for #3464 does not happen automatically. That is to say, on a fresh install, all the dvtype:dataverses are indexed, but not the dvtype:groups, which are used for a JOIN on any search. Without these groups, there is nothing to JOIN on so no results can be found. It should be relatively straighforward to index the User Private Groups (UPG) on user creation. For instance, group_user1 could be indexed when pete is created:
But what about non-User Private Groups groups such as group_public? These groups are all fake and hard coded for now until groups are implemented in #3500. For now, perhaps, we could say that when the first user is created (i.e. pete), go ahead and run an "index all" to create all the temporary fake groups such as group_public:
|
Original Redmine Comment Indexing is still not running automatically for the first time . I have a new db, new build, deleted all in solr /data/index dir. 392523 [qtp112712824-17] INFO org.apache.solr.core.SolrCore ? [collection1] webapp=/solr path=/spell params={facet=true&sort=release_or_create_date_dt+desc&hl.fl=description&wt=javabin&hl=true&version=2&rows=10&hl.snippets=1&facet.query=&start=0&q=&facet.field=hostdataverse_s&facet.field=affiliation_ss&facet.field=releasedate&facet.field=keyword_ss&facet.field=authorName_ss&facet.field=filetype_s&facet.field=dvtype&qt=/spell&fq=dvtype:(dataverses+OR+datasets)&fq=({!join+from%3Dgroups_s+to%3Dperms_ss}id:group_public+OR+{!join+from%3Dgroups_s+to%3Dperms_ss}id:group_user1)} hits=0 status=0 QTime=5 Here are the notifications: However login as pete http://dataverse.cloudapp.net/ I see no datavereses other than root. Also I see /usr/local/solr-4.6.0/example/solr/collection1/data/index dir is being repopulated. I haven't run index all yet. |
Original Redmine Comment Elda Sotiri wrote:
I'm concerned about what this means... I have never deleted files on disk from Solr... If I want to clear out the index, I do over HTTP with a curl command like this: curl http://localhost:8983/solr/update/json?commit=true -H 'Content-type: application/json' -X POST -d '{"delete": { "query":":" }}' You could also stop Solr and delete the entire Solr directory to clear it out. Then unpack the tarball to start with a fresh installation.
A shell account on this server would help me troubleshoot. Can you please provide me the output of this command: curl 'http://localhost:8983/solr/collection1/select?rows=100&wt=json&indent=true&q=*' Thanks. Passing this ticket back to you, Elda. |
Original Redmine Comment sorry, just saw this..
|
Original Redmine Comment phil, created an admin acct for you.. send info already. |
Original Redmine Comment Elda Sotiri wrote:
Thanks, I logged in so I could look at server.log. The problem is that "User Private Groups" such as "group_user1" for pete or "group_user2" for uma do not exist. There seem to have been two setup attempts on 2014-04-08, one at ~12:30 and one at ~18:30. Notice how during the more recent setup attempt the User Private Groups could not be created because Solr was down ("Server refused connection at: http://localhost:8983/solr"). As long as Solr is running during setup, everything should be fine. Please note that there is a related ticket at #3643 regarding "manage index failures for later correction" but it hasn't been worked on yet. For now, as long as Solr is up, it should work. Another way you can test a new environment is with Here are the logs:
|
Original Redmine Comment Phil updated the guide that solr should be running when setting up or rebuilding the environment, and there is no need to run indexall. Thanks Phil! |
Original Redmine Comment Elda and I updated the Dev guide to say "Solr must be running with this custom schema in place during setup" in the very first paragraph. We also talked about how editing objects such as users or dataverses that have never been indexed (due to Solr being down during setup, for example) will allow them to be indexed for the first time. This is because the indexing code is tied to "save" methods for users, dataverses, etc. I also removed the "index all" step from the "Rebuilding your dev environment" section since it's no longer necessary but added an earlier step of clearing out your Solr index (with a curl command) as you start to rebuild (i.e. right after you drop your database). This commit to the Dev guide: 738a63c Solr must be up during setup, how to clear Solr data #3800 |
Author Name: Elda Sotiri (@esotiri)
Original Redmine Issue: 3800, https://redmine.hmdc.harvard.edu/issues/3800
Original Date: 2014-04-02
Original Assignee: Elda Sotiri
Gustavo and I saw this issue on http://dataverse.cloudapp.net/, had to run index manually for the first time. After that indexing works.
Related issue(s): #57, #91, #229
Redmine related issue(s): 3464, 3500, 3643
The text was updated successfully, but these errors were encountered: