Commit graph

5045 commits

Author SHA1 Message Date
Jim Blake
9f9cea06e3 VIVO-869 More formal coordination between startup(), pause(), and unpause(). 2015-02-07 13:13:48 -05:00
Jim Blake
b213718a72 VIVO-869 A full re-inference invokes a full re-index.
IndexingChangeListener is also modified to listen for the beginning of a re-index, and discard any changes it has accumulated.
2015-02-06 15:51:29 -05:00
Jim Blake
d25176298d VIVO-869 Drastically simplify the search configuration.
Create LabelsAcrossContextNodes to replace AdditionalURIsForContextNodes. It's more configurable.
Create SelectQueryUriFinder to replace AdditionalUrisForVCards. It's more configurable.
Rename SimpleSparqlQueryDocumentModifier to SelectQueryDocumentModifier, to conform.
Radically simplify the VIVO search configuration, omitting all special cases except those for VCards.
2015-02-06 15:22:42 -05:00
hudajkhan
8e97ff3678 VIVO-957 error, updating subclass sorting method for object, using superclasses of classes instead of just comparing whether classes are subclasses of each other 2015-02-03 15:25:18 -05:00
Jim Blake
a829de7923 VIVO-950, a null range is like a range of Owl:Thing 2015-02-02 16:05:51 -05:00
Jim Blake
474b984bed VIVO-950 When looking up FauxProperties, use an unfiltered WebappDaoFactory. 2015-02-02 11:44:11 -05:00
Tim Worrall
49de7204d6 no Jira issue. Search index template file was missing a closing div tag. 2015-01-30 12:06:26 -05:00
Tim Worrall
3b85239642 Related to VIVO-954. Added handling of null vClass for odd objects like \'count per year\' 2015-01-30 11:33:17 -05:00
hudajkhan
9297c90946 reasoner error occurring with n3 files that contain [ a owl:Axiom, etc.] statements, read in as empty axiom statements 2015-01-29 12:54:39 -05:00
Tim Worrall
2433f8b7a1 no jira issue: subclass headings were not getting removed correctly when all pubs were hidden and user has no editing privileges. 2015-01-28 13:17:18 -05:00
Jim Blake
7f61b39311 VIVO-869 Don't enter duplicate class group URIs in the search document. 2015-01-28 10:56:37 -05:00
Jim Blake
b3564e31bd VIVO-869 Cleanup: Committer, shorter event type, ALLTEXTUNSTEMMED, et. al.
Get rid of the SearchIndexerSetup.Committer listener class. This functionality becomes internal to UpdateUrisTask.
Use shorter names for Event.Type values, so the display will be more compact.
ALLTEXTUNSTEMMED is not automatically populated from ALLTEXT, so change the default target field on
    SimpleSparqlQueryDocumentModifier to include both of them.
Trivial cleanup in UpdateDocumentWorkUnit.
Create StackTraceLayout, a simple Log4J layout that will print the stack trace for each call.
    Used for diagnosing the source of uninformative log messages.
2015-01-26 10:18:02 -05:00
Jim Blake
de5b80bf75 VIVO-869 ABoxRecomputer should pause the SearchIndexer during a rebuild. 2015-01-22 15:49:08 -05:00
Tim Worrall
ccb2063aa4 Merge branch 'develop' of https://github.com/vivo-project/Vitro into develop 2015-01-22 15:02:35 -05:00
Tim Worrall
d932703360 added a check to ensure that debug is enabled before a logging statement is processed. Also implemented the ToString.ontModelToString() method. 2015-01-22 15:02:15 -05:00
Jim Blake
db6893666d VIVO-870 Move VivoAgentContextNodeFields into the configuration.
Delete VivoAgentContextNodeFields and its tests and the data files for the tests.
This is the last subclass of ContextNodeFields, so delete that, too.
2015-01-22 13:10:45 -05:00
Jim Blake
989ae261ca VIVO-870 Add thread pool size to the search indexer setup. 2015-01-21 12:41:20 -05:00
Jim Blake
2fbf467ab4 Merge branch 'feature/searchIndexerTry' into develop 2015-01-21 12:32:48 -05:00
Ted Lawless
5488b30d99 Fixes ETag quoting and strips white space from generated Etag. 2015-01-21 12:31:33 -05:00
Jim Blake
d382e0efd6 VIVO-869 Don't submit updates while indexer is paused.
When the pause flag is set, the IndexingChangeListener will continue to accumulate changes even if there
are gaps in the stream. The listener will not submit a task until the indexer is unpaused.
2015-01-21 12:30:44 -05:00
Jim Blake
7d16e10357 VIVO-945 Produce more concise log messages when a failure is caused by shutdown. 2015-01-20 18:08:52 -05:00
Jim Blake
d5a497774e VIVO-869 Don't operate on null URIs.
If a UriFinder finds a null URI, issue a warning message. If a null URI is submitted to UpdateUrisTask, ignore it.
2015-01-20 17:22:43 -05:00
Jim Blake
d37c41cf65 VIVO-869 Improve throttling for indexing tasks.
Start the implementation in "paused" state, so tasks submitted before startup() are queued.
Increase the time interval on IndexingChangeListener, so we get larger batches.
Change the RejectedExecutionHandler on the pool to CallerRunsPolicy,
   so if there is no available thread for a work unit then the thread of the task itself will run it.
2015-01-20 17:21:09 -05:00
Jim Blake
fcfd2e7be6 VIVO-869 The display should not barf if no IndexHistory is present. 2015-01-20 16:09:53 -05:00
Jim Blake
83a5523ace VIVO-869 Improve the display
Show exclusions count as separate from deletions count.
When there is an error on the Ajax call, display the error status and stop asking for refreshes.
When a rebuild is requested, redirect to just status, so refreshing the page would not request another rebuild.
2015-01-20 14:06:11 -05:00
Jim Blake
78491234db VIVO-871 Improve performance when reading object properties.
Also, improve display.
2015-01-20 10:25:55 -05:00
Jim Blake
56a640fe44 VIVO-870 Make the number of threads configurable.
Also, clean up the shutdown process and some comments.
2015-01-18 18:45:14 -05:00
Jim Blake
8c3e06fc78 VIVO-870 Move the DocumentModifiers into the configuration. 2015-01-16 16:52:52 -05:00
Jim Blake
4fac4b7576 VIVO-870 Start moving DocumentModifiers into the configuration.
Create the SimpleSparqlQueryDocumentModifier.
Use it to replace VivoISFGrantFields and VivoInformationResourceContextNodeFields.
2015-01-16 11:46:58 -05:00
Jim Blake
7b7ca268c8 VIVO-871 Improve performance when getting VClasses(). 2015-01-16 11:39:56 -05:00
Jim Blake
8d37fc0d18 VIVO-870 Issue a commit() on each PROGRESS event. 2015-01-16 11:38:57 -05:00
Jim Blake
27d3141bb6 VIVO-871 Break out more tasks to improve the timings
UpdateDocumentWorkUnit contains a list of required DocumentModifiers.
They are required but because they are in the list of DocumentModifiers, they are timed like the optional ones.

Similarly, UpdateUrisTask contains a required SearchIndexExcluder.
2015-01-16 11:34:31 -05:00
Jim Blake
0c477945a8 VIVO-870 Clean up exception handling on shutdown. 2015-01-14 15:59:59 -05:00
Jim Blake
982e81aece VIVO-871 Produce timing breakdowns from a developer option 2015-01-13 16:03:16 -05:00
Jim Blake
3fd72a7110 VIVO-870 Confirm that DocumentModifiers are thread-safe. 2015-01-12 18:07:41 -05:00
Jim Blake
dee53e3aac VIVO-870 Implement UpdateStatementsTask.
Also add the listener to support it and get rid of the last of the vestigial classes.
2015-01-12 15:15:16 -05:00
Jim Blake
0602406c53 VIVO-870 Create IndividualDao.getAllIndividualUris()
This replaces IndividualDao.getAllOfThisTypeIterator(). It returns the list that was created instead of
the iterator.

This would require a change to ObjectSourceIFace, but we get rid of it instead. Nobody was using it.
2015-01-11 18:30:28 -05:00
Jim Blake
99c03ce49a Merge branch 'develop' into feature/searchIndexerTry 2015-01-09 16:41:36 -05:00
Jim Blake
39f1ed0b27 VIVO-870 Implement UpdateUrisTask.
At the same time, broke the old IndexBuilder, so no listener and no rebuild.
2015-01-09 16:38:08 -05:00
brianjlowe
5d6780620b VIVO-890 and VIVO-892 ABoxRecomputer performance improvements 2015-01-08 16:40:45 +02:00
Jim Blake
64624f2b84 VIVO-870 Make IndexingUriFinders configurable.
Use no-argument constructors, supplying data through the ContextModelsUser interface as needed.

Add toString() methods to IndexingUriFinders, DocumentModifiers and SearchIndexExcluders
2015-01-07 17:25:12 -05:00
Jim Blake
2ceab6e3df VIVO-873 Adapt the client code to the new interface.
Create a bridge implementation of SearchIndexerImpl that just wraps around an old IndexBuilder.

Modify client code:
Application, BasicAuthenticator, SearchServiceController, SparqlUpdateApiController,
UpdateUrisInIndex and VClassGroupCache

Rewrite IndexController to use AJAX and to show the current status and history of the indexer events.
2015-01-07 16:18:41 -05:00
Jim Blake
3bc42c1456 Merge branch 'develop' into feature/searchIndexerTry 2015-01-06 16:26:00 -05:00
Jim Blake
8a0d8a2639 VIVO-873 create SearchEngine.documentCount()
Also, remove the vestigial IndexConstants class and two unused methods in PagedSearchController.
2015-01-06 16:04:19 -05:00
Jim Blake
6513d34d86 VIVO-909 Improve documentation of the TripleStoreQuirks 2015-01-05 18:54:36 -05:00
Jim Blake
7639bf43e3 VIVO-907 make the the check for isomorphism more efficient.
Read the file graph into memory and check against that.
2015-01-05 12:58:03 -05:00
Jim Blake
2d44f5bec3 VIVO-873 restructure packages for SearchIndexer
documentBuilding, exclusions, and indexing, under  searchindex instead of search.
2015-01-04 17:19:13 -05:00
Jim Blake
ac39b0799e VIVO-935 Clean up code for search exclusion.
Take out the attribute key in SearchIndexerSetup. Nobody ever sets that attribute.
Take out the VClassGroupCache.removeClassesHiddenFromSearch() method. If nobody sets that attribute, this method does nothing.
Take out VClassGroupDao.removeClassesHiddenFromSearch() and VClassGroupDaoJena.removeClassesHiddenFromSearch(). They are never called.
Take out ClassProhibitedFromSearch interface. It is implemented but never referenced.
Take out FileBasedProhibitedFromSearch. It is never used.
Move ProhibitedFromSearch and ProhibitedFromSearchTest from edu.cornell.mannlib.vitro.webapp.search.beans to edu.cornell.mannlib.vitro.webapp.dao.vclassgroup
2015-01-02 14:01:06 -05:00
Jim Blake
f4f51a096f Merge branch 'feature/pelletRemoval' into develop 2014-12-31 15:13:37 -05:00
Jim Blake
b8ad8b64d0 VIVO-778 remove the pellet libraries. 2014-12-31 15:09:39 -05:00