VIVO Release 1 v1.3 Upgrade Guide

July 22, 2011 - Upgrading from Release 1 v1.2 to Release 1 v1.3

This document provides a short description of the steps involved in upgrading your installation of VIVO from Version 1.2+ to Version 1.3. This and other documentation can be found on the support page at VIVOweb.org

If you need to do a fresh install, please consult the VIVO Release 1 v1.3 Installation Guide found on vivoweb.org or the install.html file located in the doc directory of the VIVO source code distribution. The installation document also has a list of the required software and versions.

Release Announcement for V1.3

https://confluence.cornell.edu/x/3B4DCQ - get content from the wiki page before final release.

Upgrade process for V1.3

  1. Before Performing the Upgrade
  2. Triple Store (BL)
  3. The Upgrade Process
  4. Ontology Changes (SM)
    1. Verify Ontology upgrade process
    2. Ontology knowledge base manual review
  5. Template Changes (RY)
  6. List View Changes (RY)
  7. Authorization Changes (JB)
  8. Set Up SDB Store in the Background (Optional)

I. Before Performing the Upgrade

Please ensure that backups are created of the:

The upgrade process is similar to the original install process with the following EXCEPTIONS:

II.  Triple Store

VIVO 1.3 now requires you to use Jena's SPARQL database (SDB) for the triple store technology.  Jena's legacy relational database store (RDB) was used by VIVO 1.1.1 and earlier.  Both SDB and RDB were available in VIVO 1.2 and 1.2.1.  A VIVO 1.2 system running in RDB mode ... [WHAT HAPPENS IF THEY UPGRADE in RDB MODE? - elly]

SDB mode caches only a fraction of the RDF data in memory. Most queries are issued directly against the underlying database. This allows VIVO installations to display data from large RDF models while requiring only a small amount of server memory to run the application. There is a tradeoff in response time: pages make take slightly longer to load in SDB mode, and performance will depend on the configuration parameters of the database server. Additionally, advanced OWL reasoning (not enabled by default in either mode) is not possible in SDB mode. With SDB, only the default set of inferences (inferred rdf:type statements) are generated, though they are generated as soon as data is edited rather than in a background process.

This copying process can take a number of hours to complete if the installation contains a large amount of RDF data (roughly a million triples or more). See section Set Up SDB Store in the Background (Optional) for instructions on how to run this lengthy conversion process in the background while an RDB system is operating. Doing this will reduce the time necessary to start VIVO the first time it is run in SDB mode.

III. The Upgrade Process

1. Download the new distribution file and unpack it into a new source directory.

2. Create a new deploy.properties using the same values as in your previous installation and set values for the new variables as described below (vitro.local.solr.url, vitro.local.solr.ipaddress.mask, vitro.home.directory, email.smptHost, email.replyTo, rootUser.emailAddress)

Property Name Example Value
Default namespace: VIVO installations make their RDF resources available for harvest using linked data. Requests for RDF resource URIs redirect to HTML or RDF representations as specified by the client. To make this possible, VIVO's default namespace must have a certain structure and begin with the public web address of the VIVO installation. For example, if the web address of a VIVO installation is "http://vivo.example.edu/" the default namespace must be set to "http://vivo.example.edu/individual/" in order to support linked data. Similarly, if VIVO is installed at "http://www.example.edu/vivo" the default namespace must be set to "http://www.example.edu/vivo/individual/"
* The namespace must end with "individual/" (including the trailing slash).
Vitro.defaultNamespace http://vivo.mydomain.edu/individual/
Directory where Vitro code is located. In most deployments, this is set to ./vitro-core (It is not uncommon for this setting to point elsewhere in development environments).
vitro.core.dir ./vitro-core
Directory where tomcat is installed.
tomcat.home /usr/local/tomcat
Name of your VIVO application.
webapp.name vivo
URL of Solr context used in local VIVO search. Should consist of: 
 scheme + servername + port + vivocontextpath + "solr"
In the standard installation, the Solr context will be on the same server as VIVO, and in the same Tomcat instance. The path will be the VIVO webapp.name (specified above) + "solr"
vitro.local.solr.url http://localhost:8080/vivosolr
Restricts access to the Solr search platform. One or more regular expressions, separated by commas. When a request is made to Solr, the IP address of the requestor must match one of the patterns, or the request will be rejected.
Examples:
  • vitro.local.solr.ipaddress.mask = 127\.0\.0\.1
  • vitro.local.solr.ipaddress.mask = 127\.0\.0\.1,0:0:0:0:0:0:0:1
  • vitro.local.solr.ipaddress.mask = 169.254.*
vitro.local.solr.ipaddress.mask 127\.0\.0\.1,0:0:0:0:0:0:0:1
Directory where the VIVO application will store the data that it creates. This includes uploaded files (usually images) and the Solr search index. Be sure this directory exists and is writable by the user who the Tomcat service is running as.
vitro.home.directory /usr/local/vivo/data
Specify an SMTP host that the application will use for sending e-mail (Optional). If this is left blank, the contact form will be hidden and disabled, and users will not be notified of changes to their accounts.
email.smtpHost smtp.servername.edu
Specify an email address which will appear as the sender in e-mail notifications to users (Optional). If a user replies to the notification, this address will receive the reply. If a user's e-mail address is invalid, this address will receive the error notice. If this is left blank, users will not be notified of changes to their accounts.
email.replyTo vivoAdmin@my.domain.edu
Specify the JDBC URL of your database. Change the end of the URL to reflect your database name (if it is not "vivo").
VitroConnection.DataSource.url jdbc:mysql://localhost/vivo
Change the username to match the authorized user you created in MySQL.
VitroConnection.DataSource.username username
Change the password to match the password you created in MySQL.
VitroConnection.DataSource.password password
Specify the Jena triple store technology to use. SDB is Jena's SPARQL database; this setting allows RDF data to scale beyond the limits of the JVM heap. Set to RDB to use the older Jena RDB store with in-memory caching.
VitroConnection.DataSource.tripleStoreType SDB
Specify the maximum number of active connections in the database connection pool to support the anticipated number of concurrent page requests. It is not necessary to adjust this value when using the RDB configuration.
VitroConnection.DataSource.pool.maxActive 40
Specify the maximum number of database connections that will be allowed to remain idle in the connection pool. Default is 25% of the maximum number of active connections.
VitroConnection.DataSource.pool.maxIdle 10
Change the dbtype setting to use a database other than MySQL. Otherwise, leave this value unchanged. Possible values are DB2, derby, HSQLDB, H2, MySQL, Oracle, PostgreSQL, and SQLServer. Refer to http://openjena.org/wiki/SDB/Databases_Supported for additional information.
VitroConnection.DataSource.dbtype MySQL
Specify a driver class name to use a database other than MySQL. Otherwise, leave this value unchanged. This JAR file for this driver must be added to the the webapp/lib directory within the vitro.core.dir specified above.
VitroConnection.DataSource.driver com.mysql.jdbc.Driver
Change the validation query used to test database connections only if necessary to use a database other than MySQL. Otherwise, leave this value unchanged.
VitroConnection.DataSource.validationQuery SELECT 1
Specify the email address of the root user account for the VIVO application. This user will have an initial temporary password of 'rootPassword'. You will be prompted to create a new password on first login.
rootUser.emailAddress vivoAdmin@my.domain.edu
The URI of a property that can be used to associate an Individual with a user account. When a user logs in with a name that matches the value of this property, the user will be authorized to edit that Individual.
selfEditing.idMatchingProperty http://vivo.mydomain.edu/ns#networkId
The temporal graph visualization can require extensive machine resources. This can have a particularly noticable impact on memory usage if
  • VIVO is configured to use Jena SDB,
  • The organization tree is deep,
  • The number of grants and publications is large.
The VIVO developers are working to make this visualization more efficient. In the meantime, VIVO release 1.2 guards against this impact by disabling the temporal graph visualization unless the "visualization.temporal" flag is set to "enabled". To enable it, uncomment the line for this setting.
visualization.temporal enabled
The temporal graph visualization is used to compare different organizations/people within an organization on parameters like number of publications or grants. By default, the app will attempt to make its best guess at the top level organization in your instance. If you're unhappy with this selection, uncomment out the property below and set it to the URI of the organization individual you want to identify as the top level organization. It will be used as the default whenever the temporal graph visualization is rendered without being passed an explicit org. For example, to use "Ponce School of Medicine" as the top organization:
visualization.topLevelOrg = http://vivo.psm.edu/individual/n2862
visualization.topLevelOrg http://vivo-trunk.indiana.edu/individual/topLevelOrgURI

3. Apply any previous changes you have made to the new source directory.

Special notes regarding source files

4. If you had modified web.xml to configure the Pellet Reasoner (as described in the installation instructions), repeat that modification.

5. Stop Apache Tomcat and run ant by typing: ant all

6. Start Apache Tomcat and log in to VIVO.

IV. Ontology Changes

i. Verify Ontology upgrade process

After Apache Tomcat is started, these files should be reviewed to verify that the automated upgrade process was executed successfully.  The ontology alignment process will create the following files in the Tomcat webapps/vivo/WEB-INF directory:

ontologies/update/logs/knowledgeBaseUpdate.log
A log of a summary of updates that were made to the knowledge base and notes about some recommended manual reviews. This file should end with "Finished knowledge base migration". If this file contains any warnings they should be reviewed with your implementation team representative to see whether any corrective action needs to be taken.
ontologies/update/logs/knowledgeBaseUpdate.error.log
A log of errors that were encountered during the upgrade process. This file should be empty if the upgrade was successful.
ontologies/update/changedData/removedData.n3
An N3 file containing all the statements that were removed from the knowledge base.
ontologies/update/changedData/addedData.n3
An N3 file containing all the statements that were added to the knowledge base.

ii. Ontology knowledge base manual review

Changes to the VIVO core ontology may require corresponding modifications of the knowledge base instance data and local ontology extensions.

When Apache Tomcat starts up following the upgrade, it will initiate a process to examine the knowledge base and apply necessary changes. Not all of the modifications that may be required can be automated, so manual review of the knowledge base is recommended after the automated upgrade process. The automated process will make only the following types of changes:

Class or Property renaming
All references to the class (in the subject or object position) will be updated to the new name. References to the property will be updated to the new name.
Class or Property deletion
All individuals in a deleted class will be removed.
All statements using a deleted property will be changed to use the nearest available superproperty. If there is no available superproperty then the statement will be deleted from the knowledge base. Note that all removed and added data is recorded in the files in the changedData directory.
Property addition
If a newly added property is the inverse of a previously existing property, the inverse of any statements using the pre-existing property will be asserted.
Annotation property default values
If a site has modified the value of a vitro annotation (such as displayRankAnnot or displayLimitAnnot) so that it is no longer using the default, then that setting will be left unchanged.
If a site is using the default value of a vitro annotation, and the default has been changed in the new version of the ontology, then the new default value will be propagated to the knowledge base.
Please note: The vivo-basic theme has been deprecated and is no longer supported

For details on the new structure of themes in 1.2 and further information regarding the development of your own custom theme, please review the Site Administrator's Guide. This document will focus on updating an existing pre 1.2 theme.

V. Template changes [see RY for further details]

VI. List view changes [see RY for further details]

VII. Authorization changes [see JB for further details]

VIII. Set Up SDB Store in the Background (Optional)

If your VIVO installation is running in RDB mode, and you'd like to convert to SDB, you can start the conversion process in the background while the RDB system is running. This will reduce the delay in initial startup after the application is redeployed with deploy.properties set for SDB. Note that it is important not to edit any data anywhere in the application while this background conversion is running.

To start the SDB conversion, log in as a system administrator and request /sdbsetup (For example, if your VIVO is installed at http://vivo.myuniversity.edu/ you would type http://vivo.myuniversity.edu/sdbsetup into your browser).

Click the button that appears on this page.

During the course of the SDB setup, which may take several hours on a large database, subsequent requests to /sdbsetup will display a message that the operation is still in progress. When a request for this page shows a message that the SDB setup has completed successfully, shut down Tomcat, set deploy.properties to SDB mode, redeploy, and restart Tomcat. VIVO will now be running from the SDB store.