VIVO Release 1 v1.2 Upgrade Guide

January 28, 2011 - Upgrading from Release 1 v1.1 to Release 1 v1.2

This document provides a short description of the steps involved in upgrading your installation of VIVO from Release 1, Version 1.1 to Version 1.2. This and other documentation can be found on the support page at VIVOweb.org

If you need to do a fresh install, please consult VIVO Release 1 v1.2 Installation Guide or the install.html file located in the doc directoy of the VIVO source code distribution. The installation document also has a list of the required software and versions.

Release anouncement for V1.2

The VIVO 1.2 release incorporates major changes to the entire application - theming and navigation changes that will be immediately evident to any user, and underlying changes to the system architecture that are less visible but address important questions of scalability and extensibility.

Theming and Navigation

A new installation of VIVO 1.2 will look strikingly different - the User Interface team has designed a new visual theme that incorporates a new navigation and browse structure as well as a much more modular approach to page design. This theme is not only cosmetically different but leverages entirely new page templates developed with the Freemarker system, an open-source library for Java development that enables much cleaner separation of application logic from the actual page design. These changes extend the available configuration options controlling VIVO's appearance and navigation options while also simplifying the process of local customization and branding.

For existing installations of VIVO, the upgrade will not immediately transition to the new theme, navigation, or page templates. The current default theme and "tabs" (top-level and secondary navigation controls) will be left intact on upgrade and will still function as they do in version 1.1.1, with the caveat that local modifications to the default theme may conflict with internal application changes. We highly recommend that current VIVO installations use the time between release 1.2 and the upcoming release of version 1.3 (targeted for June or July 2011) to migrate local theme branding and navigation to the new VIVO template. Many legacy features such as the "tab" infrastructure have been deprecated with version 1.2 and will no longer be supported as of version 1.3.

Browsing

In addition to changes in the top-level navigation, VIVO 1.2 introduces a number of new browsing controls that will be made more configurable and extensible in version 1.3 but which already offer extensive functionality.

A fresh installation of VIVO 1.2 will feature the new theme and additional browsing options on other top-level navigation pages (Home, People, Research, Organizations, and Events). Primary among the new browsing options will be browsing by type, organized hierarchically with the same upper-level class groups currently visible in search results - people, courses, activities, topics, events, organizations, and publications. Class groups combine the similar types such as people or organizations into groups for browsing and searching, and are locally configurable using the VIVO ontology editor.

Once a group has been selected, browsing can continue to the very specific, at the level of individual people, organizations, events, or publications via A ... Z listing featuring thumbnail pictures where available. Sites will be able to configure which groups and which types within a group are exposed in search results and for browsing.

Data Storage

Before this release, VIVO has used the Jena (http://jena.sourceforge.net) relational database (RDB) subsystem for the storage of RDF data. The performance of this persistence layer has never been fast enough for an interactivity at any significant scale, so VIVO has also maintained a complete copy of data in memory. While server memory capacity has increased significantly in recent years, this requirement has put limits on the ultimate scalability of VIVO instances and also increased the cost of servers required to support VIVO.

With version 1.2 VIVO uses the SPARQL database (SDB) subsystem of Jena, specifically designed to support scalable storage and query of RDF datasets while still using standard relational database technology. This transition will significantly reduce the initial memory footprint of a VIVO application, and while the application will still require adequate processor and memory resources to generate pages from so many individual RDF statements, the scalability of VIVO installations is greatly improved.

The transition to retrieving all data via SPARQL queries also enables additional features important for tracking data provenance and access to data outside the immediate local VIVO instance. These features will be more fully explored and developed for version 1.3.

Upgrade process for V1.2

  1. Before Performing the Upgrade
  2. The Upgrade Process
  3. Ontology Upgrade
    1. Verify Ontology upgrade process
    2. Ontology knowledge base manual review
  4. File Storage System Upgrade
    1. Changes to the File Storage System
    2. Verify File Storage System upgrade process
  5. Theme Modifications

I. Before Performing the Upgrade

Please ensure that backups are created of the:

The upgrade process is similar to the original install process with the following EXCEPTIONS:

II. The Upgrade Process

1. Download the new distribution file and unpack it into a new source directory.

2. Create deploy.properties, using the same values as in your previous installation and set values for the new variables. The following table shows the default properties for deploy.properties with new V1.2 properties in blue.

Property Name Example Value
Default namespace: VIVO installations make their RDF resources available for harvest using linked data. Requests for RDF resource URIs redirect to HTML or RDF representations as specified by the client. To make this possible, VIVO's default namespace must have certain structure and begin with the public web address of the VIVO installation. For example, if the web address of a VIVO installation is "http://vivo.example.edu/" the default namespace must be set to "http://vivo.example.edu/individual/" in order to support linked data. Similarly, if VIVO is installed at "http://www.example.edu/vivo" the default namespace must be set to "http://www.example.edu/vivo/individual/"

* The namespace must end with "individual/" (including the trailing slash).
Vitro.defaultNamespace http://vivo.mydomain.edu/individual/
Directory where Vitro code is located. In most deployments, this is set to ./vitro-core (It is not uncommon for this setting to point elsewhere in development environments).
vitro.core.dir ./vitro-core
Directory where tomcat is installed.
tomcat.home /usr/local/tomcat
Name of your VIVO application.
webapp.name vivo
Directory where uploaded files will be stored. Be sure this directory exists and is writable by the user that the Tomcat service is running as.
upload.directory /usr/local/vivo/data/uploads
Directory where the Lucene search index will be built. Be sure this directory exists and is writable by the user that the Tomcat service is running as.
LuceneSetup.indexDir /usr/local/vivo/data/luceneIndex
Specify an SMTP host that the form will use for sending e-mail (Optional). If this is left blank, the contact form will be hidden and disabled.
Vitro.smtpHost smtp.servername.edu
Specify the JDBC URL of your database. Change the end of theURL to reflect your database name (if it is not "vivo").
VitroConnection.DataSource.url jdbc:mysql://localhost/vivo
Change the username to match the authorized user you created in MySQL.
VitroConnection.DataSource.username username
Change the password to match the password you created in MySQL.
VitroConnection.DataSource.password password
Specify the Jena triple store technology to use. SDB is Jena's SPARQL database; this setting allows RDF data to scale beyond the limits of the JVM heap. Set to RDB to use the older Jena RDB store with in-memory caching.
VitroConnection.DataSource.tripleStoreType SDB
Specify the maximum number of active connections in the database connection pool to support the anticipated number of concurrent page requests. It is not necessary to adjust this value when using the RDB configuration.
VitroConnection.DataSource.pool.maxActive 40
Specify the maximum number of database connections that will be allowed to remain idle in the connection pool. Default is 25% of the maximum number of active connections.
VitroConnection.DataSource.pool.maxIdle 10
Change the dbtype setting to use a database other than MySQL. Otherwise, leave this value unchanged. Possible values are DB2, derby, HSQLDB, H2, MySQL, Oracle, PostgreSQL, and SQLServer. Refer to http://openjena.org/wiki/SDB/Databases_Supported for additional information.
VitroConnection.DataSource.dbtype MySQL
Specify a driver class name to use a database other than MySQL. Otherwise, leave this value unchanged. This JAR file for this driver must be added to the the webapp/lib directory within the vitro.core.dir specified above.
VitroConnection.DataSource.driver com.mysql.jdbc.Driver
Change the validation query used to test database connections only if necessary to use a database other than MySQL. Otherwise, leave this value unchanged.
VitroConnection.DataSource.validationQuery SELECT 1
Specify the name of your first admin user for the VIVO application. This user will have an initial temporary password of 'defaultAdmin'. You will be prompted to create a new password on first login.
initialAdminUser defaultAdmin
The URI of a property that can be used to associate an Individual with a user account. When a user logs in with a name that matches the value of this property, the user will be authorized to edit that Individual.  For example, to use the netID at Cornell University as the property:
seflEditing.idMatchingProperty = http://vivo.cornell.edu/ns/hr/0.9/hr.owl#netId
selfEditing.idMatchingProperty http://vivo.mydomain.edu/ns#networkId
The temporal graph visualization is used to compare different organizations/people within an organization on parameters like number of publications or grants. By default, the app will attempt to make its best guess at the top level organization in your instance. If you're unhappy with this selection, uncomment out the property below and set it to the URI of the organization individual you want to identify as the top level organization. It will be used as the default whenever the temporal graph visualization is rendered without being passed an explicit org. For example, to use "Ponce School of Medicine" as the top organization:
visualization.topLevelOrg = http://vivo.psm.edu/individual/n2862
visualization.topLevelOrg http://vivo-trunk.indiana.edu/individual/topLevelOrgURI

3. Apply any previous changes you have made to the new source directory.

Special notes regarding source files

4. If you had modified web.xml to configure the Pellet Reasoner (as described in the installation instructions), repeat that modification.

5. Stop Apache Tomcat and run ant by typing: ant all

6. Start Apache Tomcat and log in to VIVO.

III. Ontology Changes

i. Verify Ontology upgrade process

After Apache Tomcat is started, these files should be reviewed to verify that the automated upgrade process was executed successfully.  The ontology alignment process will create the following files in the Tomcat webapps/vivo/WEB-INF directory:

ontologies/update/logs/knowledgeBaseUpdate.log
A log of a summary of updates that were made to the knowledge base and notes about some recommended manual reviews. This file should end with "Finished knowledge base migration". If this file contains any warnings they should be reviewed with your implementation team representative to see whether any corrective action needs to be taken.
ontologies/update/logs/knowledgeBaseUpdate.error.log
A log of errors that were encountered during the upgrade process. This file should be empty if the upgrade was successful.
ontologies/update/changedData/removedData.n3
An N3 file containing all the statements that were removed from the knowledge base.
ontologies/update/changedData/addedData.n3
An N3 file containing all the statements that were added to the knowledge base.

ii. Ontology knowledge base manual review

Changes to the VIVO core ontology may require corresponding modifications of the knowledge base instance data and local ontology extensions.

When Apache Tomcat starts up following the upgrade, it will initiate a process to examine the knowledge base and apply necessary changes. Not all of the modifications that may be required can be automated, so manual review of the knowledge base is recommended after the automated upgrade process. The automated process will make only the following types of changes:

Class or Property renaming
All references to the class (in the subject or object position) will be updated to the new name. References to the property will be updated to the new name.
Class or Property deletion
All individuals in a deleted class will be removed.
All statements using a deleted property will be changed to use the nearest available superproperty. If there is no available superproperty then the statement will be deleted from the knowledge base. Note that all removed and added data is recorded in the files in the changedData directory.
Property addition
If a newly added property is the inverse of a previously existing property, the inverse of any statements using the pre-existing property will be asserted.
Annotation property default values
If a site has modified the value of a vitro annotation (such as displayRankAnnot or displayLimitAnnot) so that it is no longer using the default, then that setting will be left unchanged.
If a site is using the default value of a vitro annotation, and the default has been changed in the new version of the ontology, then the new default value will be propagated to the knowledge base.

IV. File Storage System Upgrade

i. Changes to the File Storage System

Each uploaded file exists as an individual entity in VIVO. When the browser requests an upload file from VIVO, the data model is queried to find out where the file is actually stored, so it can be downloaded to the browser.

In VIVO release 1.2 this storage location, known as the "Alias URL" for the uploaded file, is stored in the file entity. That way, pages that contain many files can be displayed much more quickly.

When Apache Tomcat starts up after the upgrade, it will initiate a process to calculate the "Alias URL" for each existing file and store it in the data model for fast access.

ii. Verify File Storage System upgrade process

The File Storage upgrade process will create a log file in the VIVO upload directory. You should review this file to ensure that this upgrade worked properly.

upgrade/FileStorageAliasAdder-log.2011-00-00T00-00-00.txt
A log of the upgrade process. The actual filename includes a timestamp that tells when the upgrade executed. This file should end with Finished adding alias URLs to FileByteStreams. If this file contains any warnings they should be reviewed with your implementation team representative to see whether any corrective action needs to be taken.

V. Theme Changes

Need Nick to help with this section

VIVO 1.2 comes with a new theme called "wilma" that uses the FreeMarker template engine for generating web pages. The theme is located in /vivo/themes/wilma and the FreeMarker files have an ftl (for FreeMarker Template Language) extension.

Follow step A or B below, whichever is applicable to your site:

A. If you did not create a customized theme for your site in V1.0 or V1.1, but used the vivo-basic theme in its original directory, you need not take any action in order to convert your site to the VIVO 1.1 theme.

B. If you created your own theme directory in VIVO 1.1, follow the steps below under sections "Templates," "Stylesheets," and "Site Icons" to upgrade your theme to VIVO 1.2.

1. Templates
a. Copy the directory /vivo/themes/vivo-basic/templates into your theme directory /vivo/themes/[your-theme-name].
b. Follow step i or ii below, whichever is applicable to your theme.
  1. If you did not apply any customizations to the JSPs in your VIVO 1.0 theme, then you do not need to apply any additional changes to the VIVO 1.1 theme templates during the upgrade process.

  2. If you did apply customizations to the JSPs in your VIVO 1.0 theme,you will need to hand-replicate those modifications in the new theme template files.

    The theme template content that was previously contained in three JSP files is now contained in five FTL files. The correspondence between the 1.0 JSPs and the 1.1 FTLs is as follows: 

                identity.jsp => identity.ftl
            menu.jsp => menu.ftl and search.ftl
            footer.jsp => footer.ftl and googleAnalytics.ftl

    googleAnalytics.ftl is the file to which you add your site's Google Analytics Tracking Code (see section II).

    Because the FreeMarker Template Language uses many syntactic conventions that will be familiar to template authors from JSP or other common templating systems, the translation of your JSP changes into the new FTLs should be relatively straightforward.

    Consult the FreeMarker Template Author's Guide at http://freemarker.org/docs/dgui.html and the Reference at http://freemarker.org/docs/ref.html for complete documentation of the syntax and available built-in constructs. Template authors need not be concerned with the Programmer's Guide or Java API documentation.

c. Remove the jsp directory from your themes directory.
2. Stylesheets
VIVO 1.1 includes changes to vivo-basic stylesheets. If you modified styles in your VIVO 1.0 theme, you will not be able to simply copy the 1.0 stylesheets into your 1.1 theme, because you will then lose 1.1 style upgrades that your theme should pick up. Instead, you should use the vivo-basic 1.1 stylesheets as a starting point, and manually merge your 1.0 style modifications in as needed.
3. Site Icons
Copy the site icons from your 1.1 theme into the site_icons folder in your 1.2 theme.