Skip to main content
OCLC Support

Normalisation

  1. Last updated
  2. Save as PDF
Discover an introduction to normalisation and how to verify normalise services are functioning in OLIB.

►System Administration> OLIB Defaults
►System Administration> Normalisation Rules
►OLIB Reports> Reports

OLIB Web automatically 'normalises' catalogue data from the major bibliographic fields. Normalisation ensures that common word endings and minor spelling differences are accounted for, e.g. rain, rains, raining. From OLIB 9 Service Pack 10 the Normalisation Service is run as part of OLIB Web - if OLIB Web is running, then the Normalisation Service is running.

View the normalisation rule set

The correct rule set is defined at the time of installation:

  1. Go to System Administration> OLIB Defaults.
  2. In General defaults the Normalisation Rule Set should be set to LATIN -1 (STANDARD -1).

For the Keyword Search, terms are 'normalised' using Oracle normalisation by default. This can be changed to use OLIB's own normalisation in OLIB Defaults>Keyword Search Defaults. However this is not advised without first consulting your OCLC consultant. It is primarily useful for indexing Asian language data to avoid whole phrases being treated as individual words.

To view content of the standard normalisation rules:

  1. Go to System Administration> Normalisation Rules.

The rules can be viewed here but changes are not normally required.

The LATIN STANDARD rule sets contain the normalisation rules. They do not normally require editing, however occasionally this may be required and a new rule set may be set up. This could be for example, if your organisation is a specialist library where specific terms require explicit recognition. In this case OCLC may recommend the addition of a new rule.

The Latin Standard rules are replaced during OLIB upgrades and consequently any changes may be removed.

 Caution: do not edit these without first checking with your OCLC consultant or OCLC Support.

Re-normalise the database

If changes are made to normalisation rules it is necessary to re-build OLIB Web, and re-normalise the database so that the changes take effect on existing data. Following any consultation about changing the rule set, the system manager can then run the re-normalisation process by setting the date and time in OLIB Defaults:

  1. Go to System Administration> OLIB Defaults: General Defaults.
  2. In modify mode, click to select a date/time in Renormalisation Time.
  3. Select a date in the calendar widget and in the Time: slot click the values to increase the time and shift-click to decrease the time value. Alternatively click and drag to the right to increase the time and to the left to decrease the time.
  4. Save the changes.

The process automatically stops and afterwards restarts the Normalise Server (System Administration> Server Process Configuration). Therefore, if you make changes to any normalisation rules and you are ready to initiate the re-normalisation process, there is no need to stop and restart the process manually.

The Renormalisation Results in General Defaults indicate how long the process has taken to complete:

07-Aug-2019 16:10 - 16:16 .1 hours: Complete

When logging is enabled, each re-normalisation process records its progress in the Trace Information domain (System Administration, generally for Support purposes). Search for fdi_norm in this domain to list the records relating to the re-normalisation process. These records are listed in chronological order, most recent ones first.

Stop the process

If you want to halt the process at any time, just remove the scheduled value from the Renormalise Time field. The Renormalisation Results field will indicate that the process was told to stop rather than completed.

Verify Normalise services are functioning

Normalisation report

It is possible to verify that the Normalise Services are functional. To do this, you access a system report:

  • SYS006 - Normalisation Test

and then click the button to Preview it.

OLIB shows this brief output:

Normalisation Test
Input : "What does this normalise to?"
Result: "what do thi normalis to"

The output shown above in the Result field indicates that Normalisation is working.

Run the Normalise Test report:

  1. Go to OLIB Reports> Reports.
  2. From Other Searches choose System Reports and search e.g. %norm.
  3. Locate the report SYS006 - Normalisation Test:

-34    SYS006 - Normalisation Test

Open the report in modify and click the Preview Report button:

Normalisation Test
Input : "What does this normalise to?"
Result: "what do thi normalis to"

The Result above shows how the Input Value field situated below the Description was normalised.

Test any Normalisation results

The above Normalisation Test report can also be utilised to ascertain the normalisation result for any other text. It is also useful as a means of ascertaining a normalised value based on the rules that are in effect.

  1. Open the SYS006 - Normalisation Test report in modify mode.
  2. Below the Description, in the Input Value enter some text you want to test, e.g. The Basics of banking is in the CATALOGUE .
  3. Click Preview Report.

This example normalises as:

Normalisation Test
Input : "The Basics of banking is in the CATALOGUE"
Result: "basic of bank is in th catalogu"

OLIB will retain your test text. Re-enter the original text if needed.

Test Normalisation services remotely

The above Normalisation Test report can be used in a GETStream URL to test the Normalise Services remotely:

http://mylibrary.ac.uk/olibweb/GETReport.jsp?reportno=-34&param1=What%20does%20this%20edited%20text%20normalise%20to?

Replace...

mylibrary.ac.uk/olibweb

with your OLIB system’s baseline OLIB Web URL.

Note in the URL above that the string you want to normalise can be included as param1. It will have to be URL-encoded, e.g. substituting spaces with %20.